The custom.css file is a special stylesheet included in themes that allows you to copy styles from the main stylesheet or enter your own custom styles and is located under Appearance → Editor on the lower-right. This allows you to change the theme’s styles and colors without modifying the theme files, which get overwritten with theme updates. Most themes also include a Custom CSS area in the Theme Options. Using the Theme Options panel to store custom css is  recommended, since CSS saved there is stored in your database and will persist if you lose the custom.css file or have to reinstall the theme.

See also Custom CSS