Binary files are anything other than plain text files: programs, zip files (see section 3.3), picture or sound files, word processor documents, spreadsheets, etc. In general, unless the extension is .txt or .faq or the file is called read.me or readme, you should assume that it is a binary file. Downloading a text file as binary does not matter too much - the file should still be readable. However downloading a binary file as text will result in garbage.

See the Short tutorial on FTP for a graphic guide to using FTP.