The difference is in the location where the preprocessor searches for the included file.
#include "filename" the preprocessor searches in the same directory as the file containing the directive. This method is normally used to include programmer-defined header files.
#include <filename> the preprocessor searches in an implementation dependent manner, normally in search directories pre-designated by the compiler/IDE. This method is normally used to include standard library header files.