For the fifth reading group on the Stanford University Convolutional Neural Networks class, we went through the following slides:
You can backproject the content of a CNN in order to visualize the filters. You can fool a CNN by training an image (not specific to CNNs). The vast majority of the parameters of the CNN are at the fully connected layer, no matter how many convolution layers you have before. The CNN code can be highly discriminant.