CNN Part 3: Preprocessing, Initialization and Regularization

For the third reading group on the Stanford University Convolutional Neural Networks class, we went through the following slides:

Take-home message:

The most common regularization is L2 cross-validated. A good idea is to combine this with dropout with a reasonable value of p of 0.5, but this can be tuned. When implementing activation functions from scratch, it is important to do some gradient and sanity checks to validate your implementation.

Leave a Reply

Your email address will not be published. Required fields are marked *