Review: Andrew NG’s Deep Learning Specialization
Part 1: Neural Networks and Deep Learning
Intro
Andrew Ng is known for being a great a teacher. His new deep learning specialization on Coursera is no exception. The course contains 5 different courses to help you master deep learning:
- Neural Networks and Deep Learning
- Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
- Structuring Machine Learning Projects
- Convolutional Neural Networks
- Sequence Models
This is the first of a series of reviews covering this coursera specialization. This will cover the first course “Neural Networks and Deep Learning”
The Good Stuff
Now that the intro is over with we get to the good stuff. What is the content of this course. We will skip talking about week one because it is just an introduction to what neural networks are and a little bit about how they work. I assume if you are taking courses on deep learning you already have some understanding of the basics!
Week 2-Neural Network Basics
Now we jump right into the real heavy lifting. If you are mathematically inclined and have followed other machine learning related courses then this first part will just be a review. For me it started off light by laying down the framework of logistic regression.
This is a really nice introduction to the concept of Logistic Regression. Andrew clearly explains cost/loss function and how to train a model using gradient decent to minimize the cost function. He then eloquently transitions the concept of a simple logistic regression into a one-perceptron neural network.
Week 3 — Shallow Neural Networks
Week 3 really gets into how neural networks work and how to work with them. He talks about vectorizing, activation functions, backprop and random initialization of the weights.
Vectorizing was a very nice concept that I personally did not see in my academic career when studying machine learning. It is the concept of using the built in python functions to do calculations on entire vectors or matrices instead of looping through examples. The mathematical representation of that is seen below.
Week 4-deep neural networks
Week four teaches you how to make an L-Layer NN. Therefore it is a generalization of everything taught in the previous weeks. Andrew jokes about how neural networks are not really like the brain. Generalizing the functions I had created for a shallow neural network to L-Layer NN proved to be pretty challenging.
Andrew wraps up this week with explaining that building a neural network is very empirical and iterative process. Also that the intuitions from one field such as computer vision do not normally carry over to NLP so you just have to test out different hyperparameters.
Final Thoughts
After completing this course and all the of assignments you will know how to implement a L-Layer NN from scratch in python. The good thing though is that there are many libraries for this, such as tensor flow and Microsoft Cognitive Toolkit (CNTK). But understanding whats under the hood is crucial for building good solutions. I recommend this course for anyone with an advanced understanding of “traditional” machine learning and would like to expand it to deep learning. It is not a hard transition but Andrew gives extremely valuable insight. Stay tuned for the reviews about the rest of the specialization.
