Splits, the bias-variance recipe, regularization, initialization and the gradient check that finds the bug.
9h · 12 lessons · 1 challenge
Before this: Neural Networks & Deep Learning
A network that will not train is the normal case, and this module is the set of things to do about it. Train, dev and test sets at deep learning scale, then the recipe that turns a bias or variance diagnosis into an action.
Then the mechanisms: L2 and dropout, input normalization, the vanishing and exploding gradients that motivate careful initialization, and gradient checking to prove the backward pass is right.
Five runs that are not converging. One diagnosis and one fix each, with the loss curve as your only evidence.
Worth 450 XP.
