Recurrence, backpropagation through time, and the gates that fixed its memory.
9h · 11 lessons · 0 challenges
Before this: Convolutional Networks
Sequences break the assumption every previous architecture made: that inputs are independent and the same size. Recurrence is the answer, and this module builds it, unrolls it, and differentiates through it.
Then the failure that defined a decade: gradients vanish across time steps, which is what GRU and LSTM gates exist to fix. Both are taken apart gate by gate.
An LSTM with one gate disabled. Say which one, from the failure alone.
Worth 400 XP.
