📈Dropout
Also known as Dropout Regularization
Random neurons switch off in training. The rest learn to cope.
Think of it like
Think of it like a team where some players sit out each game. Then all of them learn every job.
Example
Jack adds dropout of 0.3 to his network. At each step, 3 in 10 neurons sit out. His validation score goes up by 5 points.
Why it matters
It is cheap, simple and works on many networks. It was a big step in making deep networks practical.
Definition
Common mistake
Key takeaways
- 1Switch off at random. Some neurons sit out.
- 2Spread the work. No neuron is leaned on too hard.
- 3Training only. All neurons back on in use.