CNNs, RNNs and LSTMs
Level 5 · Neural networks
Deep learning and the network designs behind it. Why photos suit a CNN, why things that come in order need an RNN, and how an LSTM remembers longer.
Course contentsLesson 15 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Say what makes learning deep
- Explain how a CNN reads a photo
- Tell an RNN from an LSTM
13 mincovers 4 glossary entries
Worth reading first: Gradient Descent and Backprop
Deep learning: networks built for photos and for order
Deep learning is machine learning with neural networks of many layers, often dozens or hundreds. Each layer builds on the one before, from simple patterns to complex ones. It needs lots of data and strong chips. Since 2012, it has driven most big steps in AI, in photos, speech and language.
A plain network treats every input on its own, which wastes effort on a photo. A convolutional neural network, or CNN, slides small filters over the picture. Each filter looks for one thing, like an edge or a patch of colour, wherever it is. Later layers join those into shapes and things. That is how it sees a cat.
Because the same filter is used across the whole photo, a CNN needs far fewer weights than a plain network would. It also spots a cat in any corner. An early CNN was reading handwritten numbers on bank cheques by 1998. The 2012 ImageNet winner was a CNN too.
Text and speech come in order, and the order matters. A recurrent neural network, or RNN, reads one piece at a time. After each piece it keeps a small memory and passes it to the next step. That lets 'see you' suggest 'tomorrow' on a phone keyboard.
A plain RNN forgets fast. Over a long sentence, the memory of the start fades to almost nothing, because the training signal shrinks at each step back. Long short-term memory, or LSTM, from 1997, adds small gates inside each step. They choose what to keep, what to forget and what to pass on.
LSTMs were the standard for speech and translation for many years. Since about 2018, a newer design that reads all the words at once has taken over most of those jobs. CNNs still run much of the photo work on phones and cameras, because they are small and fast.
People use deep learning and AI as if they meant the same thing. Deep learning is one part of machine learning, which is one part of AI. A spam filter built with Naive Bayes is AI and machine learning, but it is not deep learning.
The three designs side by side
Which one to pick
Worked example
Jack's farm app spots sick leaves
Jack's farm app uses a CNN trained on 30,000 leaf photos. Its early filters pick up spots and yellow edges.
Later layers join those into the signs of three diseases. It names the right disease for 9 leaves in 10, even with the leaf in a corner of the photo.
The words for this lesson4
Check yourself
Why does a CNN suit photos?
Finished this lesson?
Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.