Skip to content
BENIBIX Learn

CNNs, RNNs and LSTMs

Level 5 · Neural networks

Deep learning and the network designs behind it. Why photos suit a CNN, why things that come in order need an RNN, and how an LSTM remembers longer.

Course contentsLesson 15 of 18

After this lesson you can

  • Say what makes learning deep
  • Explain how a CNN reads a photo
  • Tell an RNN from an LSTM

13 mincovers 4 glossary entries

Worth reading first: Gradient Descent and Backprop

Deep learning: networks built for photos and for order

Deep learning is machine learning with neural networks of many layers, often dozens or hundreds. Each layer builds on the one before, from simple patterns to complex ones. It needs lots of data and strong chips. Since 2012, it has driven most big steps in AI, in photos, speech and language.

A plain network treats every input on its own, which wastes effort on a photo. A convolutional neural network, or CNN, slides small filters over the picture. Each filter looks for one thing, like an edge or a patch of colour, wherever it is. Later layers join those into shapes and things. That is how it sees a cat.

Because the same filter is used across the whole photo, a CNN needs far fewer weights than a plain network would. It also spots a cat in any corner. An early CNN was reading handwritten numbers on bank cheques by 1998. The 2012 ImageNet winner was a CNN too.

Text and speech come in order, and the order matters. A recurrent neural network, or RNN, reads one piece at a time. After each piece it keeps a small memory and passes it to the next step. That lets 'see you' suggest 'tomorrow' on a phone keyboard.

A plain RNN forgets fast. Over a long sentence, the memory of the start fades to almost nothing, because the training signal shrinks at each step back. Long short-term memory, or LSTM, from 1997, adds small gates inside each step. They choose what to keep, what to forget and what to pass on.

LSTMs were the standard for speech and translation for many years. Since about 2018, a newer design that reads all the words at once has taken over most of those jobs. CNNs still run much of the photo work on phones and cameras, because they are small and fast.

People use deep learning and AI as if they meant the same thing. Deep learning is one part of machine learning, which is one part of AI. A spam filter built with Naive Bayes is AI and machine learning, but it is not deep learning.

The three designs side by side

DesignHow it readsGood at
CNNA grid, like a photoPhotos and video
RNNOne piece at a time, with a memoryShort runs of text or sound
LSTMOne piece at a time, with gatesLonger text and speech

Which one to pick

  1. 1

    Pick a CNN

    When the input is a grid, like a photo or a scan.

  2. 2

    Pick an RNN

    When the input is a short run of words or sounds, and a small model will do.

  3. 3

    Pick an LSTM

    When the order matters over long stretches, like a long voice note.

One small filter moves over the whole photo. It lights up where its edge is.

Worked example

Jack's farm app spots sick leaves

Jack's farm app uses a CNN trained on 30,000 leaf photos. Its early filters pick up spots and yellow edges.

Later layers join those into the signs of three diseases. It names the right disease for 9 leaves in 10, even with the leaf in a corner of the photo.

The words for this lesson4

Check yourself

Why does a CNN suit photos?

Finished this lesson?

Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.