RL and Transfer Learning
Level 3 · Kinds of learning
Three more ways a model can learn. RL learns from rewards. Transfer learning starts from an old model. Federated learning trains on phones, and the data stays put.
Course contentsLesson 7 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Explain how reinforcement learning uses rewards
- Say why transfer learning needs so little data
- Describe what federated learning keeps private
12 mincovers 3 glossary entries
Worth reading first: Supervised and Unsupervised ML
Learning from rewards, from other models and on your phone
Reinforcement learning, or RL, is learning by trial and error. A program called the agent takes an action, and the world hands back a reward or a penalty. Nobody shows it the right moves, only the score. In 2017, AlphaGo Zero learned Go this way, from games against itself, and beat the earlier AlphaGo 100 games to 0.
The hard part is that rewards often come late. A game is won or lost only at the end, after hundreds of moves. The agent has to work out which of those moves earned the win. That is why RL needs huge numbers of tries, and why it shines in games and simulators, where tries are cheap.
Transfer learning reuses a model trained on one job as the start for a new job. A model that learned from millions of everyday photos already knows edges, shapes and textures. You keep those layers and train only the last part on your own data. It needs far less data and time than starting from zero.
This is how most real projects start. A clinic with 500 X-rays could never train a photo model from scratch. Starting from a model that has seen millions of photos, it trains only the final layer and gets a useful result in hours. The catch is that the old job must be close enough to the new one to help.
Federated learning trains one model across many devices without collecting their data. Each phone trains a copy on its own data and sends back only the changes. A server blends all the changes into one better model and sends it back out. The raw data never leaves the phone.
A popular phone keyboard tried it in 2017, to improve the word it suggests next. The typing stayed on each phone. Hospitals use the same idea to learn from patient records that they may not pool. The cost is speed: training across thousands of phones is slow and messy.
People mix these up because they solve different problems. RL is about where the signal comes from: a reward, not a label. Transfer learning is about where the start comes from: an earlier model. Federated learning is about where the data stays: at home. One project can use all three.
Worked example
Lucy builds a bird app
Lucy wants an app that names 40 kinds of garden bird. She has only 2,000 photos, about 50 per bird.
She starts from a model already trained on millions of photos and trains its last layer again. In two hours it names 9 birds in 10 correctly.
The words for this lesson3
Check yourself
What does federated learning send back from each phone?
Finished this lesson?
Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.