GANs and Diffusion Models
Level 5 · Neural networks
Networks that make new things. How a GAN trains a maker against a judge, how diffusion turns noise into a picture, and what an autoencoder does.
Course contentsLesson 16 of 18
Level 1 · How a machine learns
Level 2 · Getting the data ready
Level 3 · Kinds of learning
Level 4 · Classic models
Level 5 · Neural networks
Level 6 · Better models, real tools
After this lesson you can
- Explain how a GAN trains
- Say how a diffusion model makes pictures
- Say what an autoencoder is good for
12 mincovers 3 glossary entries
Worth reading first: CNNs, RNNs and LSTMs
How networks learn to make new things
Some networks make new things instead of sorting them. A generative adversarial network, or GAN, trains two networks against each other. A diffusion model learns to turn random noise into a picture, step by step. GANs came out in 2014. Diffusion models took over most picture making after 2022.
In a GAN, one network, the generator, makes fakes, such as photos of faces. The other, the discriminator, tries to tell fake from real. Each one learns from the other. When the judge can no longer tell them apart, the fakes look real.
GANs are hard to train. If one side gets too far ahead, the other stops learning. A generator can also find a few fakes that fool the judge and make only those, over and over. These troubles are a big reason why most new picture tools moved to diffusion.
A diffusion model is trained by taking real pictures and adding random noise to them, a little at a time, until nothing is left but static. The model learns to undo one small step of that noise. A well-known 2020 version used 1,000 steps from picture to pure noise.
To make a new picture, it starts from pure noise. It removes a little at each step until a picture appears. A text request can steer each step, so 'a red fox reading a book' pulls the noise toward that picture. Stable Diffusion came out in August 2022. It ran this on a home computer with a good graphics card.
An autoencoder has two halves. The first squeezes the input into a small code. The second tries to rebuild the input from that code alone. Since the code is small, the network must keep only what matters most. Many diffusion models for pictures work inside such a squeezed code, to save effort.
An autoencoder can also spot odd data. Train one only on normal machine sounds. It rebuilds normal sounds well and odd ones badly, so a big rebuild error is a warning. People think these networks paste together the pictures they trained on. They learn patterns, though a picture seen many times can come back almost whole.
Worked example
Lucy makes game art
Lucy's game studio trains a GAN on 50,000 photos of stone walls. The generator learns to make new wall patterns that never existed.
For her poster art she uses a diffusion model instead. It turns static into 'a castle at dawn' in 50 steps.
The words for this lesson3
Check yourself
How does a diffusion model make a new picture?
Finished this lesson?
Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.