Skip to content
BENIBIX Learn

GANs and Diffusion Models

Level 5 · Neural networks

Networks that make new things. How a GAN trains a maker against a judge, how diffusion turns noise into a picture, and what an autoencoder does.

Course contentsLesson 16 of 18

After this lesson you can

  • Explain how a GAN trains
  • Say how a diffusion model makes pictures
  • Say what an autoencoder is good for

12 mincovers 3 glossary entries

Worth reading first: CNNs, RNNs and LSTMs

How networks learn to make new things

Some networks make new things instead of sorting them. A generative adversarial network, or GAN, trains two networks against each other. A diffusion model learns to turn random noise into a picture, step by step. GANs came out in 2014. Diffusion models took over most picture making after 2022.

In a GAN, one network, the generator, makes fakes, such as photos of faces. The other, the discriminator, tries to tell fake from real. Each one learns from the other. When the judge can no longer tell them apart, the fakes look real.

GANs are hard to train. If one side gets too far ahead, the other stops learning. A generator can also find a few fakes that fool the judge and make only those, over and over. These troubles are a big reason why most new picture tools moved to diffusion.

A diffusion model is trained by taking real pictures and adding random noise to them, a little at a time, until nothing is left but static. The model learns to undo one small step of that noise. A well-known 2020 version used 1,000 steps from picture to pure noise.

To make a new picture, it starts from pure noise. It removes a little at each step until a picture appears. A text request can steer each step, so 'a red fox reading a book' pulls the noise toward that picture. Stable Diffusion came out in August 2022. It ran this on a home computer with a good graphics card.

An autoencoder has two halves. The first squeezes the input into a small code. The second tries to rebuild the input from that code alone. Since the code is small, the network must keep only what matters most. Many diffusion models for pictures work inside such a squeezed code, to save effort.

An autoencoder can also spot odd data. Train one only on normal machine sounds. It rebuilds normal sounds well and odd ones badly, so a big rebuild error is a warning. People think these networks paste together the pictures they trained on. They learn patterns, though a picture seen many times can come back almost whole.

The maker wins when the judge can no longer tell its fakes from real photos.

Worked example

Lucy makes game art

Lucy's game studio trains a GAN on 50,000 photos of stone walls. The generator learns to make new wall patterns that never existed.

For her poster art she uses a diffusion model instead. It turns static into 'a castle at dawn' in 50 steps.

The words for this lesson3

Check yourself

How does a diffusion model make a new picture?

Finished this lesson?

Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.