Skip to content
BENIBIX Learn

Self-Supervised Learning

Also known as Self-Supervision

The data makes its own answers. The model guesses a hidden word, then checks.

Think of it like

Think of it like a word game. Hide one word in a book, then guess it. The book has the answer.

Example

A model reads the sentence 'The cat sat on the mat' with mat hidden. It guesses rug, checks against mat, and learns a little. It does this billions of times.

Why it matters

It is how today's big language and image models learn. They read huge piles of data that no one labelled.

Definition

Self-supervised learning makes its own labels from raw data. It hides part of an example and asks the model to guess the hidden part. The real hidden part is the answer, so no person has to label anything. Large language models learn this way, by guessing the next word in huge amounts of text. It turns plain data into training data for free.
The hidden word is the label. It came from the sentence itself, so nobody had to write it.

Common mistake

Calling it unsupervised. It does learn from answers. The answers just come from the data itself, not from people.

Key takeaways

  1. 1
    Hide and guess. Cover part of the data.
  2. 2
    Data is the answer. No people needed to label.
  3. 3
    Scales up. Works on huge amounts of text.

Compare

Learn it properly