🔤Masked Language Model
Also known as Masked Language Modeling, Masked LM
A model trained by hiding words in text and guessing them back. BERT learned this way.
Think of it like
It is like a quiz with blanks. Some words are hidden, and you guess them.
Example
The model sees 'The cat sat on the [MASK].' It gives 'mat' a high chance and 'floor' a lower one.
Why it matters
It lets a model learn from huge amounts of plain text with no human labels. That made strong reading models cheap to build.
Definition
Common mistake
Key takeaways
- 1Hide and guess. Cover words, then predict them.
- 2Both sides. Uses the words before and after.
- 3No labels. The text is its own answer key.