# Masked Language Model

> A model trained by hiding words in text and guessing them back. BERT learned this way.

**Field:** Artificial Intelligence  
**Category:** NLP & Language  
**Also known as:** Masked Language Modeling, Masked LM

## Think of it like

It is like a quiz with blanks. Some words are hidden, and you guess them.

## Definition

A masked language model learns by guessing hidden words. During training, some words in a sentence are covered up with a mask. The model sees the rest of the sentence, on both sides, and must guess the covered words. BERT trained this way, hiding 15% of the tokens in each piece of text. It needs no labels, because the text itself holds the answers. This makes a model that reads well, not one that writes.

## Example

The model sees 'The cat sat on the [MASK].' It gives 'mat' a high chance and 'floor' a lower one.

## Why it matters

It lets a model learn from huge amounts of plain text with no human labels. That made strong reading models cheap to build.

## Common mistake

Mixing it up with next-word prediction. Chat models guess the next word from the left only, while a masked model uses both sides.

## Three principles

- **Hide and guess.** Cover words, then predict them.
- **Both sides.** Uses the words before and after.
- **No labels.** The text is its own answer key.

## Related terms

- [BERT (Language Model)](https://learn.benibix.com/term/ai/bert)
- [Next-Token Prediction](https://learn.benibix.com/term/ai/next-token-prediction)
- [Self-Supervised Learning](https://learn.benibix.com/term/ai/self-supervised-learning)
- [Pretraining](https://learn.benibix.com/term/ai/pretraining)
- [Language Model](https://learn.benibix.com/term/ai/language-model)
- [Token (AI)](https://learn.benibix.com/term/ai/token)

## Told apart from

- [Masked Language Model vs Next-Token Prediction](https://learn.benibix.com/compare/masked-language-model-vs-next-token-prediction): People think every language model learns by guessing the next word. Many do, but not all. A masked model hides words in the middle and guesses them from both sides, which makes it good at reading. A next-word model sees only the words before the gap, which is what lets it write. BERT learned the first way, and chat models learn the second.

![The sentence 'The cat sat on the [MASK] by the door' with arrows reaching the mask from words on both sides. Below it, guesses with bars: mat is long, floor is shorter, roof is very short.](https://learn.benibix.com/diagrams/ai5-masked-word.svg)

---

Source: https://learn.benibix.com/term/ai/masked-language-model
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
