# Next-Token Prediction

> Guessing the next token from the text so far. Repeat it and the model writes.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Next-Word Prediction, Autoregressive Generation

## Think of it like

Think of it like a game. You add one word to a story, then pass it on.

## Definition

Next-token prediction is the one job a large language model is trained to do. It looks at the text so far. Then it gives a chance for every token that could come next. To write, the model picks one token, adds it to the text, and guesses again. A long answer is just this loop run hundreds of times. In training, the model is scored on how well it guessed each real next token.

## Example

Emma types 'The capital of Italy is'. The model gives ' Rome' a chance of 0.97, picks it, then predicts what comes after ' Rome'.

## Why it matters

It explains how these models write and why they can drift. Each token depends only on the ones before it.

## Common mistake

Thinking the model plans the whole answer first. It commits to one token at a time and cannot go back and change one.

## Three principles

- **One token at a time.** A chance for each option.
- **Pick, add, repeat.** That loop writes the text.
- **The training goal.** Scored on each real next token.

## Related terms

- [Language Model](https://learn.benibix.com/term/ai/language-model)
- [Large Language Model (LLM)](https://learn.benibix.com/term/ai/large-language-model)
- [Token (AI)](https://learn.benibix.com/term/ai/token)
- [Probability Distribution](https://learn.benibix.com/term/general/probability-distribution)
- [LLM Sampling](https://learn.benibix.com/term/ai/sampling)
- [Masked Language Model](https://learn.benibix.com/term/ai/masked-language-model)
- [Text Generation](https://learn.benibix.com/term/ai/text-generation)

## Told apart from

- [Masked Language Model vs Next-Token Prediction](https://learn.benibix.com/compare/masked-language-model-vs-next-token-prediction): People think every language model learns by guessing the next word. Many do, but not all. A masked model hides words in the middle and guesses them from both sides, which makes it good at reading. A next-word model sees only the words before the gap, which is what lets it write. BERT learned the first way, and chat models learn the second.

## Taught in

- [Tokens and Tokenizers](https://learn.benibix.com/learn/ai-llms/tokens-and-tokenizers): LLMs, 2/15
- [Predictive Text on Phones](https://learn.benibix.com/learn/ai-use-case/predictive-text-on-phones): AI Assistants and Chatbots, 12/13

![A loop. The text so far goes into the model, the model gives a chance for each next token, one token is picked and added to the text, and the loop starts again.](https://learn.benibix.com/diagrams/llm-next-token.svg)

---

Source: https://learn.benibix.com/term/ai/next-token-prediction
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
