# Speculative Decoding

> A small model guesses words ahead, and the big model checks them.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Speculative Sampling, Draft Model Decoding

## Think of it like

It is like a helper who writes a few words ahead. The teacher reads them and keeps the right ones.

## Definition

Speculative decoding makes a large language model write faster without changing what it writes. A small, fast draft model guesses the next few tokens. The big model checks all of those guesses in one pass, which takes about as long as making one token. It keeps the guesses it agrees with and fixes the first one it does not. A 2022 paper showed text coming out 2 to 3 times faster, word for word the same.

## Example

A chat service pairs its big model with a tiny one. On easy text, the big model accepts most guesses, so the answer streams out faster.

## Why it matters

Big models are slow because they make one token at a time. This lets them confirm several in one go.

## Common mistake

Thinking the small model's words slip into the answer. The big model checks every token, so the text is the same.

## Three principles

- **Draft and check.** Small guesses, big decides.
- **Same output.** Only the speed changes.
- **Easy text gains most.** More guesses are kept.

## Related terms

- [Next-Token Prediction](https://learn.benibix.com/term/ai/next-token-prediction)
- [KV Cache](https://learn.benibix.com/term/ai/kv-cache)
- [Inference (AI)](https://learn.benibix.com/term/ai/inference)
- [Knowledge Distillation](https://learn.benibix.com/term/ai/knowledge-distillation)
- [LLM Streaming](https://learn.benibix.com/term/ai/streaming-response)

## Taught in

- [LLM Speed and Streaming](https://learn.benibix.com/learn/ai-llms/llm-speed-and-streaming): LLMs, 13/15

![A small draft model writes four guessed words. The big model checks all four in one pass, ticks the first three and replaces the fourth with its own word.](https://learn.benibix.com/diagrams/ai9-speculative-decoding.svg)

---

Source: https://learn.benibix.com/term/ai/speculative-decoding
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
