# Top-k Sampling

> Picking from only the top few tokens, like the best 40.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Top-k

## Think of it like

Think of it like you can only pick one of the top three cakes.

## Definition

Top-k sampling limits the draw to the k most likely tokens and ignores the rest. With k set to 40, only the 40 top tokens can be picked at each step. It is simple and cuts off rare, odd tokens. But k is fixed: when the model is sure, 40 may let in poor choices, and when it is unsure, 40 may be too few. That is why top-p is often used instead or as well.

## Example

Jack sets top-k to 3. For 'I drink', only 'tea', 'coffee' and 'water' can be picked; 'lava' is never on the list.

## Why it matters

It is the simplest guard against a model picking a strange word now and then.

## Common mistake

Setting k very high and thinking it does anything. With k at 1,000, almost no tokens are cut.

## Three principles

- **Keep the top k.** Like 40 tokens.
- **Fixed size.** Same count every step.
- **Simple guard.** Cuts rare, odd words.

## Related terms

- [Top-p Sampling](https://learn.benibix.com/term/ai/top-p-sampling)
- [LLM Temperature](https://learn.benibix.com/term/ai/temperature)
- [LLM Sampling](https://learn.benibix.com/term/ai/sampling)
- [Top-k Retrieval](https://learn.benibix.com/term/ai/top-k-retrieval)

## Told apart from

- [Top-k vs Top-p Sampling](https://learn.benibix.com/compare/top-k-vs-top-p-sampling): People think top-k and top-p are the same cut, and they are not. Top-k always keeps a set number of tokens, such as 40. Top-p keeps tokens until their chances add up to a share, such as 0.9. So top-p keeps a few tokens when the model is sure, and many when it is not. Top-k keeps 40 either way.
- [Top-k Retrieval vs Top-k Sampling](https://learn.benibix.com/compare/top-k-retrieval-vs-top-k-sampling): People see top-k in two settings and think it is one knob, and it is not. In retrieval, k is how many matching chunks the search hands to the model. In sampling, k is how many likely next words the model picks from as it writes. One decides what the model reads. The other shapes how it writes.

## Taught in

- [Temperature and Sampling](https://learn.benibix.com/learn/ai-llms/temperature-and-sampling): LLMs, 3/15

![The same sorted bars of next-word chances, twice. Top-k keeps the first three bars. Top-p keeps bars from the top until their chances add up to 0.9.](https://learn.benibix.com/diagrams/llm-top-k-top-p.svg)

---

Source: https://learn.benibix.com/term/ai/top-k-sampling
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
