Skip to content
BENIBIX Learn

Top-p Sampling

AI💬 LLMsText Generation

Also known as Nucleus Sampling, Top-p

Picking only from the top tokens whose chances add up to a set share, like 0.9.

Think of it like

Think of it like you only pick from the best snacks. Stop once they fill most of the plate.

Example

For 'The sky is', the top token 'blue' has 0.85 and 'clear' has 0.07. With top-p at 0.9, only those two are kept, and every odd word is out.

Why it matters

It cuts off nonsense words while still allowing variety. That is why many tools turn it on by default.

Definition

Top-p sampling limits the draw to the smallest group of likely tokens whose chances add up to p. With p set to 0.9, the model sorts tokens from most to least likely and keeps them until their chances reach 90 percent. The long tail of unlikely tokens is dropped. When the model is sure, few tokens are kept; when it is unsure, more are. It is also called nucleus sampling.
Top-p, on the right, keeps bars until their total reaches 0.9, however many that takes.

Common mistake

Changing top-p and temperature together and then not knowing which one fixed the problem. Change one at a time.

Key takeaways

  1. 1
    Sort by chance. Most likely first.
  2. 2
    Keep until p. Like 90 percent in total.
  3. 3
    Size adapts. Few when sure, more when not.

Compare

Learn it properly