Skip to content
BENIBIX Learn

Temperature vs Top-p Sampling

People often think temperature and top-p are two names for one setting. Temperature changes the shape of all the chances, making the top word stronger or the field flatter. Top-p does not reshape them; it cuts off the long tail of unlikely words. Most guides say to change one and leave the other alone.

AI💬 LLMs

LLM Temperature

A setting for how random the model's word picks are. Low is steady, high is wild.

Temperature is a setting that controls how random a model's word choices are. A low value, near 0, makes the most likely token win almost every time, so answers are steady and plain. A value of 1 keeps the chances as the model gave them. Higher values flatten the chances, so rare tokens get picked more and the text gets more varied, then strange. Most tools let you set it between 0 and 1 or 2.

Example
Emma sets temperature to 0 to pull dates out of invoices, so she gets the same result each time. For slogan ideas she uses 0.9 and gets more variety.
Why it matters
It is the first knob to turn when answers are too dull or too wild for the job.
Full entry
AI💬 LLMs

Top-p Sampling

Picking only from the top tokens whose chances add up to a set share, like 0.9.

Top-p sampling limits the draw to the smallest group of likely tokens whose chances add up to p. With p set to 0.9, the model sorts tokens from most to least likely and keeps them until their chances reach 90 percent. The long tail of unlikely tokens is dropped. When the model is sure, few tokens are kept; when it is unsure, more are. It is also called nucleus sampling.

Example
For 'The sky is', the top token 'blue' has 0.85 and 'clear' has 0.07. With top-p at 0.9, only those two are kept, and every odd word is out.
Why it matters
It cuts off nonsense words while still allowing variety. That is why many tools turn it on by default.
Full entry

In one line

LLM Temperature
A setting for how random the model's word picks are. Low is steady, high is wild.
Top-p Sampling
Picking only from the top tokens whose chances add up to a set share, like 0.9.