Temperature vs Top-p Sampling
People often think temperature and top-p are two names for one setting. Temperature changes the shape of all the chances, making the top word stronger or the field flatter. Top-p does not reshape them; it cuts off the long tail of unlikely words. Most guides say to change one and leave the other alone.
LLM Temperature
A setting for how random the model's word picks are. Low is steady, high is wild.
Temperature is a setting that controls how random a model's word choices are. A low value, near 0, makes the most likely token win almost every time, so answers are steady and plain. A value of 1 keeps the chances as the model gave them. Higher values flatten the chances, so rare tokens get picked more and the text gets more varied, then strange. Most tools let you set it between 0 and 1 or 2.
Top-p Sampling
Picking only from the top tokens whose chances add up to a set share, like 0.9.
Top-p sampling limits the draw to the smallest group of likely tokens whose chances add up to p. With p set to 0.9, the model sorts tokens from most to least likely and keeps them until their chances reach 90 percent. The long tail of unlikely tokens is dropped. When the model is sure, few tokens are kept; when it is unsure, more are. It is also called nucleus sampling.
In one line
- LLM Temperature
- A setting for how random the model's word picks are. Low is steady, high is wild.
- Top-p Sampling
- Picking only from the top tokens whose chances add up to a set share, like 0.9.