💬LLM Sampling
Also known as Decoding, Token Sampling, Sampling
How a model picks the next token from its list of chances.
Think of it like
Think of it like a bag with lots of word cards. You pull one out, and likely words have more cards.
Example
Lucy asks for a story opener three times and gets three different first lines. Each time, sampling drew a different token at the start.
Why it matters
It explains why answers change from run to run. It also shows which settings make them steady or varied.
Definition
Common mistake
Key takeaways
- 1Pick from the chances. One token each step.
- 2Weighted draw. Likely tokens win more.
- 3Settings shape it. Temperature, top-k, top-p.