💬Top-k Sampling
Also known as Top-k
Picking from only the top few tokens, like the best 40.
Think of it like
Think of it like you can only pick one of the top three cakes.
Example
Jack sets top-k to 3. For 'I drink', only 'tea', 'coffee' and 'water' can be picked; 'lava' is never on the list.
Why it matters
It is the simplest guard against a model picking a strange word now and then.
Definition
Common mistake
Key takeaways
- 1Keep the top k. Like 40 tokens.
- 2Fixed size. Same count every step.
- 3Simple guard. Cuts rare, odd words.