Skip to content
BENIBIX Learn

Greedy Decoding

AI💬 LLMsText Generation

Also known as Greedy Search (LLM), Argmax Decoding

Always picking the one most likely next word. Fast, but dull.

Think of it like

It is like you always grab the biggest slice of cake. You never look at the rest.

Example

Asked to finish 'The cat sat on the', a greedy model always says 'mat'. That word had the top chance.

Why it matters

It is the plain starting point that other ways of picking words are compared with.

Definition

Greedy decoding is the simplest way for a language model to pick its words. At each step it takes the one most likely next token and never looks back. It is fast and gives nearly the same answer each time. But the best word now can lead to a worse sentence later, and long greedy answers often repeat themselves. Setting the temperature to 0 is the usual way to ask a model for it.
Greedy decoding takes the top word at each step. That is the left path in this tree.

Common mistake

Expecting the best whole answer from it. The top word at each step does not always make the top sentence.

Key takeaways

  1. 1
    Top token. At every single step.
  2. 2
    No looking back. Each choice is final.
  3. 3
    Nearly the same. Much like temperature 0.

Learn it properly