Skip to content
BENIBIX Learn

Emergent Abilities

AI💬 LLMsLanguage Model Training

Also known as Emergent Ability, Emergent Abilities of LLMs, Emergence (LLM)

Skills that seem to pop up all at once when a model gets big.

Think of it like

It is like water that gets colder and colder. Nothing seems to change, then all at once it is ice.

Example

A small model gets every three-step sum wrong. A model many times its size gets most of them right. That kind of jump was called emergent.

Why it matters

If skills can jump, a bigger model may surprise the people who made it. That is one reason labs test each new size with care.

Definition

Emergent abilities are skills that a small language model does not have but a larger one suddenly does. A 2022 paper gave them the name, for tasks like sums in several steps where scores stayed near chance and then jumped as models grew. The jump could not be guessed from the smaller models. A 2023 paper argued that many jumps come from how the test is marked. With a mark that gives part credit, the same skills often grow slowly and smoothly.
Marked pass or fail, the skill seems to appear at once. With part credit, it grows step by step.

Common mistake

Taking every jump at face value. A pass or fail mark can hide steady progress underneath.

Key takeaways

  1. 1
    A sudden jump. Missing in small models, there in large ones.
  2. 2
    Hard to predict. Small models give no warning.
  3. 3
    Maybe the marking. Part credit can show smooth growth.

Learn it properly