Skip to content
BENIBIX Learn

Mechanistic Interpretability

Also known as Mech Interp, Circuit Analysis

Looking at the parts in a model that each hold one idea.

Think of it like

It is like taking a toy apart. You name each part, and see how it works.

Example

Researchers find a feature in a model for the Golden Gate Bridge. When they turn it up by hand, the model brings the bridge into every answer.

Why it matters

If you can find the parts that stand for lying or harm, you can watch for them, or even switch them off.

Definition

Mechanistic interpretability takes a neural network apart into pieces people can understand. First it looks for features. A feature is a pattern in the model that stands for one idea, such as a city. Then it looks for circuits. A circuit is a few features linked up to do a small job, such as recalling a fact. The hope is to read how a model thinks, not guess it from its answers.
Features for Paris and France link into a small circuit that gives the answer.

Common mistake

Assuming one neuron means one idea. Most ideas are spread over many neurons, which is why special tools are needed.

Key takeaways

  1. 1
    Features. One idea each.
  2. 2
    Circuits. Features working together.
  3. 3
    Read, not guess. The reasoning itself.