Skip to content
BENIBIX Learn

Interpretability (AI)

Also known as AI Interpretability, Model Interpretability

Looking inside an AI to see how it gets its answers.

Think of it like

It is like opening a clock. You look at the gears to see how the hands move.

Example

Emma's model sorts loan forms. A tool that looks inside shows it leans hard on where the person lives. So her team checks if that hides a bias.

Why it matters

You cannot fully trust what you cannot see into. Looking inside can catch problems that tests of the answers miss.

Definition

Interpretability is the study of how an AI model gets its answers, by looking inside it. A big model is billions of numbers, and no one wrote down what each one does. Tools trace which parts of the model were most active for an answer, and what those parts stand for. This can show if a model uses a shortcut, hides a goal, or leans on a bias. It is still a young field, and most of a big model is not yet understood.
A closed box, then the same box opened to show what lit up.

Common mistake

Thinking the model's own reasons are the same thing. A model can give a neat reason that is not what drove its answer.

Key takeaways

  1. 1
    Look inside. Not just at answers.
  2. 2
    Trace the parts. That were active.
  3. 3
    Young field. Much still unknown.

Compare