Skip to content
BENIBIX Learn

Dangerous Capability Evaluation

Also known as Dangerous Capability Eval, Frontier Risk Evaluation, Capability Evaluation (Safety)

A test to see if an AI model could do great harm.

Think of it like

It is like a crash test for a new car. You find the danger before anyone drives it.

Example

Before release, a lab asks experts to try to plan a cyber attack with a new model. It does better than the last model. So the lab adds more safeguards first.

Why it matters

A lab cannot guard against a power it never tested for.

Definition

A dangerous capability evaluation is a test to see if an AI model can do great harm. Teams test if it could help make germ or chemical weapons. They test if it could break into computers, or act on its own without control. Experts try hard to get the model to do these things, most often before release. The results are compared with set danger lines. If the model crosses one, stronger safeguards must be in place before it ships.
The evaluation is the test step. Its score decides which gate the model must pass.

Common mistake

Testing only with simple prompts. A real attacker will try much harder, so the tests must too.

Key takeaways

  1. 1
    Test the worst. Weapons, hacking and escape.
  2. 2
    Before release. Find it before users do.
  3. 3
    Danger lines. Crossing one means more safeguards.

Compare