Skip to content
BENIBIX Learn

ROC Curve and AUC

AI📏 Evaluation & BenchmarksClassification Metrics

Also known as ROC Curve, AUC, ROC AUC, Area Under the Curve, ROC

A curve that shows catches against false alarms. AUC sums it up in one number.

Think of it like

It is like a dial on a smoke alarm. Turn it one way to catch more fires, and it beeps at toast more.

Example

Emma's two models have AUCs of 0.81 and 0.93. The second one ranks real frauds above honest payments far more often, at any cutoff.

Why it matters

It judges a model without first choosing a cutoff. You can pick the cutoff later, to fit the cost of each mistake.

Definition

A ROC curve shows how a yes or no model trades catching cases against false alarms, at every setting of its cutoff. Most such models give a score, and a cutoff decides which scores count as yes. Each cutoff gives one point: the share of real yeses caught against the share of nos wrongly flagged. AUC, the area under the curve, sums it up in one number. An AUC of 0.5 is no better than a coin toss, and 1 is perfect.
The curve bends toward the top left. The straight line is a coin toss, and the shaded area is the AUC.

Common mistake

Trusting a high AUC when real yeses are very rare. Then look at precision too, since false alarms can still swamp the catches.

Key takeaways

  1. 1
    Every cutoff. One point each.
  2. 2
    Catches vs false alarms. The trade.
  3. 3
    AUC sums it up. 0.5 is chance.