Skip to content
BENIBIX Learn

Humanity's Last Exam (HLE)

AI📏 Evaluation & BenchmarksBenchmarks and Leaderboards

Also known as HLE, Humanity's Last Exam Benchmark

A test of 2,500 expert questions over 100+ subjects. It was built to stay hard.

Think of it like

It is like one huge test on every subject. It was made to be very, very hard.

Example

A lab posts its new model's HLE score next to last year's. Emma looks at the jump between the two, not at where the model ranks today.

Why it matters

When the old tests filled up, people needed a new one with room at the top. HLE was built to be that test.

Definition

Humanity's Last Exam is a benchmark of 2,500 hard questions across more than a hundred subjects. Nearly 1,000 experts from over 500 places around the world wrote them. Some questions come with a picture. Each one is multiple choice or a short answer that can be checked, and none can be found fast with a web search. It was made by the Center for AI Safety and Scale AI, because top models had passed 90% on older tests like MMLU.
This test is the new line near the bottom. It was made when the old one hit its top.

Common mistake

Reading a low score as a weak model. HLE was made to be hard, so low scores are normal.

Key takeaways

  1. 1
    Expert questions. Written by about 1,000 experts.
  2. 2
    Checkable answers. Multiple choice or short answer.
  3. 3
    Built to stay hard. Made when older tests filled up.