📏Humanity's Last Exam (HLE)
Also known as HLE, Humanity's Last Exam Benchmark
A test of 2,500 expert questions over 100+ subjects. It was built to stay hard.
Think of it like
It is like one huge test on every subject. It was made to be very, very hard.
Example
A lab posts its new model's HLE score next to last year's. Emma looks at the jump between the two, not at where the model ranks today.
Why it matters
When the old tests filled up, people needed a new one with room at the top. HLE was built to be that test.
Definition
Common mistake
Key takeaways
- 1Expert questions. Written by about 1,000 experts.
- 2Checkable answers. Multiple choice or short answer.
- 3Built to stay hard. Made when older tests filled up.