# Graduate-Level Google-Proof Q&A (GPQA)

> A test of 448 hard science questions. A web search does not easily answer them.

**Field:** Artificial Intelligence  
**Category:** Evaluation & Benchmarks  
**Also known as:** GPQA, GPQA Diamond, GPQA Benchmark

## Think of it like

It is like a quiz so hard that looking things up does not help much.

## Definition

GPQA is a benchmark of 448 multiple-choice questions in biology, physics and chemistry. Experts with PhDs in those fields wrote them. They are called Google-proof because a web search does not easily give the answer. In the 2023 paper, experts in the field got 65% right. Skilled people from other fields got only 34%, even with the web and over 30 minutes per question on average. A harder subset is called GPQA Diamond.

## Example

A model's release notes list its GPQA Diamond score. Jack reads it as a sign of deep science knowledge, not as proof it can run his lab.

## Why it matters

Easy quizzes stopped telling top models apart. GPQA asks questions where only real experts do well.

## Common mistake

Comparing scores from different sets. A score on GPQA Diamond and a score on the full set are not the same test.

## Three principles

- **Written by experts.** PhDs wrote every question.
- **Google-proof.** Search does not give it away.
- **Three sciences.** Biology, physics and chemistry.

## Related terms

- [AI Benchmark](https://learn.benibix.com/term/ai/ai-benchmark)
- [Massive Multitask Language Understanding (MMLU)](https://learn.benibix.com/term/ai/mmlu)
- [Humanity's Last Exam (HLE)](https://learn.benibix.com/term/ai/humanitys-last-exam)
- [Benchmark Saturation](https://learn.benibix.com/term/ai/benchmark-saturation)
- [Reasoning Model](https://learn.benibix.com/term/ai/reasoning-model)

![Scores over time for two tests. The old test's line climbs and flattens just under 100%. A new, harder test starts near the bottom with plenty of room above it.](https://learn.benibix.com/diagrams/ai5-benchmark-saturation.svg)

---

Source: https://learn.benibix.com/term/ai/gpqa
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
