# LMArena (Chatbot Arena)

> A site where people vote between two hidden AI models. The votes become a ranking.

**Field:** Artificial Intelligence  
**Category:** Evaluation & Benchmarks  
**Also known as:** Chatbot Arena, LMArena, LMSYS Chatbot Arena, LMArena Leaderboard

## Think of it like

It is like a blind taste test. You sip two drinks with no labels and say which one is better.

## Definition

LMArena, first called Chatbot Arena, is a public site that ranks AI models by people's votes. A user asks a question and gets answers from two hidden models, side by side. The user picks the better one. Only after the vote are the names shown. Millions of votes are turned into a rating for each model, the way chess players are rated. It started in 2023 as a research project at UC Berkeley and took the name LMArena in 2025.

## Example

Jack checks the LMArena leaderboard before picking a chat model. He also looks at the coding board, since that is what his team needs.

## Why it matters

It measures what people prefer, not what a fixed quiz rewards. That makes it hard to train for in advance.

## Common mistake

Reading a top spot as best at everything. Votes reward answers people like, which can favor style and length over being right.

## Three principles

- **Two hidden models.** Side by side.
- **People vote.** Then names show.
- **Votes become ratings.** Like chess.

## Related terms

- [AI Benchmark](https://learn.benibix.com/term/ai/ai-benchmark)
- [Human Evaluation](https://learn.benibix.com/term/ai/human-evaluation)
- [LLM-as-a-Judge](https://learn.benibix.com/term/ai/llm-as-a-judge)
- [Massive Multitask Language Understanding (MMLU)](https://learn.benibix.com/term/ai/mmlu)
- [AI Leaderboard](https://learn.benibix.com/term/ai/ai-leaderboard)
- [Pairwise Evaluation](https://learn.benibix.com/term/ai/pairwise-evaluation)
- [Elo Rating System](https://learn.benibix.com/term/general/elo-rating-system)

![One question goes to two hidden models, A and B. The user picks B as better, and only then are the names shown. The vote moves both ratings.](https://learn.benibix.com/diagrams/ev-chatbot-arena.svg)

---

Source: https://learn.benibix.com/term/ai/chatbot-arena
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
