Skip to content
BENIBIX Learn

LMArena (Chatbot Arena)

AI📏 Evaluation & BenchmarksBenchmarks and Leaderboards

Also known as Chatbot Arena, LMArena, LMSYS Chatbot Arena, LMArena Leaderboard

A site where people vote between two hidden AI models. The votes become a ranking.

Think of it like

It is like a blind taste test. You sip two drinks with no labels and say which one is better.

Example

Jack checks the LMArena leaderboard before picking a chat model. He also looks at the coding board, since that is what his team needs.

Why it matters

It measures what people prefer, not what a fixed quiz rewards. That makes it hard to train for in advance.

Definition

LMArena, first called Chatbot Arena, is a public site that ranks AI models by people's votes. A user asks a question and gets answers from two hidden models, side by side. The user picks the better one. Only after the vote are the names shown. Millions of votes are turned into a rating for each model, the way chess players are rated. It started in 2023 as a research project at UC Berkeley and took the name LMArena in 2025.
Two hidden models answer, the user picks one, and only then are the names shown.

Common mistake

Reading a top spot as best at everything. Votes reward answers people like, which can favor style and length over being right.

Key takeaways

  1. 1
    Two hidden models. Side by side.
  2. 2
    People vote. Then names show.
  3. 3
    Votes become ratings. Like chess.