# LLM Evaluation

> Tests that check if an AI gives good answers. Code, a model or people grade them.

**Field:** Artificial Intelligence  
**Category:** Evaluation & Benchmarks  
**Also known as:** Evals, LLM Evals, AI Evals, AI Evaluation

## Think of it like

It is like a driving test on many roads. One good drive does not prove you are a safe driver.

## Definition

LLM evaluation, often called evals, is how you check if a language model, or an app built on one, gives good answers. Most answers are free text, so there is no one right string to match. Teams build a set of test inputs and decide what a good answer looks like for each. The answers are then graded by code checks, by another model, or by people. Evals are run after each change to a prompt or model, so you can tell if things got better or worse.

## Example

Lucy changes the prompt of her support bot. Her evals run 300 saved questions through it, and the share of good answers drops from 88% to 81%, so she rolls the change back.

## Why it matters

Without evals, every prompt change is a guess. With them, you can see if a change helped or hurt.

## Common mistake

Trying a few questions by hand and calling it done. A handful of tries hides the cases where the model fails.

## Three principles

- **Free text answers.** No single right string.
- **Graded three ways.** Code, model, people.
- **Run after each change.** Catch drops.

## Related terms

- [Eval Set](https://learn.benibix.com/term/ai/eval-set)
- [LLM-as-a-Judge](https://learn.benibix.com/term/ai/llm-as-a-judge)
- [Human Evaluation](https://learn.benibix.com/term/ai/human-evaluation)
- [Large Language Model (LLM)](https://learn.benibix.com/term/ai/large-language-model)
- [Prompt Versioning](https://learn.benibix.com/term/development/prompt-versioning)
- [Faithfulness (AI)](https://learn.benibix.com/term/ai/faithfulness)
- [Eval Rubric](https://learn.benibix.com/term/ai/eval-rubric)

## Taught in

- [Model Selection and Testing](https://learn.benibix.com/learn/ai-working-with-ai/model-selection-and-testing): Prompt Engineering, 12/16

![A changed prompt runs through a set of saved questions. Each answer is graded by code checks, a judge model or people, and the share of good answers drops from 88% to 81%.](https://learn.benibix.com/diagrams/ev-llm-evaluation.svg)

---

Source: https://learn.benibix.com/term/ai/llm-evaluation
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
