📏LLM Evaluation
Also known as Evals, LLM Evals, AI Evals, AI Evaluation
Tests that check if an AI gives good answers. Code, a model or people grade them.
Think of it like
It is like a driving test on many roads. One good drive does not prove you are a safe driver.
Example
Lucy changes the prompt of her support bot. Her evals run 300 saved questions through it, and the share of good answers drops from 88% to 81%, so she rolls the change back.
Why it matters
Without evals, every prompt change is a guess. With them, you can see if a change helped or hurt.
Definition
Common mistake
Key takeaways
- 1Free text answers. No single right string.
- 2Graded three ways. Code, model, people.
- 3Run after each change. Catch drops.