📏Eval Set
Also known as Evaluation Set, Evaluation Dataset, Golden Set, Golden Dataset
A fixed list of test questions with good answers. You use it to grade an AI app.
Think of it like
It is like a box of practice questions you keep. You use the same box each time, so scores can be compared.
Example
Jack's eval set started with 50 questions. Each time a user found a wrong answer, he added that question, and a year later it had 400.
Why it matters
Scores can only be compared on the same questions. A fixed eval set makes every test fair.
Definition
Example code
json
{"input": "How do I reset my password?", "expected": "mentions the Forgot password link"}Common mistake
Key takeaways
- 1A fixed list. The same each time.
- 2Input and good answer. In each item.
- 3Grows from failures. Each bug a test.