Skip to content
BENIBIX Learn

Eval Rubric

Also known as Evaluation Rubric, LLM Grading Rubric

A written checklist for grading AI answers. Each line says what earns a point.

Think of it like

It is like the marking sheet a teacher uses. Each line says what earns a mark.

Example

Emma's rubric for support replies has four lines: the right answer, a link to the policy, under 120 words, and a kind tone. Her judge model marks each line.

Why it matters

Without a rubric, graders judge by mood and scores drift. With one, a change in score means the answers changed.

Definition

An eval rubric is a written list of what makes an AI answer good. Each line names one thing to check, such as giving the right refund date or being polite. Each line gets points, or a pass or a fail. A person or a judge model goes down the list for every answer. The rubric turns a vague feeling into a score that two graders can agree on.

Example code

text
Score each line 0 or 1.
1. Gives the 30-day refund window.
2. Links to the refund policy page.
3. Is under 120 words.
4. Is polite and does not blame.

Scores: 1, 1, 0, 1. Total: 3 of 4.
A four-line rubric, and the marks one answer earned.
On the left, a rubric marks one answer line by line. It works even when no other answer exists.

Common mistake

Writing vague lines like 'is helpful'. Each line should be something two people would mark the same way.

Key takeaways

  1. 1
    One thing per line. Easy to mark yes or no.
  2. 2
    Shared by graders. People and judges mark alike.
  3. 3
    Written first. Decide what good is before grading.