📏Pairwise Evaluation
Also known as Pairwise Comparison (AI), Side-by-Side Evaluation, Head-to-Head Evaluation
Two AI answers are put side by side. The grader picks the better one.
Think of it like
It is like an eye test. The doctor asks which of two lenses looks clearer.
Example
Jack tests a new prompt against the old one on 300 questions. The judge likes the new answers 62% of the time, so he keeps it.
Why it matters
It shows if a change made the answers better, even when there is no answer key.
Definition
Common mistake
Key takeaways
- 1Two at once. Pick the better one, or a tie.
- 2Win rate. Votes add up to a share.
- 3Swap the order. It cancels the side bias.