Skip to content
BENIBIX Learn

BLEU vs ROUGE

People think BLEU and ROUGE are one score with two names, since both count shared words. They count from opposite sides. BLEU asks how much of the model's text is in the human version. ROUGE asks how much of the human version made it into the model's text. BLEU grew up with translation and ROUGE with summaries.

AI📏 Evaluation & Benchmarks

Bilingual Evaluation Understudy (BLEU)

A score for machine translation. It counts word runs that match a person's translation.

BLEU is a score for machine translation. It compares the computer's sentence with one or more translations made by people. It counts how many short runs of words match, from single words up to runs of four. It also takes points off when the output is too short. The score goes from 0 to 1 and is often shown out of 100. It came out in 2002 and is still used, because it is fast and free to run.

Example
Emma tests two English to German models on 1,000 sentences with human translations. One scores 31 BLEU and the other 27, so she reads the first one's output more closely.
Why it matters
It lets teams compare translation models in seconds. No one has to read every line.
Full entry
AI📏 Evaluation & Benchmarks

ROUGE Score

A set of scores for summaries. They check how much of a person's summary was covered.

ROUGE is a set of scores for summaries made by a computer. It compares the summary with summaries that people wrote. It counts how many words and short word runs from the human summary also show up in the computer's one. The focus is on recall, meaning how much of the human version was covered. ROUGE-N counts runs of N words, and ROUGE-L looks for the longest run of words in the same order. It was published in 2004.

Example
Jack's app sums up meeting notes. He compares 200 of its summaries with ones his team wrote, and ROUGE-L goes from 0.38 to 0.45 after a prompt change.
Why it matters
Summaries are slow for people to grade. ROUGE gives a quick first check across hundreds of them.
Full entry

In one line

Bilingual Evaluation Understudy (BLEU)
A score for machine translation. It counts word runs that match a person's translation.
ROUGE Score
A set of scores for summaries. They check how much of a person's summary was covered.