BLEU vs ROUGE
People think BLEU and ROUGE are one score with two names, since both count shared words. They count from opposite sides. BLEU asks how much of the model's text is in the human version. ROUGE asks how much of the human version made it into the model's text. BLEU grew up with translation and ROUGE with summaries.
Bilingual Evaluation Understudy (BLEU)
A score for machine translation. It counts word runs that match a person's translation.
BLEU is a score for machine translation. It compares the computer's sentence with one or more translations made by people. It counts how many short runs of words match, from single words up to runs of four. It also takes points off when the output is too short. The score goes from 0 to 1 and is often shown out of 100. It came out in 2002 and is still used, because it is fast and free to run.
ROUGE Score
A set of scores for summaries. They check how much of a person's summary was covered.
ROUGE is a set of scores for summaries made by a computer. It compares the summary with summaries that people wrote. It counts how many words and short word runs from the human summary also show up in the computer's one. The focus is on recall, meaning how much of the human version was covered. ROUGE-N counts runs of N words, and ROUGE-L looks for the longest run of words in the same order. It was published in 2004.
In one line
- Bilingual Evaluation Understudy (BLEU)
- A score for machine translation. It counts word runs that match a person's translation.
- ROUGE Score
- A set of scores for summaries. They check how much of a person's summary was covered.