# Bilingual Evaluation Understudy (BLEU)

> A score for machine translation. It counts word runs that match a person's translation.

**Field:** Artificial Intelligence  
**Category:** Evaluation & Benchmarks  
**Also known as:** BLEU Score, BLEU, BLEU Metric

## Think of it like

It is like checking a copy of a poem against the real one. You count how many word groups match.

## Definition

BLEU is a score for machine translation. It compares the computer's sentence with one or more translations made by people. It counts how many short runs of words match, from single words up to runs of four. It also takes points off when the output is too short. The score goes from 0 to 1 and is often shown out of 100. It came out in 2002 and is still used, because it is fast and free to run.

## Example

Emma tests two English to German models on 1,000 sentences with human translations. One scores 31 BLEU and the other 27, so she reads the first one's output more closely.

## Why it matters

It lets teams compare translation models in seconds. No one has to read every line.

## Common mistake

Treating BLEU as a measure of meaning. A good translation that picks other words than the reference can still score low.

## Three principles

- **Word runs.** Matches from one to four words long.
- **Output side.** How much of the output matches.
- **Short penalty.** Too short loses points.

## Related terms

- [Machine Translation](https://learn.benibix.com/term/ai/machine-translation)
- [N-gram](https://learn.benibix.com/term/ai/n-gram)
- [ROUGE Score](https://learn.benibix.com/term/ai/rouge-score)
- [Ground Truth](https://learn.benibix.com/term/ai/ground-truth)
- [Human Evaluation](https://learn.benibix.com/term/ai/human-evaluation)

## Told apart from

- [BLEU vs ROUGE](https://learn.benibix.com/compare/bleu-vs-rouge): People think BLEU and ROUGE are one score with two names, since both count shared words. They count from opposite sides. BLEU asks how much of the model's text is in the human version. ROUGE asks how much of the human version made it into the model's text. BLEU grew up with translation and ROUGE with summaries.

![A model's sentence, 'the cat sits on a mat', above a person's, 'the cat is on the mat', with the shared words marked. A BLEU note reads from the model's side and a ROUGE note from the person's side.](https://learn.benibix.com/diagrams/ai5-text-overlap.svg)

---

Source: https://learn.benibix.com/term/ai/bleu-score
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
