# Term Frequency-Inverse Document Frequency (TF-IDF)

> A word score. It is high when a word is common in one page but rare in the rest.

**Field:** Artificial Intelligence  
**Category:** NLP & Language  
**Also known as:** TF-IDF, TFIDF

## Think of it like

Think of it like a word you see a lot in one book but not in the rest. It tells you what that book is about.

## Definition

TF-IDF scores how important a word is to one document in a set. The first part, term frequency, counts how often the word appears in that document. The second part, inverse document frequency, lowers the score of words found in many documents. So 'the' scores near zero. But 'volcano' in a page about volcanoes scores high. Search engines used it for years to rank results.

## Example

Jack has 1,000 news stories. 'Said' is in nearly all of them, so it scores low everywhere. 'Penguin' is in 3, so it scores high in those 3.

## Why it matters

It finds the words that make a document stand out, which is the core of simple search.

## Common mistake

Thinking a high score means the word is used most. A word used a lot everywhere still scores low.

## Three principles

- **Term frequency.** How often in this document.
- **Inverse document frequency.** Lower if in many documents.
- **Finds standout words.** What makes a page different.

## Related terms

- [Bag of Words](https://learn.benibix.com/term/ai/bag-of-words)
- [Corpus](https://learn.benibix.com/term/ai/corpus)
- [Stop Words](https://learn.benibix.com/term/ai/stop-words)

![Four documents in a row. A common word appears in all four and gets a low score. A rare word appears often in one document only and gets a high score there.](https://learn.benibix.com/diagrams/nlp-tf-idf.svg)

---

Source: https://learn.benibix.com/term/ai/tf-idf
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
