Skip to content
BENIBIX Learn

Stemming

AI🔤 NLP & LanguageText Processing

Also known as Word Stemming, Stemmer, Porter Stemmer

Cutting word endings off by rule, so 'jumps' and 'jumped' match.

Think of it like

Think of it like you chop the end off a word. Then 'jumping' and 'jumped' both turn into 'jump'.

Example

Emma searches her notes for 'running'. With stemming, notes that say 'runs' and 'run' show up too.

Why it matters

It helps search and word counts treat forms of one word as one, without much work.

Definition

Stemming cuts the endings off words so that different forms count as the same word. It follows simple rules, like dropping 'ing', 'ed' or 's'. That way 'connect', 'connected' and 'connecting' all become 'connect'. It is fast, but it does not check a dictionary, so the result is sometimes not a real word. A common English stemmer turns 'studies' into 'studi'.
Stemming, in the middle column, cuts endings by rule. It is quick, and it can leave a part that is not a word.

Common mistake

Expecting real words back. A stem like 'studi' is fine for matching but should never be shown to a reader.

Key takeaways

  1. 1
    Cuts endings. By simple rules.
  2. 2
    Fast. No dictionary needed.
  3. 3
    Rough. The result may not be a word.

Compare