Skip to content
BENIBIX Learn

Corpus

Also known as Text Corpus, Corpora

A big set of text that a language model learns from.

Think of it like

Think of it like a huge pile of books. A computer reads them all to learn words.

Example

Jack wants a model that reads legal letters. He builds a corpus of 50,000 old letters from his firm and trains on it.

Why it matters

A model can only learn what its corpus holds. Gaps and slants in the text become gaps and slants in the model.

Definition

A corpus is a large set of text gathered for a language tool to learn from or be tested on. It can be books, news stories, web pages, chat logs or a mix. A small one has a few thousand pages, and the ones behind big chat models hold a large part of the public web. What is in the corpus shapes what the model knows and how it writes. The plural is corpora.
Each box is one document. Together, all the documents a tool learns from are its corpus.

Common mistake

Thinking more text is always better. A corpus full of spam or repeats can teach the model bad habits.

Key takeaways

  1. 1
    A lot of text. Gathered for one purpose.
  2. 2
    Shapes the model. It learns what is there.
  3. 3
    Quality counts. Not just size.