Skip to content
BENIBIX Learn

Chunking (RAG)

Also known as Document Chunking, Text Splitting

Cutting a long file into small bits. Then search can find the right bit.

Think of it like

Think of it like cutting a cake into slices. Each person takes just the slice they want.

Example

Emma's 80-page manual becomes 400 chunks of about 300 tokens each. A question about returns now finds the two chunks on returns, not the whole manual.

Why it matters

Chunk size and borders decide what the search can find. Bad chunking breaks RAG before the model even runs.

Definition

Chunking is cutting long documents into small pieces, called chunks, before they are stored for search. A search then finds the few chunks that answer a question, not whole files. A chunk is often a few hundred tokens long. Chunks usually share a little text with the next one, so an idea on the edge is not cut in half. Cutting at headings and paragraphs works better than cutting at a fixed size.
The document is cut into four chunks. Each one shares a thin strip with the next.

Common mistake

Cutting at a fixed size through the middle of tables and lists. The halves make no sense on their own.

Key takeaways

  1. 1
    Small pieces. Often a few hundred tokens.
  2. 2
    A little overlap. Ideas are not cut in half.
  3. 3
    Follow the structure. Headings and paragraphs.