Skip to content
BENIBIX Learn

Context Window

AI💬 LLMsLLM Basics

Also known as Context Length, Context Size

The most text, in tokens, a model can read at one time.

Think of it like

Think of it like a desk. Only so many pages fit on it at once.

Example

Jack pastes a 900-page book into a chat with a 200,000-token context window. The book is about 450,000 tokens, so the tool only takes in part of it.

Why it matters

It sets how long a chat can run. It also sets how big a file you can hand over before the model starts to lose things.

Definition

The context window is the most text a language model can take in at one time. It is counted in tokens. It holds all of one request: the instructions, the chat so far, any pasted files and the answer being written. Sizes run from a few thousand tokens in small models to over a million in some large ones. Text that does not fit is cut off. The model cannot see it.
Only the tokens inside the frame reach the model. Anything older has fallen out of view.

Common mistake

Thinking a bigger window means the model remembers you. Each chat starts empty, and the window only holds what is sent this time.

Key takeaways

  1. 1
    Counted in tokens. Input and output together.
  2. 2
    Thousands to millions. Depends on the model.
  3. 3
    Outside is unseen. Cut text is simply gone.

Compare

Learn it properly