# The LLM Context Window

> How much text a model can see at once, and how a chat is sent as messages with roles. What happens when the space runs out.

**Course:** [Large Language Models (LLMs)](https://learn.benibix.com/learn/ai-llms)  
**Time:** 12 min

**After this lesson you can:**

- Explain what fills the context window
- Name the three message roles
- Pick a fix for a chat that runs out of room

## How much a model can see at once

The context window is how much text a model can take in at once. It is counted in tokens, and it holds both what you send and what the model writes back. Many models today take 200,000 tokens, about 150,000 words. A few take 1 million. Anything past the limit is cut or refused.

What you send is the prompt. In a chat it is split into messages, and each one has a role. The system prompt holds the standing instructions. User messages are what the person typed. Assistant messages are what the model wrote before. The model reads all of them, in order, every time.

That last point surprises people. The model keeps no memory between requests. A chat app sends the whole conversation again with each new message. So a long chat makes the prompt grow each time, costs a little more each time, and slowly fills the window.

Max tokens is a cap you set on the reply. Set it to 300 and the model stops after 300 tokens, even in the middle of a sentence. The service says why it stopped, so your code can tell a finished reply from a cut one. A cap set too low is a common cause of half answers.

A stop sequence is another way to end. You name a piece of text, such as a closing tag, and the model stops the moment it writes it. The stop text itself is usually left out of the reply. It helps when you only want the first part of a longer pattern.

When a long chat nears the limit, something has to give. Some apps drop the oldest messages. Others use compaction: the model writes a short summary of the earlier messages, and the summary takes their place. The chat goes on, but small details from early on can be lost.

A bigger window is not the same as better reading. Models tend to use the start and end of a long input better than the middle. So put the key instruction near the top and the question at the end. That often helps more than adding more text.

```json
[
  { "role": "system", "content": "You are a short, friendly maths tutor." },
  { "role": "user", "content": "What is 15% of 80?" },
  { "role": "assistant", "content": "15% of 80 is 12." },
  { "role": "user", "content": "And 20%?" }
]
```
The whole chat is sent again with the newest question. The model answers the last message, using all the ones above it.

![A long strip of text with a frame over its right-hand end, labelled context window. The older text to the left of the frame is marked out of view, and the newest text sits inside the frame, marked what the model can see.](https://learn.benibix.com/diagrams/llm-context-window.svg)

### Example: Emma's tutoring app runs out of room

Emma's tutoring app sends the whole chat each time. After 40 messages of about 500 tokens each, every request carries 20,000 tokens.

Each new answer now pays for all 20,000 again. She adds compaction at 15,000 tokens, which keeps every request small and the cost flat.

**Words in this lesson:** [Context Window](https://learn.benibix.com/term/ai/context-window), [Prompt (AI)](https://learn.benibix.com/term/ai/prompt), [Message Roles](https://learn.benibix.com/term/ai/message-roles), [System Prompt](https://learn.benibix.com/term/ai/system-prompt), [Max Tokens](https://learn.benibix.com/term/ai/max-tokens), [Stop Sequence](https://learn.benibix.com/term/ai/stop-sequence), [Context Compaction](https://learn.benibix.com/term/ai/context-compaction)

**Question:** How does a chat model remember what you said earlier?

**Answer:** The app sends the whole chat again each time. The model keeps nothing between requests. The app sends every earlier message again. That is why a long chat costs more and slowly fills the window.

---

Source: https://learn.benibix.com/learn/ai-llms/llm-context-window
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
