Skip to content
BENIBIX Learn

Long Context Problems

Level 5 · Where LLMs go wrong

Why a model misses facts in the middle of a long input, and why answers get worse as it grows. How to send less, in a better order.

Course contentsLesson 15 of 15

After this lesson you can

  • Explain the lost-in-the-middle pattern
  • Say why long inputs weaken answers
  • Order a long prompt so the key parts land

10 mincovers 2 glossary entries

Worth reading first: The LLM Context Window, Self-Attention and Multi-Head

Why a model misses facts in a long input

A model does not read a long input equally well from end to end. A 2023 study called 'Lost in the Middle' hid one key fact at different places in a long input. Models found it most often near the start or the end. In the middle, they missed it far more often.

The pattern makes a U shape: strong at both ends, weak in the middle. It held across many models, including ones built for long inputs. A bigger context window let more text in, but it did not make the middle any easier to use.

Context rot is the wider version. As an input gets longer, answers get worse. That holds even on easy tasks the model does well when the input is short. A 2025 test of 18 models found their answers grew less reliable as the input grew, well before the window was full.

Part of the reason is attention. Every token shares its focus across all the others. So a key line in a long input gets a thinner share. Lots of text that is nearly useful makes it worse, because the model has more wrong places to look.

The fixes are about sending less, in a better order. Send only the parts that matter, not the whole file. Put the key instruction at the top and the question at the end. Start a fresh chat for a new task, instead of carrying on a long old one.

For a long chat, compaction helps here too, since a short summary is easier to use than a long history. For a big set of files, search them first and send the best few pages. Fewer, better tokens usually beat more tokens.

The common mistake is treating a big context window as a promise. A model that takes 1 million tokens will accept them all. That does not mean it will use them all well. Test your own task at the length you plan to use.

A key fact at either end is found most often. The same fact in the middle is the one most often missed.

Worked example

Lucy moves the refund rule

Lucy's prompt holds 40 pages of policy. One line says refunds over $500 need a manager. Buried on page 20, it is missed in about 1 test in 4.

She moves that line to the top of the prompt. Now it is almost never missed, and the other 40 pages still go in below it.

The words for this lesson2

Check yourself

Where in a long input is a model most likely to miss a key fact?

Finished this lesson?

Marking it complete also marks its 2 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.