Prompt Caching and Cost
Level 4 · Prompts inside an app
How prompt caching reuses the long fixed start of a prompt for far less. Why the order of a prompt now matters, and where else the cost hides.
Course contentsLesson 11 of 16
Level 1 · What a prompt is
Level 2 · Writing a better prompt
Level 3 · Harder and bigger tasks
Level 4 · Prompts inside an app
After this lesson you can
- Order a prompt so its fixed part can be cached
- Set a sensible cap on reply length
- Find the biggest part of a request's cost
11 mincovers 2 glossary entries
Worth reading first: Structured Output and JSON
How prompt caching makes a long prompt cheaper
Prompt caching saves the model's work on the start of a prompt, so the next request that starts the same way can reuse it. On Claude, reading from the cache costs a tenth of the normal input price or less. Writing to it costs 25% more. It lasts 5 minutes, and each use resets the clock.
It helps when many requests share a long fixed start, such as a system prompt, a manual, a set of examples or a codebase. The first request pays to store it. Each request after that, inside the time limit, reads it back faster and for less.
The match is exact, and it runs from the very start. One changed letter early in the prompt breaks the match for everything after it. So the order of a prompt is now a cost choice. Fixed parts go first. Parts that change, like the date or the question, go last.
There is a floor. Very short prompts cannot be cached at all. On current Claude models, the fixed start has to be at least 512 to 4,096 tokens, depending on the model. Caching a 200-token system prompt saves nothing, so it suits long text that rarely changes.
Output is the other half of the bill. Output tokens cost more than input tokens, often several times more. Max tokens caps how long a reply can be. Leave room to spare: a cap set too low cuts the answer off in the middle, and the service says that is why it stopped.
The easiest savings are in the words. Remove rules the model already follows without being told. Cut examples that do not change the answer. Ask for the length you need, because a 50-word answer costs a tenth of a 500-word one, and it is often the better answer.
Measure before you cut. Count the tokens in a real request, split into the fixed start, the history, the documents and the reply. The biggest part is where the saving is. It is often the history or a pasted document, not the prompt you spent a week on.
Worked example
Emma's bot reads one manual all day
Emma's help bot sends the same 40,000-token manual with every question. The question comes last. Before caching, each one paid full price for the whole manual.
Questions arrive every minute or two, so the cache never runs out. The first question pays 25% more for the manual. Nearly every one after it pays a tenth or less.
The words for this lesson2
Check yourself
Where should the date go in a prompt that uses caching?
Finished this lesson?
Marking it complete also marks its 2 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.