Skip to content
BENIBIX Learn

Prompt Caching

AI🛠️ Working with AIOutput Control

Also known as Context Caching, Prefix Caching

Saving the work on a prompt's fixed start, so the next request reuses it.

Think of it like

Think of it like a teacher who keeps notes on a long book. Next time, they skip straight to the new question.

Example

Lucy's bot sends the same 50-page manual with every question. With prompt caching, only the first request pays full price for the manual. The rest reuse it.

Why it matters

For apps that resend a big fixed part, it can cut both cost and wait time a lot.

Definition

Prompt caching saves the model's work on the start of a prompt, so the next request that starts the same way can reuse it. It helps when many requests share a long fixed part, like a system prompt, a manual or a codebase. The reused part is read much faster and billed at a lower price. The cache only lasts a short time, often minutes, and the start has to match exactly. Any change early in the prompt breaks the match for everything after it.
The second request starts the same way, so that part comes from the cache. Only the new question is worked out.

Common mistake

Putting changing text, like the date, at the very start. The cache only matches up to the first change, so fixed parts must come first.

Key takeaways

  1. 1
    Reuse the fixed start. Same start, less work.
  2. 2
    Exact match. One change breaks it.
  3. 3
    Short-lived. Often minutes.

Compare

Learn it properly