# Semantic Cache

> A store that gives back a saved answer when a new question means the same.

**Field:** Software Development  
**Category:** AI Engineering  
**Also known as:** Semantic Caching, LLM Cache

## Think of it like

Think of it like a teacher who hears the same question in new words. They give the same answer again.

## Definition

A semantic cache stores model answers and reuses them for new questions that mean the same thing. Each question is turned into an embedding, a list of numbers that captures its meaning. When a new question comes in, the cache looks for a saved one that is close enough. If it finds one, it returns the saved answer at once, with no model call. If not, the model answers and the new pair is saved.

## Example

Lucy's help bot gets "how do I reset my password" in dozens of ways. The semantic cache answers most of them in a few milliseconds. It cuts her bill by a third.

## Why it matters

Repeat questions are common, and a cache hit is both much faster and almost free.

## Common mistake

Setting the match too loose. Two questions can be close in meaning but need different answers, and a loose cache serves the wrong one.

## Three principles

- **Match by meaning.** Not exact words.
- **A hit skips the model.** Fast and cheap.
- **Tune the threshold.** Too loose is wrong.

## Related terms

- [Cache](https://learn.benibix.com/term/development/cache)
- [Prompt Caching](https://learn.benibix.com/term/ai/prompt-caching)
- [Embedding (AI)](https://learn.benibix.com/term/ai/embedding)
- [Cosine Similarity](https://learn.benibix.com/term/ai/cosine-similarity)

## Told apart from

- [Semantic Cache vs Prompt Caching](https://learn.benibix.com/compare/semantic-cache-vs-prompt-caching): People often think a semantic cache and prompt caching are the same money saving trick. A semantic cache skips the model and returns a saved answer. It does that when a new question means the same as an old one. Prompt caching still calls the model every time, but it reuses the work done on the start of a long prompt that has not changed.

![A new question is turned into a vector and compared with saved questions. A close match returns the saved answer at once. No match goes on to the model.](https://learn.benibix.com/diagrams/dev-semantic-cache.svg)

---

Source: https://learn.benibix.com/term/development/semantic-cache
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-20
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
