🧬Semantic Cache
Also known as Semantic Caching, LLM Cache
A store that gives back a saved answer when a new question means the same.
Think of it like
Think of it like a teacher who hears the same question in new words. They give the same answer again.
Example
Lucy's help bot gets "how do I reset my password" in dozens of ways. The semantic cache answers most of them in a few milliseconds. It cuts her bill by a third.
Why it matters
Definition
Common mistake
Key takeaways
- 1Match by meaning. Not exact words.
- 2A hit skips the model. Fast and cheap.
- 3Tune the threshold. Too loose is wrong.