🧬LLM Observability
Also known as AI Observability, LLM Tracing, LLM Monitoring
Keeping a record of each step an AI app takes, so you can fix it.
Think of it like
Think of it like the black box on a plane. It keeps a record of each step, so you can see what went wrong.
Example
Emma's chatbot gave a customer the wrong refund rule. She opens the trace and sees the search step pulled last year's rule. So she fixes the index, not the prompt.
Why it matters
Model apps fail in quiet ways, with answers that look fine. Traces are how you find out why.
Definition
Common mistake
Key takeaways
- 1Trace every step. Prompt to reply.
- 2Time, tokens, cost. On each call.
- 3Find quiet failures. Not only errors.