# LLM Observability

> Keeping a record of each step an AI app takes, so you can fix it.

**Field:** Software Development  
**Category:** AI Engineering  
**Also known as:** AI Observability, LLM Tracing, LLM Monitoring

## Think of it like

Think of it like the black box on a plane. It keeps a record of each step, so you can see what went wrong.

## Definition

LLM observability means keeping a record of what an AI app does on each request. Then you can see what went wrong and fix it. A trace holds each step of one request. That is the prompt, the files it found, each model and tool call, the replies, the time, the tokens and the cost. Charts then show trends, such as slow calls, a rising bill or more bad answers. Without it, a wrong answer is very hard to explain.

## Example

Emma's chatbot gave a customer the wrong refund rule. She opens the trace and sees the search step pulled last year's rule. So she fixes the index, not the prompt.

## Why it matters

Model apps fail in quiet ways, with answers that look fine. Traces are how you find out why.

## Common mistake

Logging only errors. Most model failures return a normal reply with wrong content, so you need the full trace of good calls too.

## Three principles

- **Trace every step.** Prompt to reply.
- **Time, tokens, cost.** On each call.
- **Find quiet failures.** Not only errors.

## Related terms

- [Token Usage](https://learn.benibix.com/term/development/token-usage)
- [LLM Gateway](https://learn.benibix.com/term/development/llm-gateway)
- [Prompt Versioning](https://learn.benibix.com/term/development/prompt-versioning)
- [Latency](https://learn.benibix.com/term/infrastructure/latency)
- [Model Monitoring](https://learn.benibix.com/term/ai/model-monitoring)
- [Observability](https://learn.benibix.com/term/infrastructure/observability)
- [LangSmith](https://learn.benibix.com/term/development/langsmith)

![A trace of one user request, drawn as a timeline. It shows the retrieval step, two model calls and a tool call, each with its time, tokens and cost.](https://learn.benibix.com/diagrams/dev-llm-observability.svg)

---

Source: https://learn.benibix.com/term/development/llm-observability
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-20
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
