Skip to content
BENIBIX Learn

LangSmith

Development🧬 AI EngineeringLLM App Operations

Also known as LangSmith Observability, LangSmith Tracing

A tool from LangChain that records, tests and watches LLM apps.

Think of it like

It is like a black box on a plane. After a bad trip, you can see what happened.

Example

Emma's agent gives a wrong refund amount. In LangSmith she opens that run's trace and sees that a tool returned last year's price.

Why it matters

An LLM app can fail in many quiet ways. A trace shows the exact step that went wrong, so you do not have to guess.

Definition

LangSmith is a platform from LangChain for tracing, testing and watching LLM apps. Each run of your app becomes a trace, with every model call and tool step inside. You turn it on with two settings: LANGSMITH_TRACING set to true, and your key. It works with many frameworks, not only LangChain. It also has tests, dashboards and alerts, in the cloud or on your own servers.

Example code

bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="<your-langsmith-api-key>"
Two settings turn tracing on. Each run then shows up in your project.
LangSmith shows a run like this, each step with its time and tokens.

Common mistake

Thinking it only works with LangChain. It traces plain code and other frameworks too, through its own code kit.

Key takeaways

  1. 1
    Traces. Every step of a run.
  2. 2
    Evaluations. Score runs against tests.
  3. 3
    Any framework. Not only for LangChain.

Compare