Skip to content
BENIBIX Learn

Token Usage

Development🧬 AI EngineeringLLM APIs

Also known as Token Count, LLM Usage, Input and Output Tokens

How many tokens a call used. Input and output are each counted.

Think of it like

Think of it like a taxi meter. It counts the trip, and you pay for what it counts.

Example

Emma logs the usage of every call. She finds one chat sends 9,000 input tokens of old history each time for a 50-token reply, and trims it.

Why it matters

Usage is what you pay for. Watching it is how you find calls that are slow or cost too much.

Definition

Token usage is how many tokens a call to a language model used, split into input and output. Input tokens are all you sent: the system prompt, the chat so far, any files and the new question. Output tokens are the reply the model wrote. The API sends back both counts with each answer. They set the cost of the call, and show how close it came to the model's limits.

Example code

json
"usage": {
  "input_tokens": 1240,
  "output_tokens": 312
}
The usage block that comes back with each reply.
The input holds the system prompt, the history and the question. The output is the reply.

Common mistake

Counting only the reply. In a long chat the input, which repeats the history each turn, is often most of the bill.

Key takeaways

  1. 1
    Input and output. Counted apart.
  2. 2
    Comes with each reply. Log it.
  3. 3
    History adds up. Each turn resends it.