# Tokens per Second

> How fast a model writes, counted in tokens each second.

**Field:** Artificial Intelligence  
**Category:** Running Locally  
**Also known as:** Tokens/s, Generation Speed, Inference Speed

## Think of it like

Think of it like how fast a friend talks. Some talk slowly, some talk very fast.

## Definition

Tokens per second is how fast a model writes its answer, counted as the tokens it puts out each second. It depends on the size of the model, how much it was shrunk, and the chip and memory it runs on. For each token, the model reads all its weights from memory. So memory speed counts for a lot. About 10 tokens per second is faster than most people read. A second number says how fast the model reads your prompt, and that one is most often much higher.

## Example

Lucy tries the same model on two machines. Her old laptop writes about 6 tokens per second, and her new desktop with a graphics card writes about 70.

## Why it matters

It decides whether a local model feels like a quick chat or a slow wait.

## Common mistake

Judging speed on a short prompt. Long prompts and long chats slow things down, so test with what you will really send.

## Three principles

- **Speed of writing.** Tokens out each second.
- **Memory speed rules.** Weights are read each token.
- **Reading is faster.** A separate number.

## Related terms

- [Throughput](https://learn.benibix.com/term/infrastructure/throughput)
- [Memory Bandwidth](https://learn.benibix.com/term/infrastructure/memory-bandwidth)
- [Latency](https://learn.benibix.com/term/infrastructure/latency)
- [Local AI Model](https://learn.benibix.com/term/ai/local-ai-model)

![Three bars for the speed of the same model on different machines, measured in tokens per second. A line marks how fast a person reads.](https://learn.benibix.com/diagrams/lo-tokens-per-second.svg)

---

Source: https://learn.benibix.com/term/ai/tokens-per-second
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
