Skip to content
BENIBIX Learn

Tokens per Minute (TPM)

AI☁️ Cloud AI PlatformsCloud AI Basics

Also known as Token Rate Limit, AI Rate Limit, Requests per Minute (AI), RPM

A cap on how many tokens an app may use each minute. Over it, you wait.

Think of it like

It is like a tap that lets out only so much water each minute. The rest has to wait.

Example

Jack's app sends a burst of long chats to the Claude API. It passes its input tokens per minute limit and gets error 429, so the app waits and tries again.

Why it matters

A busy app hits this wall first. Knowing the limit lets you plan for peaks instead of failing at them.

Definition

Tokens per minute is a limit on how much text you can send to or get from an AI model each minute. Model services count it in tokens, the small pieces of text a model reads and writes. There is often a second limit on the number of requests per minute. When you go over either one, the service turns the request away with an error. Your app should wait a moment and try again. Bigger accounts can usually get higher limits.
The left panel is a tokens per minute limit. A request that would cross the line is turned away.

Common mistake

Retrying at once in a tight loop. The new requests also fail and keep the app over the limit.

Key takeaways

  1. 1
    Counted in tokens. Plus a cap on requests each minute.
  2. 2
    Over means wait. Error 429, then try again later.
  3. 3
    Grows with the account. Higher tiers get higher limits.

Compare