Tokens per Minute vs Provisioned Throughput
People think provisioned throughput is just a higher tokens per minute limit, and it is not. A tokens per minute limit is a cap on shared capacity, and you pay only for what you use. Provisioned throughput is capacity set aside for you alone. You pay for it by the hour or month, even on quiet days. One is a ceiling. The other is a room you rent.
Tokens per Minute (TPM)
A cap on how many tokens an app may use each minute. Over it, you wait.
Tokens per minute is a limit on how much text you can send to or get from an AI model each minute. Model services count it in tokens, the small pieces of text a model reads and writes. There is often a second limit on the number of requests per minute. When you go over either one, the service turns the request away with an error. Your app should wait a moment and try again. Bigger accounts can usually get higher limits.
Provisioned Throughput
You rent a fixed share of a model, for a fixed price.
Provisioned throughput means you reserve a fixed amount of model capacity from a cloud, for a fixed price. You pay by the hour or the month, whether you use it or not. In return, your requests do not share space with everyone else, so speed stays steady. Each cloud has its own name and unit for it. It suits apps with steady, heavy traffic. For light or spiky use, paying per token is usually cheaper.
In one line
- Tokens per Minute (TPM)
- A cap on how many tokens an app may use each minute. Over it, you wait.
- Provisioned Throughput
- You rent a fixed share of a model, for a fixed price.