# LLM Batch API

> Send lots of model requests at once. The answers come back later and cost less.

**Field:** Software Development  
**Category:** AI Engineering  
**Also known as:** Batch API (LLM), Batch Processing (LLM), Message Batches

## Think of it like

Think of it like dropping off a big pile of laundry. It costs less, and you pick it up later.

## Definition

A batch API lets you send lots of model requests in one go and get the answers later, for less. You send a file of requests, each with its own id. The model maker runs them when it has spare room. It sends back a file of results, most often within a day. The price is often about half that of a normal call. It suits big jobs where no one is waiting, such as tagging a whole archive.

## Example

Jack needs short summaries of 40,000 old product reviews. He sends them as one batch overnight and pays about half of what single calls would cost.

## Example code

```json
{"custom_id": "review-1", "params": {"model": "model-name", "max_tokens": 100, "messages": [{"role": "user", "content": "Summarize: ..."}]}}
```

One line of a batch file. Each request carries its own id so the results can be matched up.

## Why it matters

Big jobs that can wait cost about half as much. They also leave your rate limits free for the live app.

## Common mistake

Using it for anything a user is waiting on. Results can take hours, so live features need normal calls.

## Three principles

- **Many at once.** One file in.
- **Answers later.** Often within a day.
- **Lower price.** Often about half.

## Related terms

- [LLM API](https://learn.benibix.com/term/development/llm-api)
- [Token Pricing](https://learn.benibix.com/term/development/token-pricing)
- [Rate Limiting](https://learn.benibix.com/term/development/rate-limiting)

![A file of many requests is sent at once. The provider works through it when it has spare room, and later returns a file of answers at a lower price.](https://learn.benibix.com/diagrams/dev-batch-api.svg)

---

Source: https://learn.benibix.com/term/development/batch-api
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-20
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
