# API Ingestion

> Pulling data out of an app through its API, a page at a time.

**Field:** Data Engineering  
**Category:** Ingestion & Streaming  
**Also known as:** API Data Extraction, REST API Ingestion, API Extraction

## Think of it like

It is like asking for books at a desk, a few at a time. You ask until you have them all.

## Definition

API ingestion pulls data from an app by calling its API. A script asks for records, and the app sends back a page of them, often as JSON. The script asks for the next page until there are none left. It must stay under the app's rate limit and try again when a call fails. Most apps that run in the cloud share their data this way.

## Example

Jack's script asks a payments app for yesterday's charges, 100 to a page. It makes 40 calls and loads 4,000 charges into the warehouse.

## Example code

```console
$ curl "https://api.example.com/v1/charges?page=2"
{"data": [ ... ], "next_page": 3}
```

Page two of the charges. The reply says which page to ask for next.

## Why it matters

Many tools hold data you need and give no other way out. The API is the only door.

## Common mistake

Ignoring the rate limit. The app starts to refuse calls, and the load stops half way.

## Three principles

- **Page by page.** Ask until no pages are left.
- **Mind the limit.** Stay under the calls allowed.
- **Try again.** Calls fail, so retry them.

## Related terms

- [Data Ingestion](https://learn.benibix.com/term/data/data-ingestion)
- [REST API](https://learn.benibix.com/term/development/rest-api)
- [Pagination](https://learn.benibix.com/term/development/pagination)
- [Rate Limiting](https://learn.benibix.com/term/development/rate-limiting)
- [Data Connector](https://learn.benibix.com/term/data/data-connector)
- [Webhook](https://learn.benibix.com/term/development/webhook)

![Three sources on the left: a payments app with a REST API, a well-known sales tool and website clicks. Each feeds its own path: API ingestion, where a script asks page by page; a data connector, set up in a form; and highlighted streaming ingestion, where each event lands in seconds. All three lead to a warehouse or lake.](https://learn.benibix.com/diagrams/dat2-ingestion-paths.svg)

---

Source: https://learn.benibix.com/term/data/api-ingestion
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
