# Data Backfill

> Run a pipeline again for past days. It fills gaps or fixes old data.

**Field:** Data Engineering  
**Category:** Ingestion & Streaming  
**Also known as:** Backfill, Backfilling

## Think of it like

You missed a few days in your diary. So you go back and fill in those pages.

## Definition

A backfill runs a pipeline over past dates to fill in or fix old data. It is needed when a new pipeline starts, so it has history to work with. It is also needed when a bug is fixed, so old wrong rows can be rebuilt. The job runs once for each missing day or hour. It only works well if the pipeline is safe to run twice.

## Example

A bug cut tax from orders for three weeks. After the fix, the team backfills those 21 days, and the reports are right again.

## Why it matters

Data is never right the first time forever. Backfills are how history gets repaired.

## Common mistake

Backfilling a pipeline that appends rows. Each day ends up with two copies of every order.

## Three principles

- **Past dates.** Rerun the job for old days.
- **Fill or fix.** New history or repaired history.
- **Safe to rerun.** The job must not double data.

## Related terms

- [Data Pipeline](https://learn.benibix.com/term/data/data-pipeline)
- [Idempotency](https://learn.benibix.com/term/development/idempotency)
- [Data Ingestion](https://learn.benibix.com/term/data/data-ingestion)
- [Batch Processing](https://learn.benibix.com/term/data/batch-processing)
- [Airflow Catchup](https://learn.benibix.com/term/data/airflow-catchup)

## Told apart from

- [Airflow Catchup vs Backfill](https://learn.benibix.com/compare/airflow-catchup-vs-data-backfill): People think catchup and a backfill are the same thing. Catchup is a switch on a DAG. When the DAG is turned on late, Airflow runs every missed date by itself, all at once. A backfill is a run you ask for, over dates you choose, when you are ready. Catchup happens to you. A backfill is something you plan.

---

Source: https://learn.benibix.com/term/data/data-backfill
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
