# Data Pipeline

> A chain of steps that moves data from where it starts to where it ends up.

**Field:** Data Engineering  
**Category:** Foundations

## Think of it like

It is like a belt in a factory. Things go in one end, get worked on, and come out the other end.

## Definition

A data pipeline is a set of steps that moves data from one place to another. It reads data from a source, such as an app's database. It may clean, join or add up the data on the way. Then it writes the result to a destination, such as a warehouse. Pipelines usually run on a schedule or whenever new data arrives.

## Example

Every hour, a pipeline copies new orders from the shop. It adds tax. Then it writes them to the sales table.

## Why it matters

A pipeline turns a one-time copy into a job. The job runs each day by itself. No one has to press a button.

## Common mistake

Building a pipeline that breaks if it runs twice. A good one gives the same result when it is run again.

## Three principles

- **Source to destination.** Data flows one way through it.
- **Steps in order.** Each step feeds the next one.
- **Safe to rerun.** Running it twice must not double the data.

## Related terms

- [Extract, Transform, Load (ETL)](https://learn.benibix.com/term/data/etl)
- [Extract, Load, Transform (ELT)](https://learn.benibix.com/term/data/elt)
- [Batch Processing](https://learn.benibix.com/term/data/batch-processing)
- [Stream Processing](https://learn.benibix.com/term/data/stream-processing)
- [Idempotency](https://learn.benibix.com/term/development/idempotency)

## Taught in

- [ETL, ELT and Reverse ETL](https://learn.benibix.com/learn/data-foundations/etl-elt-and-reverse-etl): Data Engineering Basics, 4/11

---

Source: https://learn.benibix.com/term/data/data-pipeline
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-20
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
