# Data Engineering and Big Data

> What data engineering is and why a company with data needs it. The six stages data passes through, and the point where data counts as big.

**Course:** [Data Engineering Basics](https://learn.benibix.com/learn/data-foundations)  
**Time:** 10 min

**After this lesson you can:**

- Say what a data engineer builds, and why
- Name the six stages of the data lifecycle
- Tell big data from data that fits on one machine

## What data engineering is, and what it is for

Data engineering is the work of moving data to where people use it. A shop makes data in three places. There is its website, its tills and its help desk inbox. A data engineer builds the pipes that bring all three into one place. Every report and every model sits on top of those pipes.

Data passes through six stages on that trip, and together they are called the data lifecycle. It is made, then brought in, then stored. Next it is cleaned and reshaped. Then it is served to the people and tools that use it. Last, it is archived or deleted. A data engineer owns the middle four.

The plan for how those stages fit together is the data architecture. It says which tool sits at each stage, where the data is kept, and who may read it. A small firm can draw it on one page. A bank may need a whole team just to keep that plan up to date.

Big data is data too large or too fast for one machine to handle. People describe it with three words that start with V. Volume is how much there is. Velocity is how fast it arrives. Variety is how many shapes it comes in, from tables to photos. Any one of the three can be enough.

The line moves every year, because one machine keeps getting stronger. A laptop today can sort a file of tens of millions of rows in seconds. Work that needed a room of servers fifteen years ago now often fits on one. So the honest test is simple: does the job still finish on one machine in time?

This course covers the ideas that every data job shares. It stops before the tools. Running a warehouse is a subject of its own. So is moving events with Kafka, and so is running jobs on a timer with Airflow. If you need tables and queries first, start with [Databases and SQL](/learn/development-database).

People often mix up the jobs of data engineer and data scientist. A data scientist asks questions of the data and builds models. A data engineer makes sure the data is there before anyone asks. It has to be on time and right. When the pipes break, every chart down the line shows the wrong number.

![Six stages in a row, joined by arrows from left to right: generate, ingest, store, transform, serve, and last archive or delete.](https://learn.benibix.com/diagrams/data-lifecycle.svg)

### Example: A bakery chain counts its sales

Lucy runs 4 bakeries. Each till saves a file at closing, and every Monday she pastes all 4 into one sheet. It takes her 3 hours, and one week she pastes Tuesday twice.

A small pipeline now copies each file into one table at midnight. Her Monday report is ready by 7am. No day is ever counted twice.

**Words in this lesson:** [Data Engineering](https://learn.benibix.com/term/data/data-engineering), [Data Lifecycle](https://learn.benibix.com/term/data/data-lifecycle), [Data Architecture](https://learn.benibix.com/term/data/data-architecture), [Big Data](https://learn.benibix.com/term/data/big-data)

**Question:** When does data count as big data?

**Answer:** When one machine can no longer handle it in time. There is no fixed size. Data is big when its volume, its speed or its variety is too much for one machine. That line moves as machines get stronger.

---

Source: https://learn.benibix.com/learn/data-foundations/data-engineering-and-big-data
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
