Skip to content
BENIBIX Learn

Full Load

Data📥 Ingestion & StreamingData Ingestion

Also known as Full Refresh, Full Extract, Full Reload, Truncate and Load

Copy the whole table again each time. Simple, but slow when the table is big.

Think of it like

It is like tipping out a toy box and putting each toy back. Nothing is missed, but it is slow.

Example

A shop reloads its list of 300 stores every night, and it takes seconds. Its 50 million orders are loaded a bit at a time instead.

Why it matters

It is the safe choice when deletes and edits are hard to track. Every run ends with an exact copy.

Definition

A full load copies every row from a source each time it runs. It first empties the old copy, or replaces it. Then it copies the whole table again. It is the simplest way to load data, and nothing can slip past. But it gets slow and costly as the table grows. Small tables are often loaded this way every night.

Example code

sql
TRUNCATE TABLE stores;
INSERT INTO stores
SELECT * FROM source_stores;
Empty the copy, then copy every row back in. That is a full load.
The top path is a full load. It copies all 10,000 rows, even though only 200 changed.

Common mistake

Fully loading a huge table every hour. The bill and the run time grow with every new row.

Key takeaways

  1. 1
    Every row. Nothing depends on what changed.
  2. 2
    Simple. No dates or markers to track.
  3. 3
    Costly at size. Run time grows with the table.

Compare