Skip to content
BENIBIX Learn

Delta Lake

Also known as Delta Table, Delta Format

A table format first made by Databricks. It is Parquet files plus a log of changes.

Think of it like

It is like a diary next to a photo book. Each page says which photos came in or went out.

Example

A job deletes one customer's rows to meet a privacy request. Delta writes new files and logs the change as version 42.

Why it matters

It is the main table format on Databricks. So a large share of lakehouse data is kept in it.

Definition

Delta Lake is an open table format first built by Databricks. It stores data as Parquet files plus a folder called _delta_log. Each change adds a new entry to that log. The log gives safe, all-or-nothing writes, even with many writers. You can read the table as it was at any past version.

Example code

sql
CREATE TABLE orders (id BIGINT, total DOUBLE) USING DELTA;
DESCRIBE HISTORY orders;
Make a Delta table, then list every change the log has recorded.

Common mistake

Copying the Parquet files but not the _delta_log folder. The files are not the table without the log.

Key takeaways

  1. 1
    Parquet underneath. The data is plain Parquet files.
  2. 2
    The _delta_log. A log of every change.
  3. 3
    Versions. Any past version can be read.

Compare