🗄️Delta Lake
Also known as Delta Table, Delta Format
A table format first made by Databricks. It is Parquet files plus a log of changes.
Think of it like
It is like a diary next to a photo book. Each page says which photos came in or went out.
Example
A job deletes one customer's rows to meet a privacy request. Delta writes new files and logs the change as version 42.
Why it matters
It is the main table format on Databricks. So a large share of lakehouse data is kept in it.
Definition
Example code
sql
CREATE TABLE orders (id BIGINT, total DOUBLE) USING DELTA;
DESCRIBE HISTORY orders;Common mistake
Key takeaways
- 1Parquet underneath. The data is plain Parquet files.
- 2The _delta_log. A log of every change.
- 3Versions. Any past version can be read.