Skip to content
BENIBIX Learn

File Compaction

Also known as Table Compaction, Small File Compaction, Bin-Packing

Packing lots of small files into a few big ones. Reads get fast.

Think of it like

It is like putting a pile of loose photos in one book. Now there is just one thing to pick up.

Example

A team compacts its events table each night. About 260,000 small files become 400 large ones, and the morning report takes seconds, not minutes.

Why it matters

It is the standard fix for the small files problem. Without it, a busy lake table gets slower every week.

Definition

File compaction merges many small files in a table into a few large ones. The rows stay the same. Only the way they are packed changes. Readers then open far fewer files, so queries run faster. Delta Lake and Apache Iceberg both have a command for it. It is often run each night.

Example code

sql
OPTIMIZE events WHERE event_date >= '2026-11-01';
One Delta Lake command. It packs only this month's small files into larger ones.
Compaction is the arrow in the middle. The same rows end up in three large files.

Common mistake

Compacting a whole huge table every night. Compact only the new part, such as the last day, to save time.

Key takeaways

  1. 1
    Same rows. Only the packing changes.
  2. 2
    Fewer files. Fewer opens per query.
  3. 3
    Run often. Usually each night, on new data.