# File Compaction

> Packing lots of small files into a few big ones. Reads get fast.

**Field:** Data Engineering  
**Category:** Storage & Warehousing  
**Also known as:** Table Compaction, Small File Compaction, Bin-Packing

## Think of it like

It is like putting a pile of loose photos in one book. Now there is just one thing to pick up.

## Definition

File compaction merges many small files in a table into a few large ones. The rows stay the same. Only the way they are packed changes. Readers then open far fewer files, so queries run faster. Delta Lake and Apache Iceberg both have a command for it. It is often run each night.

## Example

A team compacts its events table each night. About 260,000 small files become 400 large ones, and the morning report takes seconds, not minutes.

## Example code

```sql
OPTIMIZE events WHERE event_date >= '2026-11-01';
```

One Delta Lake command. It packs only this month's small files into larger ones.

## Why it matters

It is the standard fix for the small files problem. Without it, a busy lake table gets slower every week.

## Common mistake

Compacting a whole huge table every night. Compact only the new part, such as the last day, to save time.

## Three principles

- **Same rows.** Only the packing changes.
- **Fewer files.** Fewer opens per query.
- **Run often.** Usually each night, on new data.

## Related terms

- [Small Files Problem](https://learn.benibix.com/term/data/small-files-problem)
- [Delta Lake](https://learn.benibix.com/term/data/delta-lake)
- [Apache Iceberg](https://learn.benibix.com/term/data/apache-iceberg)
- [Data Partitioning](https://learn.benibix.com/term/data/data-partitioning)
- [Open Table Format](https://learn.benibix.com/term/data/open-table-format)
- [Apache Parquet](https://learn.benibix.com/term/data/apache-parquet)

![Left, a box of thirty tiny grey files that a stream adds one every few seconds, and every query opens all of them. An arrow marked compact leads right to a box holding three large highlighted files with the same rows and nothing lost, so a query opens three files.](https://learn.benibix.com/diagrams/dat3-compaction.svg)

---

Source: https://learn.benibix.com/term/data/file-compaction
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-16
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
