# Data Partitioning

> A big table cut into parts by one column. Most often that column is a date.

**Field:** Data Engineering  
**Category:** Storage & Warehousing  
**Also known as:** Table Partitioning, Partitioned Table, Partition Key

## Think of it like

It is like a filing box with one drawer per month. To find May, you open just the May drawer.

## Definition

Partitioning splits one big table into parts by the value of a column. The column is often a date, so each day or month gets its own part. In a data lake, each part is usually its own folder. Queries that filter on that column only read the parts they need. Old parts can also be dropped in one step.

## Example

An events table holds three years of clicks, split by day. A report on last week reads 7 folders, not 1,095.

## Example code

```console
$ ls events/
date=2026-11-14/  date=2026-11-15/  date=2026-11-16/
```

A table split by day in a data lake. Each day is its own folder.

## Why it matters

It is the cheapest way to make big tables fast. It also makes deleting old data easy.

## Common mistake

Splitting by a column with too many values, such as user ID. You get millions of tiny parts, and everything slows down.

## Three principles

- **Split by a column.** Most often a date.
- **Read fewer parts.** Filters skip whole parts.
- **Not too many.** Tiny parts are slow.

## Related terms

- [Partition Pruning](https://learn.benibix.com/term/data/partition-pruning)
- [Clustering Key](https://learn.benibix.com/term/data/clustering-key)
- [Database Sharding](https://learn.benibix.com/term/development/sharding)
- [Data Lake](https://learn.benibix.com/term/data/data-lake)

## Told apart from

- [Kafka Partition vs Data Partitioning](https://learn.benibix.com/compare/kafka-partition-vs-data-partitioning): People think a Kafka partition and a table partition are the same, and they are not. A table partition splits stored rows by a column, such as the date. Then a query can skip the parts it does not need. A Kafka partition splits a stream into lanes, each kept in order. Then more readers can work at once. One helps you skip, the other helps you share.
- [Data Partitioning vs Clustering Key](https://learn.benibix.com/compare/data-partitioning-vs-clustering-key): People think partitioning and clustering are the same trick. One cuts a table into separate parts, often one per day. The other keeps the table whole but sorts the rows in it. Split on a column with few values, like a date. Cluster on one with many, like a customer ID.
- [Data Partitioning vs Database Sharding](https://learn.benibix.com/compare/data-partitioning-vs-sharding): People think partitioning and sharding are the same thing. Both split data into parts. One splits a table into parts inside one system, often by date, so queries can skip parts. The other spreads rows over many database servers, so each server takes part of the load.

![A table split into seven daily folders. A query that filters on one day reads only that folder, and the other six are greyed out as skipped.](https://learn.benibix.com/diagrams/partition-pruning.svg)

---

Source: https://learn.benibix.com/term/data/data-partitioning
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-16
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
