# Data Shuffle

> Data is sent between computers. Rows with the same key end up in one place.

**Field:** Data Engineering  
**Category:** Processing  
**Also known as:** Shuffle, Spark Shuffle

## Think of it like

It is like a class sorting cards by colour. Each child walks their red cards to the red desk.

## Definition

A shuffle moves data between computers so rows with the same key end up together. A join, a group by or a sort on a big table needs one. It is often the slowest part of a job, because data is written to disk and sent over the network. If one key has far more rows than the rest, one computer gets most of the work. That is called skew.

## Example

A job groups a billion sales by store. One store has half the sales, so one computer works for an hour while the rest wait.

## Why it matters

Shuffles decide how fast most big jobs run. Cutting one shuffle can halve a job's time.

## Common mistake

Joining a small table to a huge one with a full shuffle. Sending the small table to every computer is much faster.

## Three principles

- **Same key, same place.** Needed for joins and group by.
- **Slow by nature.** Disk and network.
- **Watch for skew.** One big key slows all.

## Related terms

- [MapReduce](https://learn.benibix.com/term/data/mapreduce)
- [Apache Spark](https://learn.benibix.com/term/data/apache-spark)
- [Data Partitioning](https://learn.benibix.com/term/data/data-partitioning)
- [Distributed System](https://learn.benibix.com/term/development/distributed-system)
- [Data Skew](https://learn.benibix.com/term/data/data-skew)
- [Broadcast Join](https://learn.benibix.com/term/data/broadcast-join)

## Told apart from

- [Data Skew vs Data Shuffle](https://learn.benibix.com/compare/data-skew-vs-data-shuffle): People think data skew and a data shuffle are the same slowdown, because both show up when a big job on many machines is slow. A shuffle is the step where rows move between machines, so rows with the same key meet. Skew is when one key has far more rows than the rest, so one machine gets most of them. A shuffle is often needed. Skew is a problem you can fix.

---

Source: https://learn.benibix.com/term/data/data-shuffle
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
