# MapReduce

> Split the work. Many computers each do a part. Then join the results.

**Field:** Data Engineering  
**Category:** Processing  
**Also known as:** Map Reduce, Hadoop MapReduce

## Think of it like

It is like a class counting a huge pile of beads. Each child counts a handful, then one child adds up all the counts.

## Definition

MapReduce is a way to process huge data on many computers at once. In the map step, each computer works on its own slice and outputs key and value pairs. In the shuffle, pairs with the same key are sent to the same computer. In the reduce step, each computer combines the values for its keys. It was first described in 2004, and Hadoop made it open source.

## Example

To count words in a billion web pages, each computer counts words in its own pages. Then all counts for "cat" go to one computer to be added up.

## Why it matters

It showed how to use thousands of cheap computers as one. Spark and most big data tools grew from the idea.

## Common mistake

Writing new jobs in raw MapReduce. Spark and SQL engines do the same work faster and with far less code.

## Three principles

- **Map.** Each computer works on its slice.
- **Shuffle.** Same key, same computer.
- **Reduce.** Combine each key's values.

## Related terms

- [Apache Hadoop](https://learn.benibix.com/term/data/apache-hadoop)
- [Data Shuffle](https://learn.benibix.com/term/data/data-shuffle)
- [Apache Spark](https://learn.benibix.com/term/data/apache-spark)
- [Distributed System](https://learn.benibix.com/term/development/distributed-system)
- [Big Data](https://learn.benibix.com/term/data/big-data)

## Told apart from

- [Apache Hadoop vs MapReduce](https://learn.benibix.com/compare/apache-hadoop-vs-mapreduce): People think Hadoop and MapReduce are the same thing. MapReduce is a way to split a job into map and reduce steps over many computers. Hadoop is a whole system. It has a file store called HDFS, a part that shares out the computers, and a MapReduce engine. Spark can run on Hadoop with no MapReduce at all.

![Three input slices of text each go to a map box that outputs word and count pairs. A shuffle band sends all pairs for the same word to one reduce box. Two reduce boxes output the final count for each word.](https://learn.benibix.com/diagrams/mapreduce.svg)

---

Source: https://learn.benibix.com/term/data/mapreduce
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
