Apache Hadoop vs MapReduce
People think Hadoop and MapReduce are the same thing. MapReduce is a way to split a job into map and reduce steps over many computers. Hadoop is a whole system. It has a file store called HDFS, a part that shares out the computers, and a MapReduce engine. Spark can run on Hadoop with no MapReduce at all.
Apache Hadoop
A free system for huge data. It runs on lots of cheap computers.
Apache Hadoop is a free, open source system for storing and processing huge data on many cheap computers. It has three main parts. HDFS stores the files across the computers. YARN shares out the computers' power. MapReduce runs the jobs. It sends the work to the computers that hold the data, rather than moving the data.
MapReduce
Split the work. Many computers each do a part. Then join the results.
MapReduce is a way to process huge data on many computers at once. In the map step, each computer works on its own slice and outputs key and value pairs. In the shuffle, pairs with the same key are sent to the same computer. In the reduce step, each computer combines the values for its keys. It was first described in 2004, and Hadoop made it open source.
In one line
- Apache Hadoop
- A free system for huge data. It runs on lots of cheap computers.
- MapReduce
- Split the work. Many computers each do a part. Then join the results.