Skip to content
BENIBIX Learn

Amazon EMR vs AWS Glue

People think Amazon EMR and AWS Glue are the same Spark service, and they are not. EMR builds clusters you can tune and run many tools on. You pay for the machines while they run. Glue is serverless. You hand it an ETL job, and AWS runs the Spark for you. Pick EMR for control and Glue for less work.

Data⚙️ Processing

Amazon EMR

The AWS service that runs Spark and Hadoop clusters for you.

Amazon EMR is a cloud service from AWS that runs big data tools for you. It used to be called Amazon Elastic MapReduce. It sets up clusters for Apache Spark, Hadoop and similar tools. Data usually sits in Amazon S3, and clusters come and go around it. You pay for the machines while a cluster runs.

Example
A team starts an EMR cluster each night. It runs its Spark jobs on data in S3, then shuts the cluster down by morning.
Why it matters
Building a Spark cluster by hand is slow, fiddly work. EMR does it for you, so a team can focus on the jobs.
Full entry
Data⚙️ Processing

AWS Glue

Amazon's tool to find, clean and move data. It keeps a list of tables.

AWS Glue is a serverless tool from Amazon Web Services for finding and moving data. Its crawlers scan files and databases and guess each table's columns. The results go into the Glue Data Catalog, a shared list of tables. Athena and other tools can query it. Glue also runs ETL jobs on a Spark engine that AWS manages for you. It can connect to more than 70 kinds of data source.

Example
Jack points a Glue crawler at a folder of sales files in S3. It adds a sales table to the catalog. A minute later, Athena can query it.
Why it matters
One catalog lets many AWS tools agree on what tables exist. The jobs run without servers to look after.
Full entry

In one line

Amazon EMR
The AWS service that runs Spark and Hadoop clusters for you.
AWS Glue
Amazon's tool to find, clean and move data. It keeps a list of tables.