Amazon EMR vs AWS Glue
People think Amazon EMR and AWS Glue are the same Spark service, and they are not. EMR builds clusters you can tune and run many tools on. You pay for the machines while they run. Glue is serverless. You hand it an ETL job, and AWS runs the Spark for you. Pick EMR for control and Glue for less work.
Amazon EMR
The AWS service that runs Spark and Hadoop clusters for you.
Amazon EMR is a cloud service from AWS that runs big data tools for you. It used to be called Amazon Elastic MapReduce. It sets up clusters for Apache Spark, Hadoop and similar tools. Data usually sits in Amazon S3, and clusters come and go around it. You pay for the machines while a cluster runs.
AWS Glue
Amazon's tool to find, clean and move data. It keeps a list of tables.
AWS Glue is a serverless tool from Amazon Web Services for finding and moving data. Its crawlers scan files and databases and guess each table's columns. The results go into the Glue Data Catalog, a shared list of tables. Athena and other tools can query it. Glue also runs ETL jobs on a Spark engine that AWS manages for you. It can connect to more than 70 kinds of data source.
In one line
- Amazon EMR
- The AWS service that runs Spark and Hadoop clusters for you.
- AWS Glue
- Amazon's tool to find, clean and move data. It keeps a list of tables.