# Spark Executor

> A worker in Spark. It runs the tasks the driver sends it.

**Field:** Data Engineering  
**Category:** Processing  
**Also known as:** Executor (Spark), Spark Worker Process

## Think of it like

It is like a cook at one station in a big kitchen. The cook does the tasks the head chef sends.

## Definition

A Spark executor is a worker process that runs tasks for one Spark program. Each one runs on a machine in the cluster. It does the real work, such as reading files and adding up rows. It can also keep data in memory or on disk between tasks. More executors means more tasks can run at the same time.

## Example

A cluster has 20 executors. Each one reads its own share of the files, so the job runs 20 ways at once.

## Why it matters

Executors are where the time and the money go. Their number and size decide how fast a job runs.

## Common mistake

Giving executors too little memory. They spill data to disk or crash, and the job slows to a crawl.

## Three principles

- **Does the work.** Runs the tasks it is sent.
- **One machine each.** Spread across the cluster.
- **Keeps data.** In memory or on disk between tasks.

## Related terms

- [Apache Spark](https://learn.benibix.com/term/data/apache-spark)
- [Spark Driver](https://learn.benibix.com/term/data/spark-driver)
- [Data Shuffle](https://learn.benibix.com/term/data/data-shuffle)
- [Data Skew](https://learn.benibix.com/term/data/data-skew)
- [Databricks Compute](https://learn.benibix.com/term/data/databricks-compute)
- [Airflow Executor](https://learn.benibix.com/term/data/airflow-executor)

## Told apart from

- [Spark Driver vs Spark Executor](https://learn.benibix.com/compare/spark-driver-vs-spark-executor): People think the Spark driver does the heavy work, and it does not. The driver runs your code, plans the job and hands out tasks. The executors on other machines do the reading and the adding up. Only the final answer comes back to the driver. That is why pulling a huge table into it makes it crash.

![A highlighted driver at the top runs your code and splits it into tasks. Arrows marked tasks lead down to three executors. Each executor holds its own slice of the big table and a highlighted copy of the small table. A note says that in a broadcast join each executor gets the small table and joins locally, so the big table never moves between machines.](https://learn.benibix.com/diagrams/dat3-spark-cluster.svg)

---

Source: https://learn.benibix.com/term/data/spark-executor
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
