Skip to content
BENIBIX Learn

Airflow Task

Data🗓️ OrchestrationApache Airflow

Also known as Task Instance, Airflow Task Instance

One step in an Airflow DAG. It runs once in each DAG run.

Think of it like

It is one job on a chore chart, like feeding the cat. Each day it gets its own tick.

Example

In yesterday's run, the load task failed. Jack clears just that task instance, and only it runs again.

Why it matters

Tasks are the size at which Airflow retries, logs and reruns work. Small, clear tasks make failures easy to find and fix.

Definition

An Airflow task is one step inside a DAG, such as load the orders. It is made from an operator or from a Python function with the task decorator. Each time the DAG runs, the task gets a task instance for that run. The instance has a state, such as queued, running, success or failed. You can clear one instance to run just that step again.
One task, many instances. Each run gives the task a fresh instance with its own state.

Common mistake

Packing a whole pipeline into one task. A failure near the end then reruns every step from the start.

Key takeaways

  1. 1
    One step. A single unit of work in a DAG.
  2. 2
    An instance per run. Each run gets its own copy and state.
  3. 3
    Rerun alone. Clear one instance to repeat one step.