Airflow DAG vs DAG Run
People say DAG when they mean one run of it, and the two are not the same. An Airflow DAG is the pipeline itself: a Python file with tasks, arrows and a schedule. A DAG run is one time that pipeline ran, tied to one date of data. One daily DAG makes 365 runs a year, and each run can pass or fail on its own.
Airflow DAG
One Airflow pipeline. It is a Python file with tasks, arrows and a time to run.
An Airflow DAG is one pipeline, written as a Python file. DAG stands for directed acyclic graph: tasks joined by one-way arrows, with no loops. The file names the tasks and the arrows between them. It also sets the schedule and the start date. Airflow reads the file and shows the graph on its web page.
Airflow DAG Run
One run of an Airflow DAG. It is tied to the date of its data.
A DAG run is one run of an Airflow DAG, from its first task to its last. Each run has a logical date, which says which slice of time it covers. A daily run for 3 March covers that day's data, even if it starts the next morning. Every task in the DAG gets its own task instance inside the run. A run can be started by the schedule, by a person or by a backfill.
In one line
- Airflow DAG
- One Airflow pipeline. It is a Python file with tasks, arrows and a time to run.
- Airflow DAG Run
- One run of an Airflow DAG. It is tied to the date of its data.