Skip to content
BENIBIX Learn

Airflow TaskFlow API

Data🗓️ OrchestrationApache Airflow

Also known as TaskFlow API, TaskFlow, Airflow @task Decorator

Airflow tasks written as plain Python code, with @task on top.

Think of it like

It is like passing a note straight to the next person in line. Nobody has to post it on a board first.

Example

Lucy writes extract and load as two functions with @task. She writes load(extract()), and Airflow runs extract first and hands its rows to load.

Why it matters

It makes a DAG read like normal Python. Passing data between tasks takes one line, not a pair of XCom calls.

Definition

The TaskFlow API is a way to write Airflow tasks as plain Python functions. You put @task above a function, and it becomes a task. When one task's function returns a value, the next task gets it as an input. Airflow stores the value as an XCom for you, so you never call XCom by hand. Calling one function inside another also sets the order of the tasks.

Example code

python
@task
def extract():
    return {"1001": 301.27}

@task
def load(rows):
    print(rows)

load(extract())
Two functions become two tasks. The value extract returns is handed to load.
With @task, each function is a task. The value extract returns is passed on to load.

Common mistake

Returning a whole table from a task. The value is kept in Airflow's database, so pass a file path.

Key takeaways

  1. 1
    Plain functions. @task turns one into a task.
  2. 2
    Values flow on. Return values become inputs.
  3. 3
    Order from calls. Calling a task sets the order.