Skip to content
BENIBIX Learn

Prefect

Data🗓️ OrchestrationOrchestration Tools

Also known as Prefect Flow, Prefect Cloud

A Python tool to run pipelines. Plain functions become flows that are tracked and retried.

Think of it like

It is like putting stickers on your homework. Each sticker makes a page a step that gets checked.

Example

Emma already has a Python script that pulls prices. She adds two decorators, and it now runs every hour with retries and a log page.

Why it matters

It asks for few changes to code you already have. That makes it a quick way to add retries and tracking to scripts.

Definition

Prefect is an open source orchestrator for Python. You add a flow decorator to a normal function, and a task decorator to its steps. Prefect then tracks each run, retries failed steps and logs what happened. Flows can branch and loop like any Python code, so they need no fixed graph. You can run them on a schedule or by an event. A paid cloud version runs the control side for you.

Example code

python
from prefect import flow, task

@task(retries=3, retry_delay_seconds=60)
def fetch_orders():
    ...

@flow
def nightly_load():
    orders = fetch_orders()
Two lines turn plain Python into a Prefect flow. Its task gets three more tries.
In Prefect, this graph comes from a Python function calling its tasks. It is drawn as the flow runs.

Common mistake

Expecting it to work like Airflow. Prefect has no fixed DAG file, so the graph only shows up after a run.

Key takeaways

  1. 1
    Plain Python. Decorators on normal functions.
  2. 2
    Dynamic flows. Branches and loops are allowed.
  3. 3
    Tracked runs. Logs, retries and state for each step.

Compare