Skip to content
BENIBIX Learn

Task Retry

Data🗓️ OrchestrationWorkflow Orchestration

Also known as Retries, Retry Policy, Retry Delay

A failed task runs again on its own. It gets a set number of tries.

Think of it like

The ball missed the hoop, so you take another shot. After three misses, you ask for help.

Example

A load hits a timeout at 2 am because the database is busy. Five minutes later the retry works, and no one is woken up.

Why it matters

Most night-time failures are brief. Retries turn them into a short delay instead of a broken morning report.

Definition

A task retry runs a failed task again by itself. You set how many tries it gets and how long to wait between them. Many failures pass on their own, like a busy database or a short network drop. So a second try often works. When the last try fails, the task is marked failed and someone is told. Tasks after it do not run.

Example code

python
@task(
    retries=3,
    retry_delay=timedelta(minutes=5),
    retry_exponential_backoff=True,
)
def load_orders():
    ...
An Airflow task that gets three more tries. The wait grows longer after each failure.
The join task failed and is on its second retry. Build report waits until it works or runs out of tries.

Common mistake

Retrying a task that adds rows each time it runs. Each retry leaves another copy of the same data.

Key takeaways

  1. 1
    Count. Set how many extra tries it gets.
  2. 2
    Wait. Leave a gap, often growing longer.
  3. 3
    Then alert. Tell a person after the last try.