Skip to content
BENIBIX Learn

Pipeline Alerting

Data🗓️ OrchestrationWorkflow Orchestration

Also known as Pipeline Alerts, Failure Alerts, Task Failure Notification

A note to a person when a pipeline breaks or runs late.

Think of it like

It is like a smoke alarm in a kitchen. It shouts before the whole house fills with smoke.

Example

The sales load fails at 03:10. An alert lands in the team chat with the task name and a link to the log. Jack fixes it before the 08:00 report goes out.

Why it matters

A failed pipeline that nobody hears about turns into a wrong report in the morning. An alert turns it into a quick fix.

Definition

Pipeline alerting sends a message to a person when a pipeline goes wrong. The two main triggers are a task that fails and a run that misses its deadline. The message goes to a chat channel, an email or the person on call. A good alert says which task broke, which run it was and where the log is. Too many alerts are as bad as none, because people stop reading them.

Example code

python
@task(on_failure_callback=send_to_chat)
def load_sales():
    ...
If this Airflow task fails, the function send_to_chat runs and posts an alert.
Two things raise an alert: a task that fails, and a run that misses its deadline.

Common mistake

Alerting on every retry. People learn to ignore the channel, and they miss the real failure.

Key takeaways

  1. 1
    Fail or late. Both are worth a message.
  2. 2
    Say where. Task, run and a link to the log.
  3. 3
    Few and real. Noise teaches people to ignore it.

Compare