Skip to content
BENIBIX Learn

Airflow Connection

Data🗓️ OrchestrationApache Airflow

Also known as Connection (Airflow), conn_id, AIRFLOW_CONN

A saved set of login details for an outside system. Tasks find it by name.

Think of it like

It is like a saved contact on your phone. You tap a name, and the phone knows the number.

Example

The shop's database is saved as a connection named shop_db. Every DAG that reads orders asks for shop_db. When the password changes, it changes in one place.

Why it matters

Passwords stay out of the code and out of git. One change to a connection fixes every DAG that uses it.

Definition

An Airflow connection stores the details for reaching an outside system. It holds the host, the login, the password and the type of system. Each one has a short name, called its conn_id. Tasks and hooks ask for a connection by that name. So no password is written in the DAG file. Connections can be set in the web page, from the command line or in a variable named AIRFLOW_CONN_ and the name.

Example code

bash
export AIRFLOW_CONN_SHOP_DB='postgres://etl:secret@db.example.com:5432/shop'
A connection named shop_db, set in a variable. No DAG file holds it.
The connection on the left holds the login. The hook takes it from there to reach the database.

Common mistake

Writing the password straight into the DAG file. Anyone who can read the code can then read the password.

Key takeaways

  1. 1
    Stored once. Host, login and password.
  2. 2
    Found by name. Tasks ask for the conn_id.
  3. 3
    Out of the code. No secrets in DAG files.