🗓️Airflow XCom
Also known as XCom, Cross-Communication
A small note one Airflow task leaves for another. It might be a file name or a count.
Think of it like
It is like a sticky note left on the fridge for the next person. It holds a few words, not a whole book.
Example
An extract task writes a file to the cloud and pushes its path. The load task pulls the path and reads the file from there.
Why it matters
Tasks may run on different machines, so they cannot share memory. XCom is the safe way to hand over a pointer to the data.
Definition
Example code
python
@task
def count_rows():
return 1250 # pushed to XCom
@task
def report(n):
print(f"{n} rows") # pulled from XCom
report(count_rows())Common mistake
Key takeaways
- 1Small values. IDs, paths and counts, not data.
- 2Push and pull. One task writes, a later one reads.
- 3Kept in the database. Stored with the run by default.