🗓️Dagster
A free tool to run pipelines. It plans the work around the tables it must build.
Think of it like
Most planners list chores. This one lists the cakes you want, and works out the steps.
Example
A team moves from Airflow to Dagster. Now their page shows each table, when it was last built, and which ones are stale.
Why it matters
People care about tables, not tasks. Planning around assets makes it easy to see what data is fresh and what broke it.
Definition
Example code
python
import dagster as dg
@dg.asset
def raw_orders():
return load_orders()
@dg.asset
def daily_sales(raw_orders):
return raw_orders.groupby("day").sum()Common mistake
Key takeaways
- 1Assets first. Declare the tables you want to exist.
- 2Graph from inputs. Links come from what each asset reads.
- 3Freshness shown. See when each table last updated.