⚙️Apache Beam
Also known as Beam, Beam SDK
Write a pipeline once. Run it on Flink, Spark or Dataflow.
Think of it like
It is like writing a song once. Any band can then play it. The words stay the same.
Example
A team writes a Beam pipeline that cleans sensor data. It runs on Dataflow in the cloud and on Flink in their own data centre.
Why it matters
It keeps a pipeline from being tied to one engine. Its model shaped how streaming is taught.
Definition
Common mistake
Key takeaways
- 1One model. Batch and stream alike.
- 2Many runners. Flink, Spark, Dataflow.
- 3Windows and triggers. Built into the model.