Skip to content
BENIBIX Learn

Kappa Architecture

Data🧭 FoundationsData Pipelines

Also known as Kappa

One stream does all the work. Old data is played again when rules change.

Think of it like

It is like a video you can rewind. To fix a mistake, you play the tape again from the start.

Example

A team changes how it counts active users. It starts the new job at the oldest event in the log and lets it catch up.

Why it matters

It removes the double code of Lambda. One job means one place for bugs and one place to fix them.

Definition

Kappa architecture uses only a stream, with no separate batch path. Every event is kept in a log that can be read again. Live data flows through one stream job. When the logic changes, you replay the old events through the new job. So there is one set of code to write and fix.
Lambda keeps two paths. Kappa keeps one stream and replays it when the logic changes.

Common mistake

Not keeping events long enough. If the log only holds seven days, you cannot replay a year.

Key takeaways

  1. 1
    Stream only. No separate batch path.
  2. 2
    Keep the log. Old events can be read again.
  3. 3
    Replay to fix. New logic reruns over old events.

Compare

Learn it properly