Skip to content
BENIBIX Learn

Watermark (Stream Processing)

Data⚙️ ProcessingEvent Time and Windows

Also known as Watermark, Event-Time Watermark

A line that says no older events are expected now.

Think of it like

It is like a teacher saying the test is closed. Papers that come in after that are marked late.

Example

With a 5-minute watermark, the 10:00 to 10:05 window closes once an event stamped 10:10 has been seen.

Why it matters

Without it, a stream would never know when to give a final count. It sets the trade between being right and being fast.

Definition

A watermark is a stream's guess of how far event time has got. It says no more events older than this time are expected. When the watermark passes the end of a window, the window is closed and its result is sent. A watermark is often set as the newest event time minus a delay, such as five minutes. A longer delay catches more late events but sends results later.

Example code

sql
CREATE TABLE clicks (
  page STRING,
  ts   TIMESTAMP(3),
  WATERMARK FOR ts AS ts - INTERVAL '5' SECOND
) WITH ('connector' = 'kafka', ...);
In Flink SQL, the watermark trails the newest event time by five seconds.
Event time is when it happened. The watermark says how far the stream is sure it has seen.

Common mistake

Setting the delay too short. Many real events arrive after it and are thrown away.

Key takeaways

  1. 1
    How far event time got. A best guess.
  2. 2
    Closes windows. Then results are sent.
  3. 3
    Delay is a trade. More right or more fast.