Skip to content
BENIBIX Learn

Late-Arriving Data

Data⚙️ ProcessingEvent Time and Windows

Also known as Late Data, Late Events, Late-Arriving Events

Events that come in after their window has closed.

Think of it like

It is like a guest who comes after the cake is cut. You can save a slice, or say the party is over.

Example

A game counts plays per hour. A player's phone was in flight mode, so 30 plays arrive three hours late and go to a side table.

Why it matters

Every real stream has late data. What you do with it decides if a report is right or quietly wrong.

Definition

Late data is an event that reaches the system after its window was closed by the watermark. Phones that were offline, slow networks and retries all cause it. A stream can drop it or send it to a side table. It can also reopen the window and send a fixed result. Batch jobs face the same problem when a day's data keeps coming in after the job ran.
Event time is when it happened. The watermark says how far the stream is sure it has seen.

Common mistake

Dropping late events without counting them. You never learn how much data you are losing.

Key takeaways

  1. 1
    After the window. The watermark has passed.
  2. 2
    Drop, park or fix. Pick on purpose.
  3. 3
    Count it. Know how much is late.