# Spark Structured Streaming

> Spark's way to process streams, using the same code as batch jobs.

**Field:** Data Engineering  
**Category:** Processing  
**Also known as:** Structured Streaming, Spark Streaming

## Think of it like

It is like a bus that leaves every few seconds. Each bus takes whoever is at the stop.

## Definition

Spark Structured Streaming is the part of Apache Spark that processes streams. You write the same DataFrame code you would write for a table. Spark runs it again and again on the new data, in small micro-batches by default. It supports event time, watermarks and exactly-once output to many sinks. A batch job can often become a stream with a few changes.

## Example

A team already runs Spark batch jobs. They turn the nightly sales job into a stream that updates every 30 seconds.

## Example code

```python
(clicks
  .withWatermark("ts", "10 minutes")
  .groupBy(F.window("ts", "5 minutes"), "page")
  .count())
```

Count clicks per page in 5-minute windows, and wait up to 10 minutes for late events.

## Why it matters

One engine and one codebase serve both batch and stream. Teams on Spark need no new tool.

## Common mistake

Expecting a reply in a few thousandths of a second. Small batches add some delay, often a second or more.

## Three principles

- **Same code as batch.** DataFrames over a stream.
- **Micro-batches.** Small batches, run again and again.
- **Event time built in.** Windows and watermarks.

## Related terms

- [Apache Spark](https://learn.benibix.com/term/data/apache-spark)
- [Micro-Batching](https://learn.benibix.com/term/data/micro-batching)
- [Apache Flink](https://learn.benibix.com/term/data/apache-flink)
- [Watermark (Stream Processing)](https://learn.benibix.com/term/data/watermark)
- [Stream Processing](https://learn.benibix.com/term/data/stream-processing)

## Told apart from

- [Apache Flink vs Spark Structured Streaming](https://learn.benibix.com/compare/apache-flink-vs-spark-structured-streaming): People think Apache Flink and Spark Structured Streaming do the same thing at different speeds, and that is close but not all of it. Flink handles each event as it arrives, in milliseconds. Spark gathers events into tiny batches and runs the same code it uses for batch jobs. Pick Flink for speed and big state, and Spark to reuse what your team knows.

---

Source: https://learn.benibix.com/term/data/spark-structured-streaming
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
