At-Least-Once vs Exactly-Once
People think exactly-once means each message travels only once. It does not. Under at-least-once, a message that is sent again is counted again. Under exactly-once, it may still be sent twice, but the copy is spotted by its ID and has no effect.
At-Least-Once Delivery
Every message comes, but some may come twice.
At-least-once delivery means every message arrives, but some may arrive more than once. The sender keeps trying until it gets a reply. If the reply is lost, it sends the same message again. Most streaming tools work this way by default. The reader has to cope with copies, often by making its work idempotent.
Exactly-Once Semantics
Each message counts one time, even if it is sent twice.
Exactly-once semantics means each message changes the result one time only, even when sends are retried. The message may still travel twice, but the copy has no effect. Each message gets an ID. The result and the read position are saved together. Kafka offers it through safe producers and transactions. It is slower and harder to set up than the other two.
In one line
- At-Least-Once Delivery
- Every message comes, but some may come twice.
- Exactly-Once Semantics
- Each message counts one time, even if it is sent twice.