Skip to content
BENIBIX Learn

Flaky Test

Development🧪 Testing & QualityTest Practices

Also known as Flaky Tests, Flakiness, Intermittent Test

A test that passes on one run and fails the next, with no code change.

Think of it like

Think of it like a smoke alarm that goes off at random. Soon nobody trusts it.

Example

Lucy's checkout test fails about one run in ten. It waits 500 milliseconds for the page, and on a busy build server that is not enough.

Why it matters

One flaky test makes the whole suite less trusted. People start pressing rerun instead of reading the failure.

Definition

A flaky test passes sometimes and fails other times, with no change to the code. Common causes are timing, tests that depend on each other's order, and calls to the real network. A test that waits a fixed half second for a page can fail when the machine is busy. Flaky tests teach people to ignore red results. That is how a real failure slips through.

Common mistake

Fixing a flaky test by adding a longer sleep. Wait for the thing you need, like the button showing up.

Key takeaways

  1. 1
    Same code, different result. Pass, then fail.
  2. 2
    Timing, order, network. The usual causes.
  3. 3
    Fix it fast. Or nobody trusts red.