Skip to content
BENIBIX Learn

Game Day

Infrastructure📟 ReliabilityResilient Systems

Also known as Failure Drill, Resilience Drill, Disaster Recovery Drill

A planned team drill. A failure is caused on purpose to practise the response.

Think of it like

Think of it like a school fire drill, on a day picked in advance.

Example

Emma's team holds a game day each quarter. This time they switch off the main database, and find the runbook still names a server that was retired.

Why it matters

A real outage is a bad time to learn the runbook is wrong. A drill finds that out on a calm day.

Definition

A game day is a planned session where a team practises a failure together. They pick a fault, such as losing a database, and set it off on purpose at a set time. Everyone works through it as if it were real, using the real alerts and runbooks. Afterwards they write down what was slow or missing. It tests the people and the steps, not just the machines.
The team at the table is a game day. They run the drill together, on a day they chose.

Common mistake

Running it with only the senior engineers. The people who will be paged at night are the ones who need the practice.

Key takeaways

  1. 1
    Planned. The time and the fault are chosen.
  2. 2
    Real tools. Use the real alerts and runbooks.
  3. 3
    Write it up. Fix what was slow or missing.