Skip to content
BENIBIX Learn

A/A Test

Marketing📐 Attribution & TestingExperiments

Also known as A/A Testing

A test of a page against an exact copy of itself. It checks the test tool works.

Think of it like

It is like weighing one apple on two scales. If they do not match, a scale is broken.

Example

A team sets up a new testing tool and splits a page into two copies. One copy shows 12% more sales, and they find a tracking bug before any real test.

Why it matters

A broken setup makes every later A/B test wrong. An A/A test catches it cheaply.

Definition

An A/A test shows the very same version to two random groups. Since nothing differs, the results should come out about the same. If they do not, something is wrong with the test setup. It may be the way traffic is split or the way results are counted. Teams run one before they trust a new testing tool.
The left panel is an A/A test: the same page twice, so any big gap points to a broken setup.

Common mistake

Panicking at a small gap. By chance, about 1 in 20 A/A tests looks like a win at the usual 95% bar.

Key takeaways

  1. 1
    Same page twice. Nothing should differ.
  2. 2
    Tests the test. Finds bugs in the setup.
  3. 3
    Expect some noise. Small gaps happen by chance.