Skip to content
BENIBIX Learn

A/B Test

Marketing📐 Attribution & TestingExperiments

Also known as A/B Testing, Split Test, Split Testing

Two versions, shown to two random groups. You keep the one that does better.

Think of it like

It is like baking two cakes with one change. Friends taste both, and you keep the one they like.

Example

A shop tests a green Buy button against its blue one. Over two weeks and 40,000 visits, green gets 4.1% of visitors to buy, against 3.8%.

Why it matters

It swaps opinions for evidence, one change at a time.

Definition

An A/B test shows two versions of something to two random groups at the same time. Version A is the current one, and version B has one change. You then compare which one gets more of the result you want, such as sign-ups. Enough people must take part, or the winner may be luck. A statistics check tells you if the gap is real.
An A/B test is the middle panel: one change, traffic split in two.

Common mistake

Stopping the test the moment one side pulls ahead. Early leads often vanish, so set the run length first.

Key takeaways

  1. 1
    One change. So you know what worked.
  2. 2
    Random groups. Split the traffic fairly.
  3. 3
    Enough people. Small tests crown lucky winners.

Compare

Learn it properly