Preference Testing vs A/B Testing
People think a preference test is an A/B test, and it is not. A preference test asks people which design they like, and why. An A/B test shows each design to real visitors on a live site and counts what they do. One measures what people say. The other measures what they do, and the two can disagree.
Preference Testing
People see two or more designs and pick the one they like. Then they say why.
Preference testing asks people which of two or more designs they like best. Then it asks why. It is quick and works on still pictures, so no working product is needed. It helps pick a look, a logo or a layout early on. But it measures what people say they like, not what helps them get things done.
A/B Test
Two versions, shown to two random groups. You keep the one that does better.
An A/B test shows two versions of something to two random groups at the same time. Version A is the current one, and version B has one change. You then compare which one gets more of the result you want, such as sign-ups. Enough people must take part, or the winner may be luck. A statistics check tells you if the gap is real.
In one line
- Preference Testing
- People see two or more designs and pick the one they like. Then they say why.
- A/B Test
- Two versions, shown to two random groups. You keep the one that does better.
These live in different corners of the glossary (Design and Marketing), which is exactly why they get confused.