📐Holdout Group
Also known as Holdout, Holdout Test
People who do not see a campaign, on purpose. They show what it really added.
Think of it like
It is like keeping a few plants away from the plant food. Then you can see what the food really did.
Example
An app sends a welcome series to 90% of new users and holds back 10%. After a month, 31% of the emailed group pay, against 28% of the holdout.
Why it matters
Without one, you cannot tell if a campaign changed a thing. It may just have taken the credit.
Definition
Common mistake
Key takeaways
- 1Picked at random. Both groups must start alike.
- 2Kept out. They see none of the campaign.
- 3Big enough. Small groups give noisy answers.