🔬P-Hacking
Also known as Data Dredging, Fishing for Significance, Significance Chasing
Doing test after test, then showing just the one that came out the way you hoped.
Think of it like
It is like a dart game. You draw the target after you throw.
Example
A team finds no effect of a diet. They drop some people, split by age, then by sex, and at last find one group where p is 0.04. The paper reports only that group.
Why it matters
It is one reason many published findings fail to repeat. Writing down the plan before the data comes in guards against it.
Definition
Common mistake
Key takeaways
- 1Many tries. Each test adds a chance of a fluke.
- 2One reported. The misses stay hidden.
- 3Plan first. Write down the analysis before you look.