# Hypothesis Test and P-Value

> How a test asks whether a result could be luck. What a p-value really means, and why a result that passes the test may still not matter.

**Course:** [Probability and Statistics](https://learn.benibix.com/learn/general-math)  
**Time:** 12 min

**After this lesson you can:**

- Walk through the steps of a hypothesis test
- Say what a p-value does and does not mean
- Tell significant from important

## What does a p-value really mean?

A hypothesis test checks whether data shows a real effect or could just be chance. It starts by supposing there is no effect, called the null hypothesis. The p-value is the chance of seeing data at least this odd if that were true. If it falls below a bar set in advance, often 0.05, the result passes. It is then said to have statistical significance.

Take a coin you suspect is unfair. You flip it 100 times and get 60 heads. If the coin were fair, how often would you see 60 or more? About 3 times in 100. That small chance makes it hard to believe the coin is fair.

A p-value of 0.03 does not mean a 3 percent chance that the coin is fair. It means that a fair coin would give a result this lopsided about 3 percent of the time. It is a statement about the data, given no effect, not about the idea itself.

The 0.05 bar means that, when there is truly no effect, about 1 test in 20 will still clear it by luck. Run 20 tests on things that do nothing, and on average one will look like a discovery. That is why a single surprising result needs to be repeated.

Statistical Significance is not the same as importance. With a big enough sample, a tiny effect can be significant. A diet that cuts weight by 100 grams over a year might clear the bar in a study of 50,000 people. It is real, and almost useless.

Not clearing the bar does not prove there is no effect either. A small study may simply be too small to see a real one. The fair conclusion is that this study could not tell.

The common mistake is to run many tests and report only the ones that cleared the bar. That makes chance look like real findings. Good studies say in advance what they will test, and report every result.

![A bell curve showing the results you would expect if there were no real effect. A dashed line on the right marks the 0.05 bar. A solid line further out marks your result. The thin tail of the curve past your result is shaded and marked p-value.](https://learn.benibix.com/diagrams/gen-p-value.svg)

### Example: Twenty jelly bean colours

A team tests whether jelly beans cause spots. Overall, they find nothing. Then they test 20 colours one by one, and green clears the 0.05 bar.

A headline says green jelly beans cause spots. With 20 tests, one passing by luck is just what you would expect.

**Words in this lesson:** [Hypothesis Test](https://learn.benibix.com/term/general/hypothesis-test), [P-Value](https://learn.benibix.com/term/general/p-value), [Statistical Significance](https://learn.benibix.com/term/general/statistical-significance)

**Question:** A study reports p = 0.04. What does that mean?

**Answer:** If there were no effect, data this odd would show up about 4 percent of the time. A p-value is about the data, assuming no effect. It does not give the chance the idea is true, and it says nothing about size.

---

Source: https://learn.benibix.com/learn/general-math/hypothesis-test-and-p-value
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
