Mathematics.

inferential statistics

The p-value

Statistics25 minDifficulty3 out of 10

You should know: hypothesis testing

Overview

The p-value is the probability, computed under the assumption that the null hypothesis is true, of observing a test statistic at least as extreme as the one actually observed. It is not the probability that the null hypothesis is true, nor the probability that the results happened by chance — a common and important misconception. A small p-value indicates the observed data would be unusual if the null hypothesis were true, providing evidence against it. Researchers typically compare the p-value to a pre-chosen significance level α (commonly 0.05): if p ≤ α, the result is called statistically significant and the null hypothesis is rejected.

Intuition

Think of the p-value as answering: 'If nothing interesting were actually going on (H₀ is true), how surprising would data like this be?' A p-value of 0.01 means that under the null hypothesis, results this extreme (or more so) would occur only 1% of the time by chance — strong evidence that something other than pure chance produced the data. A large p-value, like 0.6, means the observed data is quite ordinary under the null hypothesis, giving no reason to doubt it.

Formal Definition

Definition

For an observed test statistic t_obs computed from the data, under the null hypothesis H₀, the p-value for a two-tailed test is:

p=P(∣T∣≥∣tobs∣∣H0 true)p = P(|T| \ge |t_{\text{obs}}| \mid H_0 \text{ true})
Two-tailed p-value
p=P(T≥tobs∣H0 true)p = P(T \ge t_{\text{obs}} \mid H_0 \text{ true})
One-tailed (upper) p-value
Reject H0 if p≤α\text{Reject } H_0 \text{ if } p \le \alpha
Decision rule at significance level α

Worked Examples

  1. For a one-tailed upper test, the p-value is P(Z ≥ 2.0).

    p=P(Z≥2.0)p = P(Z \ge 2.0)

  2. From the standard normal table, P(Z ≤ 2.0) ≈ 0.9772, so P(Z ≥ 2.0) = 1 − 0.9772 = 0.0228.

    p≈0.0228p \approx 0.0228

Answer: p ≈ 0.0228.

Practice Problems

Difficulty 3/10

A one-tailed test gives z = 1.5. Find the p-value (upper tail).

Difficulty 3/10

A two-tailed test yields a one-tailed p-value of 0.02. What is the two-tailed p-value, and is it significant at α = 0.05?

Difficulty 5/10

A study reports p = 0.03 for a new drug's effect on blood pressure. Explain what this p-value does and does NOT mean.

Quiz

The p-value is best described as:
If p = 0.002 and α = 0.05, the appropriate conclusion is:
A common misinterpretation of the p-value is:

Summary

  • The p-value is the probability of observing a test statistic as extreme or more extreme than what was observed, assuming H₀ is true.
  • A small p-value (≤ α, commonly 0.05) is evidence against the null hypothesis and leads to rejecting it.
  • The p-value is NOT the probability that the null hypothesis is true — it is a conditional probability given H₀.

References

  1. WebsiteWikipedia — P-value