Mathematics.

inferential statistics

Type I and Type II Errors

Statistics20 minDifficulty3 out of 10

You should know: hypothesis testing

Overview

In hypothesis testing, a Type I error occurs when the null hypothesis is rejected even though it is actually true — a 'false positive.' A Type II error occurs when the null hypothesis is not rejected even though it is actually false — a 'false negative.' The probability of a Type I error is denoted α (the significance level, chosen by the researcher, commonly 0.05), while the probability of a Type II error is denoted β. The power of a test, 1 − β, is the probability of correctly rejecting a false null hypothesis. There is an inherent tradeoff: for a fixed sample size, decreasing α (to reduce Type I errors) typically increases β (more Type II errors), and vice versa.

Intuition

Think of a medical test for a disease. A Type I error is a false alarm: the test says 'disease' when the patient is actually healthy. A Type II error is a missed diagnosis: the test says 'healthy' when the patient actually has the disease. Making the test more cautious about false alarms (lowering α, requiring stronger evidence to declare 'disease') tends to make it more likely to miss real cases (raising β) — this is the fundamental tradeoff between the two error types.

Formal Definition

Definition

Given a null hypothesis H₀ and a decision to reject or not reject it based on sample data:

α=P(reject H0∣H0 true)\alpha = P(\text{reject } H_0 \mid H_0 \text{ true})
Type I error rate (significance level)
β=P(fail to reject H0∣H0 false)\beta = P(\text{fail to reject } H_0 \mid H_0 \text{ false})
Type II error rate
Power=1−β=P(reject H0∣H0 false)\text{Power} = 1 - \beta = P(\text{reject } H_0 \mid H_0 \text{ false})
Power of the test

Worked Examples

  1. By definition, α is exactly the probability of a Type I error when H₀ is true.

    P(Type I error)=α=0.05P(\text{Type I error}) = \alpha = 0.05

Answer: 0.05 (5%).

Practice Problems

Difficulty 2/10

A test uses α = 0.01. What is the probability of a Type I error?

Difficulty 3/10

If β = 0.35 for a test, what is its power?

Difficulty 5/10

A jury trial can be viewed as a hypothesis test with H₀: defendant is innocent. Describe what a Type I error and a Type II error represent in this context.

Quiz

A Type I error occurs when:
The power of a test, 1 − β, represents:
For a fixed sample size, decreasing α (making the test more conservative) generally:

Summary

  • Type I error (false positive): rejecting a true null hypothesis, with probability α.
  • Type II error (false negative): failing to reject a false null hypothesis, with probability β.
  • Power (1 − β) measures a test's ability to detect a real effect; there's an inherent tradeoff between α and β at fixed sample size.

References

  1. WebsiteWikipedia — Type I and type II errors