Probability Measure — Mathematics Knowledge System

Overview

A probability measure is a measure P on a measurable space (Omega, F) satisfying P(Omega) = 1. The triple (Omega, F, P) is called a probability space. This is Kolmogorov's axiomatic foundation for probability theory (1933), which unified all of classical probability within the measure-theoretic framework. Random variables become measurable functions, expectations become Lebesgue integrals, and convergence concepts (almost sure, in probability, in L^p) are precisely defined.

Intuition

A probability space (Omega, F, P) models a random experiment: Omega is the set of all possible outcomes, F tells you which events (subsets of outcomes) are observable, and P assigns each observable event a probability between 0 and 1. The normalisation P(Omega) = 1 (certainty that some outcome occurs) is the only extra condition that distinguishes probability from general measure theory.

Formal Definition

Definition

A probability space is a measure space (Omega, F, P) where P is a probability measure.

P : \mathcal{F} \to [0,1]

Probability measure maps events to [0,1]

P(\Omega) = 1

Normalisation — the whole sample space has probability 1

A_1, A_2, \ldots \in \mathcal{F} \text{ pairwise disjoint} \implies P\!\left(\bigsqcup_{n=1}^{\infty}A_n\right) = \sum_{n=1}^{\infty}P(A_n)

Countable additivity (sigma-additivity)

X : (\Omega, \mathcal{F}) \to (\mathbb{R}, \mathcal{B}(\mathbb{R})) \text{ measurable} \implies X \text{ is a random variable}

Random variable = measurable function

\mathbb{E}[X] = \int_\Omega X\, dP

Expectation = Lebesgue integral w.r.t. P

Notation

Notation	Meaning
$(\Omega, \mathcal{F}, P)$	Probability space
$P(A)$	Probability of event A
$P(A \mid B)$	Conditional probability
$\mathbb{E}[X]$	Expectation of random variable X
$P\text{-a.s.}$	Almost surely — with probability 1

Properties

Complementation

P(A^c) = 1 - P(A)

Inclusion–exclusion

P(A \cup B) = P(A) + P(B) - P(A \cap B)

Monotonicity

A \subseteq B \implies P(A) \le P(B)

Union bound (Boole's inequality)

P\!\left(\bigcup_{n=1}^\infty A_n\right) \le \sum_{n=1}^\infty P(A_n)

Continuity

A_n \nearrow A \implies P(A_n) \nearrow P(A);\quad A_n \searrow A \implies P(A_n) \searrow P(A)

Condition: Second follows since P(Omega) = 1 < ∞

Worked Examples

The sample space is Omega = {H, T}.
$\Omega = \{H, T\}$
The sigma-algebra is the power set F = {empty, {H}, {T}, {H,T}}.
$\mathcal{F} = 2^\Omega = \{\emptyset, \{H\}, \{T\}, \{H,T\}\}$
Define P: P(empty) = 0, P({H}) = 1/2, P({T}) = 1/2, P(Omega) = 1.
$P(\{H\}) = P(\{T\}) = \tfrac{1}{2},\quad P(\Omega) = 1$
Verify sigma-additivity: P({H} union {T}) = P({H}) + P({T}) = 1/2 + 1/2 = 1 = P(Omega). All axioms are satisfied.

Answer: ({H,T}, 2^{H,T}, P) with P({H}) = P({T}) = 1/2 is a valid probability space.

Practice Problems

Difficulty 5/10

Prove that for any event A, P(A^c) = 1 - P(A).

Difficulty 7/10

Prove the Borel–Cantelli Lemma: if sum P(A_n) < ∞, then P(limsup A_n) = 0.

Difficulty 7/10

Let X be a non-negative random variable with E[X] < ∞. Use the Markov inequality to bound P(X >= t) for t > 0.

Common Mistakes

Common Mistake

Probability 0 means impossible and probability 1 means certain

In measure theory, P(A) = 0 means A is a null event but it can still occur. For example, choosing a specific rational from [0,1] at random has probability 0 but is not impossible.

Common Mistake

Mutually exclusive events are independent

Mutually exclusive events A, B with P(A), P(B) > 0 are actually dependent: knowing A occurred means B definitely did not.

Common Mistake

Pairwise independence implies mutual independence

Three events can be pairwise independent but not mutually independent. Mutual independence requires P(A_i ∩ A_j ∩ A_k) = P(A_i)P(A_j)P(A_k) for all triples, not just P(A_i ∩ A_j) = P(A_i)P(A_j) for all pairs.

Quiz

Historical Background

Before Kolmogorov, probability theory was a collection of techniques without a rigorous foundation. Andrei Kolmogorov's 1933 monograph 'Grundbegriffe der Wahrscheinlichkeitsrechnung' (Foundations of the Theory of Probability) placed probability theory on a measure-theoretic footing, ending a century of foundational ambiguity. The sigma-algebra formalism resolves classical paradoxes about conditional probability and allows a unified treatment of discrete and continuous probability.

1933
Kolmogorov axiomatises probability theory via measure spaces
Andrei Kolmogorov
1940s
Doob develops martingale theory using conditional expectation as a Radon-Nikodym derivative
Joseph Doob
1950s
Ito develops stochastic calculus on probability spaces
Kiyosi Ito

Summary

A probability space (Omega, F, P) is a measure space with P(Omega) = 1.
Random variables are measurable functions; expectations are Lebesgue integrals with respect to P.
Kolmogorov's axioms (1933) provide the rigorous foundation unifying discrete and continuous probability.
Key identities: P(A^c) = 1 - P(A), inclusion–exclusion, union bound (Boole's inequality).
Independence of events means P(A ∩ B) = P(A)P(B); this extends naturally to sigma-algebras and random variables.

References

BookKolmogorov, A.N. — Foundations of the Theory of Probability (1933, English transl. 1956)
BookBillingsley, P. — Probability and Measure, 3rd ed. (1995)
WebsiteWikipedia — Probability space