Mathematics.

limit theorems

Almost Sure Convergence

Probability40 minDifficulty7 out of 10

You should know: convergence in probability

Overview

Almost sure convergence (also called strong convergence or convergence with probability 1) is the strongest of the classical modes of convergence for sequences of random variables. A sequence Xₙ converges almost surely to X if the set of sample paths on which the sequence fails to converge has probability zero. It is the probabilistic analogue of pointwise convergence almost everywhere in measure theory. The strong law of large numbers — which guarantees that sample averages converge to the true mean on every 'typical' sample path — is the most celebrated example.

Intuition

Imagine running infinitely many repeated experiments and recording the full infinite sequence of outcomes for each 'scenario'. Convergence in probability says: for large n, the fraction of scenarios where Xₙ is far from X is small. Almost sure convergence is stronger: it says that the set of scenarios where the sequence ever again deviates from X — for large enough n — has probability zero. The sequence must eventually settle down to X on essentially every scenario, not just on most scenarios for each fixed n.

Formal Definition

Definition

A sequence X₁, X₂, ... of random variables converges almost surely to X, written Xₙ →ᵃ·ˢ· X, if:

P ⁣(lim⁡n→∞Xn=X)=1P\!\left(\lim_{n \to \infty} X_n = X\right) = 1
Almost sure convergence (probability-1 convergence)
P ⁣({ω:Xn(ω)→X(ω)})=1P\!\left(\{\omega : X_n(\omega) \to X(\omega)\}\right) = 1
Equivalent sample-path form
Xn→a.s.X  ⟹  Xn→PXX_n \xrightarrow{\text{a.s.}} X \implies X_n \xrightarrow{P} X
a.s. convergence implies convergence in probability
Xn→a.s.X  ⟺  P ⁣(lim sup⁡n→∞∣Xn−X∣>ε)=0∀ ε>0X_n \xrightarrow{\text{a.s.}} X \iff P\!\left(\limsup_{n\to\infty}|X_n - X| > \varepsilon\right) = 0 \quad \forall\, \varepsilon > 0
Characterisation via limsup

Worked Examples

  1. 1

    By the Borel–Cantelli lemma, check whether Σ P(Xₙ = 1) converges.

    ∑n=1∞P(Xn=1)=∑n=1∞1n=∞\sum_{n=1}^{\infty} P(X_n = 1) = \sum_{n=1}^{\infty} \frac{1}{n} = \infty
  2. 2

    Since the series diverges, the first Borel–Cantelli lemma does not guarantee a.s. convergence (and in fact the events are independent, so by the second Borel–Cantelli, Xₙ = 1 infinitely often a.s.).

  3. 3

    Therefore Xₙ does NOT converge to 0 almost surely, even though P(Xₙ = 1) = 1/n → 0 (convergence in probability holds).

✓ Answer

No — Xₙ converges in probability to 0 but not almost surely to 0.

Practice Problems

Mediumfree response

State the relationship between almost sure convergence and convergence in probability. Give a counterexample showing the converse fails.

Mediumfree response

Let Xₙ = 1/n for all ω. Does Xₙ converge almost surely? To what limit?

Quiz

Almost sure convergence means Xₙ → X:
Which convergence mode is stronger?
The strong law of large numbers states that the sample mean converges to μ:

Summary

  • Almost sure convergence: P(lim Xₙ = X) = 1 — the sequence converges on every sample path except a set of measure zero.
  • It implies convergence in probability, which implies convergence in distribution.
  • The Borel–Cantelli lemma is the main tool for establishing or ruling out almost-sure convergence.
  • The strong law of large numbers is the canonical example: sample means converge a.s. to the true mean.

References

  1. WebsiteWikipedia — Convergence of random variables