Mathematics.

limit theorems

Convergence in Probability

Probability35 minDifficulty6 out of 10

You should know: random variables, law of large numbers

Overview

Convergence in probability is a mode of convergence for sequences of random variables. A sequence Xₙ converges in probability to a limit X if, for every fixed tolerance ε > 0, the probability that Xₙ and X differ by more than ε tends to zero as n → ∞. This is weaker than almost-sure convergence (which requires the sequence to converge on every sample path except a set of measure zero) but stronger than convergence in distribution. The weak law of large numbers is the prototype example: the sample mean converges in probability to the true mean.

Intuition

Think of Xₙ as a sequence of noisy measurements of a true quantity X. Convergence in probability says: for any tolerance band ε > 0 around X, the chance that your measurement falls outside that band goes to zero. You may still have occasional wild outliers even for large n, but they become increasingly rare. This differs from almost-sure convergence, which requires every individual sample path to eventually stay inside the band — a much stricter demand.

Formal Definition

Definition

A sequence of random variables X₁, X₂, ... converges in probability to a random variable X, written Xₙ →ᴾ X, if for every ε > 0:

lim⁡n→∞P(∣Xn−X∣>ε)=0∀ ε>0\lim_{n \to \infty} P(|X_n - X| > \varepsilon) = 0 \quad \forall\, \varepsilon > 0
Definition of convergence in probability
Xn→PXX_n \xrightarrow{P} X
Notation
Xn→a.s.X  ⟹  Xn→PXX_n \xrightarrow{\text{a.s.}} X \implies X_n \xrightarrow{P} X
Almost sure convergence implies convergence in probability
Xn→PX  ⟹  Xn→dXX_n \xrightarrow{P} X \implies X_n \xrightarrow{d} X
Convergence in probability implies convergence in distribution

Worked Examples

  1. 1

    Compute the mean and variance of the sample mean.

    E[Xˉn]=μ,Var⁡(Xˉn)=σ2nE[\bar{X}_n] = \mu, \quad \operatorname{Var}(\bar{X}_n) = \frac{\sigma^2}{n}
  2. 2

    Apply Chebyshev's inequality to X̄ₙ with threshold ε.

    P(∣Xˉn−μ∣>ε)≤Var⁡(Xˉn)ε2=σ2nε2P(|\bar{X}_n - \mu| > \varepsilon) \leq \frac{\operatorname{Var}(\bar{X}_n)}{\varepsilon^2} = \frac{\sigma^2}{n\varepsilon^2}
  3. 3

    As n → ∞, the right-hand side tends to 0 for any fixed ε > 0.

    σ2nε2→0  ⟹  Xˉn→Pμ\frac{\sigma^2}{n\varepsilon^2} \to 0 \implies \bar{X}_n \xrightarrow{P} \mu

✓ Answer

The sample mean converges in probability to μ (the weak law of large numbers).

Practice Problems

Mediumfree response

Let Xₙ have P(Xₙ = 1) = 1/n and P(Xₙ = 0) = 1 − 1/n. Does Xₙ converge in probability to 0? Justify.

Mediumfree response

If Xₙ →ᴾ X and Yₙ →ᴾ Y, does Xₙ + Yₙ →ᴾ X + Y?

Quiz

Convergence in probability of Xₙ to X means:
Which implication is TRUE?
The weak law of large numbers is an example of:

Summary

  • Xₙ →ᴾ X means P(|Xₙ − X| > ε) → 0 for every ε > 0.
  • It is weaker than almost-sure convergence but stronger than convergence in distribution.
  • The weak law of large numbers is the canonical example.
  • Chebyshev's inequality is a standard tool for establishing convergence in probability from moment information.

References

  1. WebsiteWikipedia — Convergence of random variables