Stein's Method — Mathematics Knowledge System

Overview

Stein's method is a powerful probabilistic technique for bounding the distance between probability distributions, particularly between a distribution of interest and a reference distribution such as the normal or Poisson. Developed by Charles Stein in the 1970s, it provides explicit, often optimal, error bounds in limit theorems — going beyond the classical CLT to say how fast the convergence happens and quantifying the error. Unlike Fourier/characteristic function methods, Stein's method works without independence assumptions and can handle dependency structures such as local dependence, exchangeability, and size-bias couplings.

Intuition

The key insight behind Stein's method: a random variable Z is standard normal iff E[Zf(Z)] = E[f'(Z)] for all smooth f. If W is approximately normal, then E[Wf(W) − f'(W)] is approximately zero. To bound how close W is to normal for a test function h, one solves the 'Stein equation' f' − wf = h − E[h(Z)], expresses the distance as E[f'(W) − Wf(W)], and bounds this remainder using information about W (moments, dependency structure). The power is that the method avoids characteristic functions, working directly with expectations and allowing dependent variables.

Formal Definition

Definition

The core of Stein's method for the normal distribution exploits the characterisation: Z ~ N(0,1) if and only if E[f'(Z) − Zf(Z)] = 0 for all sufficiently smooth f. For a random variable W, the Stein equation for a test function h is:

f'(w) - w\,f(w) = h(w) - E[h(Z)], \quad Z \sim N(0,1)

Normal Stein equation

d_{\mathcal{H}}(W, Z) = \sup_{h \in \mathcal{H}} |E[h(W)] - E[h(Z)]| \leq \sup_{h \in \mathcal{H}} |E[f_h'(W) - W f_h(W)]|

Stein bound on distributional distance

\text{If } W = \sum_{i=1}^n X_i \text{ (independent, mean 0, } \operatorname{Var}(W) = 1\text{):}

d_{\mathrm{W}}(W, Z) \leq \sum_{i=1}^n E[|X_i|^3] \quad (\text{Wasserstein-1 CLT rate})

Berry–Esseen type bound via Stein's method

Worked Examples

1
Step 1 (Stein characterisation): Z ~ N(0,1) iff E[f'(Z)] = E[Zf(Z)] for all absolutely continuous f with E|f'(Z)| < ∞.
2
Step 2 (Stein equation): For a test function h, solve f'(w) − wf(w) = h(w) − Φh where Φh = E[h(Z)]. The solution fₕ is smooth and ‖fₕ'‖ ≤ 2‖h‖.
3
Step 3 (Bound the remainder): The distributional distance |E[h(W)] − E[h(Z)]| = |E[fₕ'(W) − Wfₕ(W)]|.
$|E[h(W)] - \Phi h| = |E[f_h'(W) - W f_h(W)]|$
4
Step 4 (Exploit independence): Expand and use independence of the Xᵢ to show the remainder is bounded by a sum of third moments.
$\leq C \sum_{i=1}^n E[|X_i|^3]$

✓ Answer

The Stein bound gives |E[h(W)] − E[h(Z)]| ≤ C·Σ E[|Xᵢ|³], recovering the Berry–Esseen rate.

Practice Problems

Mediumfree response

Why can Stein's method give distributional bounds in settings where the classical CLT (via characteristic functions) does not easily apply?

Hardfree response

What is the Berry–Esseen theorem and how does it relate to Stein's method?

Quiz

Summary

Stein's method provides explicit distributional distance bounds by exploiting the characterisation E[f'(Z) − Zf(Z)] = 0 for Z ~ N(0,1).
For a target random variable W, the distance |E[h(W)] − E[h(Z)]| = |E[f'(W) − Wf(W)]|, bounded using moments and structure of W.
The method handles dependent summands via exchangeable pairs, size-biased coupling, and local dependency graphs.
Stein's method proves the Berry–Esseen theorem: the CLT error is O(Σ E[|Xᵢ|³]).

References

WebsiteWikipedia — Stein's method