Mathematics.

transforms

Characteristic Functions

Probability45 minDifficulty7 out of 10

You should know: moment generating functions, expectation

Overview

The characteristic function of a random variable X is defined as the expected value of e^{itX} — the Fourier transform of its probability distribution. Unlike moment generating functions (MGFs), characteristic functions exist for every probability distribution without any moment conditions, making them the most general and powerful analytic tool in probability theory. They are the key instrument used to prove the central limit theorem rigorously and to characterise important distribution families such as stable and infinitely divisible distributions.

Intuition

The characteristic function is the Fourier transform of the probability distribution. Just as the Fourier transform uniquely characterises a function, the characteristic function uniquely characterises a probability distribution — two distributions are equal if and only if they have the same characteristic function. The key advantage over moment generating functions is that the characteristic function is always finite (since |e^{itX}| = 1), so it exists even for heavy-tailed distributions like the Cauchy where the MGF diverges everywhere except at 0.

Formal Definition

Definition

The characteristic function of a random variable X is the function φ_X : ℝ → ℂ defined by:

φX(t)=E[eitX]=E[cos⁡(tX)]+i E[sin⁡(tX)],t∈R\varphi_X(t) = E[e^{itX}] = E[\cos(tX)] + i\,E[\sin(tX)], \quad t \in \mathbb{R}
Definition of the characteristic function
φX(t)=∫−∞∞eitx fX(x) dx(continuous case)\varphi_X(t) = \int_{-\infty}^{\infty} e^{itx}\, f_X(x)\,dx \quad \text{(continuous case)}
For absolutely continuous distributions
φX+Y(t)=φX(t) φY(t)(X, Y independent)\varphi_{X+Y}(t) = \varphi_X(t)\,\varphi_Y(t) \quad \text{(X, Y independent)}
Multiplication rule for independent sums
E[Xn]=1in φX(n)(0)E[X^n] = \frac{1}{i^n}\,\varphi_X^{(n)}(0)
Moments from derivatives (when moments exist)

Worked Examples

  1. 1

    Compute the expectation using the Gaussian integral.

    φX(t)=∫−∞∞eitx⋅12πe−x2/2 dx\varphi_X(t) = \int_{-\infty}^{\infty} e^{itx} \cdot \frac{1}{\sqrt{2\pi}} e^{-x^2/2}\, dx
  2. 2

    Complete the square in the exponent: itx − x²/2 = −(x − it)²/2 − t²/2.

    =e−t2/2∫−∞∞12πe−(x−it)2/2 dx=e−t2/2= e^{-t^2/2} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}} e^{-(x-it)^2/2}\, dx = e^{-t^2/2}

✓ Answer

φ_X(t) = e^{−t²/2}.

Practice Problems

Mediumfree response

Find the characteristic function of the Bernoulli(p) distribution.

Mediumfree response

If X has characteristic function φ_X(t), what is the characteristic function of aX + b?

Quiz

Why do characteristic functions exist for every probability distribution?
The characteristic function of the sum of two independent random variables equals:
The characteristic function of N(0,1) is:

Summary

  • The characteristic function φ_X(t) = E[e^{itX}] is the Fourier transform of the distribution and always exists.
  • It uniquely determines the distribution; two distributions are equal iff their characteristic functions agree.
  • For independent variables, characteristic functions multiply: φ_{X+Y} = φ_X · φ_Y.
  • Moments can be recovered from derivatives at zero: E[Xⁿ] = φ_X^{(n)}(0) / iⁿ.

References

  1. WebsiteWikipedia — Characteristic function (probability theory)