Moment Generating Functions — Mathematics Knowledge System

Overview

The moment generating function (MGF) of a random variable X is M(t) = E[e^{tX}], defined for all t where this expectation is finite. It is called a 'moment generating' function because its derivatives at t=0 recover the moments of X: the k-th derivative evaluated at 0 equals E[X^k]. MGFs are useful because they uniquely determine a distribution (when they exist in a neighborhood of 0), and the MGF of a sum of independent random variables is the product of their individual MGFs, which makes MGFs a powerful tool for finding the distribution of sums.

Intuition

Think of the MGF as a 'compressed file' that encodes every moment of a distribution (mean, variance, skewness, and beyond) into a single function. Taking derivatives and plugging in t=0 is like unzipping that file to extract one moment at a time: the first derivative gives the mean, the second gives E[X²] (from which variance follows), and so on. The real power shows up when combining independent random variables — instead of doing a messy convolution to find the distribution of a sum, you just multiply their MGFs together, which is often far simpler algebra.

Formal Definition

Definition

For a random variable X, the moment generating function is:

M_X(t) = E\left[e^{tX}\right]

Definition

M_X^{(k)}(0) = E[X^k]

k-th derivative at 0 gives the k-th moment

M_{X+Y}(t) = M_X(t)\,M_Y(t) \quad \text{(X, Y independent)}

MGF of a sum of independent variables

M_{aX+b}(t) = e^{bt}M_X(at)

Effect of a linear transformation

Worked Examples

Compute E[e^{tX}] directly from the definition.
$M_X(t) = E[e^{tX}] = (1-p)e^{0} + p\,e^{t} = (1-p) + pe^t$
Differentiate with respect to t.
$M_X'(t) = p e^t$
Evaluate at t=0 to get the first moment.
$M_X'(0) = p e^0 = p$

Answer: M_X(t) = 1 − p + pe^t, and M_X'(0) = p = E[X], as expected.

Practice Problems

Difficulty 5/10

Given M_X(t) = e^{2t + 3t^2}, find E[X^2] using M_X''(0), and hence Var(X). (Hint: M_X'(t) = (2+6t)e^{2t+3t^2}.)

Difficulty 4/10

If X and Y are independent with MGFs M_X(t) = e^{t^2} and M_Y(t) = e^{2t^2}, find the MGF of X+Y.

Difficulty 5/10

A Poisson(λ) random variable has MGF M(t) = e^{λ(e^t - 1)}. Use M'(0) to confirm that E[X] = λ. (Hint: M'(t) = λe^t · e^{λ(e^t-1)}.)

Quiz

Summary

M_X(t) = E[e^{tX}] encodes all the moments of X: M^{(k)}(0) = E[X^k].
For independent random variables, the MGF of their sum is the product of their individual MGFs.
When an MGF exists near t=0, it uniquely determines the distribution, making it a powerful identification tool.

References

WebsiteWikipedia — Moment-generating function