Mathematics.

transforms

Probability Generating Functions

Probability35 minDifficulty6 out of 10

You should know: discrete probability distributions, expectation

Overview

The probability generating function (PGF) is a power-series transform that encodes the entire probability distribution of a non-negative integer-valued random variable in a single analytic function. It is especially useful for discrete distributions (Binomial, Poisson, Geometric, Negative Binomial) and for analysing branching processes, queueing systems, and random walks. The PGF of a sum of independent variables is the product of their PGFs — mirroring the role of the characteristic function but tailored to discrete settings.

Intuition

Think of the PGF as a polynomial whose coefficients are probabilities: the coefficient of zᵏ is exactly P(X = k). Multiplying two PGFs is equivalent to convolving two distributions (computing probabilities of the sum), since (Σ pₖ zᵏ)(Σ qₖ zᵏ) = Σ (Σ pⱼ q_{k-j}) zᵏ. Evaluating the PGF and its derivatives at z = 1 gives moments: G'(1) = E[X], G''(1) = E[X(X−1)], and so on.

Formal Definition

Definition

For a non-negative integer-valued random variable X, the probability generating function G_X(z) is defined for |z| ≤ 1 (and often for |z| in a neighbourhood of 1) by:

GX(z)=E[zX]=∑k=0∞P(X=k) zkG_X(z) = E[z^X] = \sum_{k=0}^{\infty} P(X = k)\,z^k
Definition of the probability generating function
P(X=k)=GX(k)(0)k!P(X = k) = \frac{G_X^{(k)}(0)}{k!}
Recovering probabilities from derivatives
E[X]=GX′(1),E[X(X−1)]=GX′′(1)E[X] = G_X'(1), \quad E[X(X-1)] = G_X''(1)
Moments from derivatives at z = 1
GX+Y(z)=GX(z) GY(z)(X, Y independent)G_{X+Y}(z) = G_X(z)\,G_Y(z) \quad \text{(X, Y independent)}
Multiplication rule for independent sums

Worked Examples

  1. 1

    Sum the geometric series.

    GX(z)=∑k=1∞(1−p)k−1p zk=pz∑k=0∞[(1−p)z]k=pz1−(1−p)zG_X(z) = \sum_{k=1}^{\infty} (1-p)^{k-1}p\,z^k = pz \sum_{k=0}^{\infty} [(1-p)z]^k = \frac{pz}{1-(1-p)z}
  2. 2

    This is valid for |z| < 1/(1−p).

✓ Answer

G_X(z) = pz / [1 − (1−p)z].

Practice Problems

Mediumfree response

Find the PGF of X ~ Bernoulli(p).

Mediumfree response

If X ~ Binomial(n, p), what is the PGF? Use the Bernoulli PGF and independence.

Quiz

The probability generating function G_X(z) encodes which information?
For independent non-negative integer-valued X and Y, what is G_{X+Y}(z)?
To find E[X] from the PGF G(z), you compute:

Summary

  • The PGF G_X(z) = E[z^X] = Σ P(X=k) zᵏ encodes the full distribution of a non-negative integer-valued variable.
  • Probabilities are the Taylor coefficients: P(X=k) = G^{(k)}(0)/k!
  • Moments: E[X] = G'(1), E[X(X−1)] = G''(1).
  • For independent variables, PGFs multiply: G_{X+Y} = G_X · G_Y.

References

  1. WebsiteWikipedia — Probability generating function