Gaussian (Normal) Distribution — Mathematics Knowledge System

Overview

The Gaussian or normal distribution N(mu, sigma^2) is the most important continuous probability distribution in statistics and probability theory. It arises naturally as the limit of sums of independent random variables (Central Limit Theorem), describes measurement errors, and underpins much of statistical inference. Its bell-shaped density is symmetric about the mean and determined entirely by two parameters: mean mu and variance sigma^2.

Intuition

The bell curve shape emerges whenever a quantity is the sum of many small independent random influences -- measurement error, height, test scores. Each influence pushes the outcome slightly up or down; their sum clusters tightly near the mean with rapidly decaying probability in the tails. The standard normal Z ~ N(0,1) is the universal 'unit' normal from which all others come by scaling and shifting.

Formal Definition

Definition

A random variable X has the Gaussian distribution N(mu, sigma^2) if its probability density function is:

f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)

Normal density

\Phi(z) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^z e^{-t^2/2}\,dt

Standard normal CDF

M_X(t) = \exp\!\left(\mu t + \frac{\sigma^2 t^2}{2}\right)

Moment generating function

Notation

Notation	Meaning
$N(\mu, \sigma^2)$	Normal distribution with mean mu and variance sigma^2
$\Phi(z)$	Standard normal CDF
$\phi(z) = \Phi'(z)$	Standard normal density
$Z = (X-\mu)/\sigma$	Standardization: transforms X to standard normal

Theorems

Theorem 1: 68-95-99.7 Rule

P(|X-\mu| \le \sigma) \approx 0.6827, \quad P(|X-\mu| \le 2\sigma) \approx 0.9545, \quad P(|X-\mu| \le 3\sigma) \approx 0.9973

Theorem 2: Closure under Linear Combinations

\text{If } X_i \sim N(\mu_i,\sigma_i^2) \text{ independent, then } \sum a_i X_i \sim N\!\left(\sum a_i\mu_i,\, \sum a_i^2\sigma_i^2\right)

Theorem 3: Central Limit Theorem (informal)

\text{If } X_1,\ldots,X_n \text{ are i.i.d. with mean } \mu \text{ and variance } \sigma^2 < \infty, \text{ then } \sqrt{n}(\bar{X}-\mu)/\sigma \xrightarrow{d} N(0,1)

Worked Examples

1
Standardize: sigma = 15. Z1 = (85-100)/15 = -1, Z2 = (115-100)/15 = 1.
$P(85 < X < 115) = P(-1 < Z < 1) = \Phi(1) - \Phi(-1)$
2
Use the 68-95-99.7 rule: P(-1 < Z < 1) = 0.6827.
$\approx 0.6827$

✓ Answer

P(85 < X < 115) = 0.6827 (approximately 68.3%).

Practice Problems

Mediumfree response

SAT scores follow N(1060, 195^2). What percentage of students score above 1250?

Mediumproof writing

Show that the moment generating function of N(0,1) is M(t) = e^{t^2/2}.

Common Mistakes

Common Mistake

Assuming that all continuous distributions are approximately normal

The CLT applies to sums/averages of many i.i.d. variables. Individual heavy-tailed distributions (Cauchy, Pareto) may not satisfy the CLT's conditions and their sums do not converge to Gaussian.

Common Mistake

Confusing the normal distribution with the uniform or symmetric distribution

Symmetry alone does not make a distribution normal. The normal has sub-Gaussian tails decaying as exp(-x^2/2); a uniform distribution has hard cutoffs.

Quiz

Historical Background

De Moivre approximated the binomial distribution with a normal curve in 1733 while computing odds in games of chance. Gauss independently derived the distribution in 1809 while modeling astronomical measurement errors, arguing it was the unique distribution making the sample mean the maximum likelihood estimator. Laplace proved the CLT (1810), explaining why the Gaussian appears so universally. The term 'normal' was introduced by Galton in the 1880s.

1733
De Moivre approximates binomial with normal curve
Abraham de Moivre
1809
Gauss derives the distribution from least-squares theory
Carl Friedrich Gauss
1810
Laplace proves the Central Limit Theorem
Pierre-Simon Laplace

Summary

The Gaussian N(mu, sigma^2) has bell-shaped density (1/sqrt(2pi sigma^2)) exp(-(x-mu)^2/(2sigma^2)).
68-95-99.7 rule: 68%/95%/99.7% of mass lies within 1/2/3 standard deviations of the mean.
Closed under linear combinations: sums of independent Gaussians are Gaussian.
Central Limit Theorem: the Gaussian arises as the universal limit of sums of i.i.d. random variables.
Standard normal Z ~ N(0,1) obtained by standardization Z = (X-mu)/sigma.

References

BookCasella, G. and Berger, R. -- Statistical Inference (2nd ed., 2001), Chapter 2
WebsiteWikipedia -- Normal distribution