Positive Definite Matrices — Mathematics Knowledge System

Overview

A symmetric (or Hermitian) matrix A is positive definite if the quadratic form v^T A v > 0 for all nonzero vectors v. Equivalently, all eigenvalues are strictly positive. Positive definite matrices represent genuine inner products, arise in optimization (Hessians at minima), statistics (covariance matrices), and admit the Cholesky factorization A = L L^T.

Intuition

A positive definite matrix A defines a 'stretched inner product': the pairing (u,v) = u^T A v satisfies all inner product axioms. Think of A as an ellipse — the set {v : v^T A v = 1} is an ellipsoid, with axes aligned to the eigenvectors of A and lengths inversely proportional to the square roots of eigenvalues. Positive definiteness means the ellipsoid is a proper, bounded shape — no flat directions.

Formal Definition

Definition

A real symmetric matrix A is positive definite (PD) if, and positive semidefinite (PSD) if:

A \succ 0 \iff v^\top A v > 0 \text{ for all } v \neq 0

Positive definite

A \succeq 0 \iff v^\top A v \geq 0 \text{ for all } v

Positive semidefinite

A \succ 0 \iff \text{all eigenvalues of } A \text{ are strictly positive}

Eigenvalue criterion

A \succ 0 \iff A = LL^\top \text{ (Cholesky factorization, } L \text{ lower triangular with positive diagonal)}

Cholesky criterion

Notation

Notation	Meaning
$A \succ 0$	A is positive definite
$A \succeq 0$	A is positive semidefinite
$v^\top A v$	Quadratic form associated with A

Properties

All eigenvalues positive

A \succ 0 \iff \lambda_i > 0 \text{ for all eigenvalues.}

Positive diagonal

A \succ 0 \implies a_{ii} > 0 \text{ for all } i.

Invertible

A \succ 0 \implies A \text{ is invertible, and } A^{-1} \succ 0.

Sylvester's criterion

A \succ 0 \iff \text{all leading principal minors of } A \text{ are positive.}

Closure

\text{If } A,B \succ 0 \text{ and } \alpha,\beta > 0, \text{ then } \alpha A + \beta B \succ 0.

Applications

Real-world · Machine Learning

Covariance matrices in multivariate Gaussian distributions are positive semidefinite. Positive definite covariance matrices correspond to non-degenerate distributions.

Worked Examples

1
Sylvester's criterion: check leading principal minors.
$\Delta_1 = 2 > 0, \quad \Delta_2 = \det\begin{pmatrix}2&1\\1&2\end{pmatrix} = 4-1 = 3 > 0$
2
Both minors are positive, so A is positive definite by Sylvester's criterion.
3
Confirm with eigenvalues: characteristic polynomial.
$\lambda^2 - 4\lambda + 3 = (\lambda-1)(\lambda-3) = 0 \implies \lambda = 1, 3 > 0 \checkmark$

✓ Answer

A is positive definite: eigenvalues 1 and 3 are both positive, and leading minors 2 and 3 are positive.

Practice Problems

Mediumfree response

For what values of c is the matrix A = [[4,c],[c,1]] positive definite?

Mediumfree response

Show that if A is positive definite and B is any matrix with full column rank, then B^T A B is positive definite.

Common Mistakes

Common Mistake

Positive diagonal entries imply positive definiteness.

Positive diagonal entries are necessary but not sufficient. The full matrix A = [[1,2],[2,1]] has positive diagonal but eigenvalue -1, so it is not positive definite.

Quiz

Summary

A is positive definite (A ≻ 0) if v^T A v > 0 for all nonzero v — equivalently, all eigenvalues are positive.
Sylvester's criterion: A ≻ 0 iff all leading principal minors are positive.
A ≻ 0 has the Cholesky factorization A = LL^T with L lower triangular, positive diagonal.
Positive definite matrices are invertible; their inverse is also positive definite.
Key applications: covariance matrices (statistics), Hessians at local minima (optimization), stiffness matrices (FEM).

References

WebsiteWikipedia — Definite matrix