Matrix Norms — Mathematics Knowledge System

Overview

A matrix norm is a function that assigns a non-negative number to each matrix, generalizing the notion of vector length. Matrix norms measure the 'size' of a matrix and are used to bound errors in numerical computation, characterize convergence of iterative methods, and study perturbation theory for linear systems.

Intuition

Just as vector norms measure how 'big' a vector is, matrix norms measure how much a matrix can 'stretch' vectors. The most important matrix norms — the operator norms — are defined as the maximum stretching factor: ||A|| = max{||Av|| : ||v||=1}. Different norms measure size differently: the Frobenius norm is like the Euclidean norm applied to all matrix entries at once.

Formal Definition

Definition

A matrix norm on M_{m×n}(F) is a function ||·||: M_{m×n} → R satisfying:

\|A\| \geq 0, \quad \|A\| = 0 \iff A = 0

Positive definiteness

\|\alpha A\| = |\alpha|\|A\|

Absolute homogeneity

\|A+B\| \leq \|A\| + \|B\|

Triangle inequality

\|AB\| \leq \|A\|\|B\| \quad (\text{sub-multiplicativity, for square matrices})

Sub-multiplicativity

Notation

Notation	Meaning
$\\|A\\|_F$	Frobenius norm: square root of sum of squared entries
$\\|A\\|_2$	Spectral norm (operator 2-norm): largest singular value
$\\|A\\|_1$	Column-sum norm: max column sum of absolute values
$\\|A\\|_\infty$	Row-sum norm: max row sum of absolute values

Properties

Frobenius norm

\|A\|_F = \sqrt{\sum_{i,j} |a_{ij}|^2} = \sqrt{\operatorname{tr}(A^*A)}

Spectral norm

\|A\|_2 = \sigma_{\max}(A) \text{ (largest singular value of } A).

Column-sum norm

\|A\|_1 = \max_j \sum_i |a_{ij}|

Row-sum norm

\|A\|_\infty = \max_i \sum_j |a_{ij}|

Inequality with spectral radius

\rho(A) \leq \|A\| \text{ for any sub-multiplicative norm } \|\cdot\|.

Worked Examples

1
Frobenius norm: sum all squared entries.
$\|A\|_F = \sqrt{1^2 + (-2)^2 + 3^2 + 4^2} = \sqrt{1+4+9+16} = \sqrt{30}$
2
1-norm: max column absolute sum. Col 1: |1|+|3|=4. Col 2: |-2|+|4|=6.
$\|A\|_1 = \max(4, 6) = 6$

✓ Answer

||A||_F = √30 ≈ 5.48; ||A||_1 = 6.

Practice Problems

Mediumfree response

Compute the Frobenius norm of A = [[0,1],[1,0]] (the permutation/exchange matrix).

Mediumfree response

Verify sub-multiplicativity for ||·||_∞: show ||AB||_∞ ≤ ||A||_∞ ||B||_∞ for 2×2 matrices A = [[1,1],[0,1]], B = [[2,0],[1,3]].

Common Mistakes

Common Mistake

The spectral norm ||A||_2 equals the largest eigenvalue of A.

||A||_2 equals the largest SINGULAR value σ_max(A), not the largest eigenvalue. For symmetric PD matrices they coincide, but in general they differ.

Quiz

Summary

Matrix norms assign a non-negative size to matrices, satisfying positivity, homogeneity, and triangle inequality.
The Frobenius norm is ||A||_F = sqrt(sum of squared entries) = sqrt(tr(A*A)).
The spectral norm ||A||_2 = σ_max(A) (largest singular value); it is the induced 2-norm.
The 1-norm is the max column sum; the ∞-norm is the max row sum of absolute values.
For any sub-multiplicative norm, ρ(A) ≤ ||A|| (spectral radius is bounded by any norm).

References

WebsiteWikipedia — Matrix norm