Mathematics.

orthogonality

Vector Norms

Linear Algebra25 minDifficulty3 out of 10

You should know: vectors

Overview

A norm is a function that assigns a nonnegative 'length' or 'size' to every vector, generalizing the everyday notion of distance. The most familiar is the Euclidean norm (2-norm), the straight-line length given by the Pythagorean theorem, but other useful norms exist, such as the 1-norm (sum of absolute values) and the infinity-norm (largest absolute entry). Every norm must satisfy three axioms: it's zero only for the zero vector, it scales linearly with scalar multiplication, and it obeys the triangle inequality. Norms let us measure error, define convergence, and compare vectors of any dimension.

Intuition

The Euclidean norm ‖v‖₂ is just the length of the arrow v, computed with the Pythagorean theorem: stack the squared components and take the square root. Other norms answer 'how big' in different senses — the 1-norm treats movement like walking city blocks (you add up horizontal and vertical distance separately, like taxicab geometry), while the infinity-norm cares only about the single largest coordinate, ignoring how many other coordinates are also large. Choosing a norm is choosing which notion of 'size' or 'error' matters for the problem at hand.

Formal Definition

Definition

A norm \|\cdot\|: V \to \mathbb{R} satisfies, for all vectors u, v and scalars c:

∥v∥≥0, and ∥v∥=0  ⟺  v=0\|v\| \ge 0, \text{ and } \|v\| = 0 \iff v = \mathbf{0}
Positive definiteness
∥cv∥=∣c∣ ∥v∥\|cv\| = |c|\,\|v\|
Absolute homogeneity
∥u+v∥≤∥u∥+∥v∥\|u+v\| \le \|u\| + \|v\|
Triangle inequality
∥v∥p=(∑i∣vi∣p)1/p,∥v∥∞=max⁡i∣vi∣\|v\|_p = \left(\sum_i |v_i|^p\right)^{1/p}, \qquad \|v\|_\infty = \max_i |v_i|
p-norm family and the infinity norm

Worked Examples

  1. Sum the squares of the components and take the square root.

    ∥v∥2=32+42=9+16=25\|v\|_2 = \sqrt{3^2 + 4^2} = \sqrt{9+16} = \sqrt{25}

Answer: ‖v‖₂ = 5.

Practice Problems

Difficulty 3/10

Compute the 2-norm of v = (1, 2, 2).

Difficulty 4/10

Compute the 1-norm and infinity-norm of v = (5, -3, 1, -1).

Difficulty 5/10

A machine learning model's weight update is regularized using an L1 penalty (‖w‖₁) versus an L2 penalty (‖w‖₂²). Why does the L1 norm tend to produce sparse (many-zero) weight vectors while L2 tends not to?

Quiz

Which of these is NOT a required property of a norm?
The infinity-norm ‖v‖∞ of a vector v equals:
For v = (3,4), which norm gives exactly 5?

Summary

  • A norm assigns a nonnegative length to vectors, satisfying positive definiteness, homogeneity, and the triangle inequality.
  • Common norms: 1-norm (sum of absolute values), 2-norm (Euclidean length), infinity-norm (max absolute entry).
  • Different norms encode different notions of 'size' or 'error,' with major consequences in optimization (e.g. L1 induces sparsity, L2 does not).

References

  1. WebsiteWikipedia — Norm (mathematics)