Mathematics.

orthogonality

Orthogonality

Linear Algebra30 minDifficulty4 out of 10

You should know: vector norms

Overview

Two vectors are orthogonal if their dot product is zero — the generalization of 'perpendicular' to any number of dimensions. A set of vectors is orthogonal if every pair in the set is orthogonal, and orthonormal if additionally every vector has unit length. Orthogonal sets are enormously convenient in linear algebra because they behave like independent, non-interfering coordinate axes: computing a vector's coordinates in an orthonormal basis reduces to simple dot products instead of solving a linear system.

Intuition

Orthogonal vectors point in genuinely independent directions, with zero 'overlap' — like the x-axis and y-axis, moving along one doesn't get you any closer to or farther from anywhere along the other. This is why orthogonal bases are so useful: if you decompose a vector into orthogonal-axis components, each component can be found on its own, independent of the others, by simple projection, whereas with a skewed (non-orthogonal) basis you'd need to solve a coupled system to disentangle the coordinates.

Formal Definition

Definition

Vectors u and v in an inner product space are orthogonal if their inner product vanishes:

u⊥v  ⟺  ⟨u,v⟩=u⋅v=0u \perp v \iff \langle u, v \rangle = u \cdot v = 0
Orthogonality condition
∥u+v∥2=∥u∥2+∥v∥2if u⊥v\|u+v\|^2 = \|u\|^2 + \|v\|^2 \quad \text{if } u \perp v
Pythagorean theorem for orthogonal vectors
{q1,…,qk} orthonormal  ⟺  ⟨qi,qj⟩=δij\{q_1,\ldots,q_k\} \text{ orthonormal} \iff \langle q_i,q_j\rangle = \delta_{ij}
Orthonormal set (Kronecker delta)
proju(v)=⟨v,u⟩⟨u,u⟩ u\text{proj}_u(v) = \frac{\langle v,u\rangle}{\langle u,u\rangle}\, u
Orthogonal projection of v onto u

Worked Examples

  1. Compute the dot product.

    u⋅v=1(3)+2(−1)+(−1)(1)=3−2−1=0u \cdot v = 1(3) + 2(-1) + (-1)(1) = 3 - 2 - 1 = 0

Answer: Yes — the dot product is 0, so u and v are orthogonal.

Practice Problems

Difficulty 3/10

Determine whether u = (2, -1, 3) and v = (1, 5, 1) are orthogonal.

Difficulty 5/10

Find the projection of v = (3, 4) onto u = (1, 1), and verify v minus the projection is orthogonal to u.

Difficulty 6/10

In digital signal processing, why are orthogonal basis signals (like sines and cosines of different frequencies) preferred for decomposing a signal into frequency components?

Quiz

Two nonzero vectors u, v are orthogonal exactly when:
For orthogonal vectors u and v, the Pythagorean-style identity states:
An orthonormal set additionally requires, beyond pairwise orthogonality, that:

Summary

  • Vectors u and v are orthogonal when u·v = 0, generalizing perpendicularity to any dimension.
  • Orthogonal vectors satisfy a Pythagorean identity: ‖u+v‖² = ‖u‖²+‖v‖².
  • Projection onto u, proj_u(v) = (v·u/u·u)u, decomposes v into components along and perpendicular to u.

References

  1. WebsiteWikipedia — Orthogonality