Linear Independence — Mathematics Knowledge System

Overview

A set of vectors is linearly independent if no vector in the set can be written as a combination of the others — equivalently, the only way to combine them to get the zero vector is to use all-zero coefficients. If some nontrivial combination does produce zero, the set is linearly dependent, meaning at least one vector is redundant: it lies in the span of the rest. Linear independence is the precise notion of 'no redundancy' that underlies the idea of a basis, and it is tested algebraically by checking whether a homogeneous linear system has only the trivial solution.

Intuition

Picture vectors as arrows from the origin. Two vectors in the plane are linearly dependent exactly when they point along the same line — one is just a scaled copy of the other, so together they only 'reach' a line's worth of directions instead of the whole plane. Three vectors in 3D space are dependent when they all lie flat in a single plane, wasting the third dimension. Linear independence means every vector in the set contributes a genuinely new direction that the others cannot fake by scaling and adding.

Formal Definition

Definition

Vectors v_1, \dots, v_k in a vector space V are linearly independent if the only scalars c_1,\dots,c_k satisfying the equation below are all zero:

c_1 v_1 + c_2 v_2 + \cdots + c_k v_k = \mathbf{0} \implies c_1 = c_2 = \cdots = c_k = 0

Definition of linear independence

\text{dependent} \iff \exists\, c_i \text{ not all } 0 \text{ with } \sum_i c_i v_i = \mathbf{0}

Linear dependence

\{v_1,\ldots,v_k\} \text{ dependent} \iff \det[v_1 \cdots v_k] = 0 \ (k=n)

Determinant test (square case)

Worked Examples

Set up c_1 v_1 + c_2 v_2 = 0 and look for a nontrivial solution.
$c_1(1,2) + c_2(2,4) = (0,0)$
Notice v_2 = 2v_1, so choosing c_1 = -2, c_2 = 1 gives a nonzero solution.
$-2(1,2) + 1(2,4) = (0,0)$

Answer: Dependent — v_2 is exactly 2 times v_1.

Practice Problems

Difficulty 4/10

Determine whether v_1 = (1, 1), v_2 = (3, -1) are linearly independent.

Difficulty 5/10

Are (1,2,3), (2,4,6), (1,0,0) linearly independent in ℝ³?

Difficulty 5/10

A dataset's three feature columns turn out to be linearly dependent. Why is this a practical problem for a regression model, and what does it mean geometrically?

Quiz

Summary

A set of vectors is linearly independent if the only linear combination equaling zero uses all-zero coefficients.
For n vectors in ℝⁿ, independence is equivalent to a nonzero determinant of the matrix formed by them as columns.
Dependence means at least one vector is redundant — it lies in the span of the others, wasting a dimension.

References

WebsiteWikipedia — Linear independence