Convex Functions — Mathematics Knowledge System

Overview

A convex function is a function whose epigraph (the set of points on or above its graph) is a convex set, or equivalently one that lies below any chord connecting two of its points. Convex functions are the central objects of convex analysis: any local minimum is automatically a global minimum, gradient-based methods converge reliably, and a rich duality theory applies. They arise throughout mathematics, statistics, machine learning, and economics.

Intuition

A convex function's graph is 'bowl-shaped' — it curves upward. No matter which two points you connect with a chord, the function lies at or below the chord. This simple geometric property has far-reaching analytical consequences: any critical point is a global minimum, and the function can be characterised entirely by its tangent lines (subdifferentials).

Formal Definition

Definition

Let C ⊆ ℝⁿ be a convex set. A function f: C → ℝ is convex if for all x, y ∈ C and θ ∈ [0,1]:

f(\theta x + (1-\theta)y) \le \theta f(x) + (1-\theta) f(y)

Convexity (Jensen's inequality in two-point form)

f \text{ is concave} \iff -f \text{ is convex}

Concavity

\text{First-order condition: } f(y) \ge f(x) + \nabla f(x)^T(y-x) \quad \forall x, y \in \operatorname{dom} f

Gradient inequality (differentiable case)

\text{Second-order condition: } \nabla^2 f(x) \succeq 0 \quad \forall x \in \operatorname{dom} f

Positive semidefinite Hessian (twice-differentiable case)

Notation

Notation	Meaning
$\operatorname{epi}(f) = \{(x,t) : f(x) \le t\}$	Epigraph of f
$\operatorname{dom}(f)$	Effective domain of f (where f < +∞)
$f^*(y) = \sup_{x}\{y^T x - f(x)\}$	Convex conjugate (Fenchel conjugate)
$\partial f(x)$	Subdifferential of f at x — set of subgradients

Properties

Local minima are global

\text{If } f \text{ is convex and } x^* \text{ is a local minimum, then } x^* \text{ is a global minimum}

Jensen's inequality

f\!\left(\sum_{i=1}^k \theta_i x_i\right) \le \sum_{i=1}^k \theta_i f(x_i), \quad \theta_i \ge 0,\; \sum_i \theta_i = 1

Sublevel sets are convex

\{x \in \operatorname{dom} f : f(x) \le \alpha\} \text{ is convex for every } \alpha \in \mathbb{R}

Nonneg weighted sum preserves convexity

f = \sum_i w_i f_i \text{ is convex when each } f_i \text{ is convex and } w_i \ge 0

Worked Examples

1
f is twice differentiable on ℝ. Compute f''(x).
$f''(x) = 2 > 0 \quad \forall x \in \mathbb{R}$
2
Since f''(x) ≥ 0 for all x, f is convex.
$f''(x) \ge 0 \implies f \text{ is convex}$

✓ Answer

f(x) = x² is strictly convex since f'' = 2 > 0.

Practice Problems

Mediumproof writing

Prove that if f and g are convex and g is non-decreasing, then h(x) = g(f(x)) is convex.

Common Mistakes

Common Mistake

Thinking every quasiconvex function is convex

Quasiconvex functions have convex sublevel sets, but need not satisfy the midpoint inequality. Example: f(x) = x³ is quasiconvex on ℝ but not convex.

Common Mistake

Assuming convex functions are always differentiable

Convex functions can be non-differentiable; they have subdifferentials (sets of subgradients) at non-smooth points. For example, f(x) = |x| is convex but not differentiable at 0.

Quiz

For a twice-differentiable function f: ℝⁿ → ℝ, convexity is equivalent to:

∇f(x) = 0 for some x∇²f(x) ≻ 0 for all x∇²f(x) ≽ 0 for all xf has a unique minimum

The epigraph of a function f is:

{(x,t) : f(x) = t}{(x,t) : f(x) ≤ t}{x : f(x) ≤ 0}{x : f(x) ≥ 0}

Historical Background

The study of convex functions began with Jensen's inequality (1906), which formalised the 'midpoint convexity' notion. Fenchel's conjugate transform (1949) unified convex analysis and provided the foundation for duality theory. Rockafellar's 1970 monograph 'Convex Analysis' became the canonical reference.

1906
Jensen proves Jensen's inequality for convex functions
Johan Jensen
1949
Fenchel introduces the convex conjugate (Legendre–Fenchel transform)
Werner Fenchel
1970
Rockafellar's Convex Analysis provides the definitive systematic treatment
R. Tyrrell Rockafellar

Summary

f is convex if f(θx+(1-θ)y) ≤ θf(x)+(1-θ)f(y) for all x,y and θ ∈ [0,1].
Differentiable convexity: gradient inequality f(y) ≥ f(x)+∇f(x)ᵀ(y-x); twice-differentiable: ∇²f ≽ 0.
Any local minimum of a convex function is a global minimum.
Key examples: norms, exponentials, log-sum-exp, quadratic forms with PSD matrix, negative entropy.

References

BookBoyd, S. and Vandenberghe, L. — Convex Optimization (2004), Chapter 3. Free at https://web.stanford.edu/~boyd/cvxbook/
WebsiteWikipedia — Convex function
WebsiteMathWorld — Convex Function