Numerical Methods — Mathematics Knowledge System

Overview

Numerical methods are systematic recipes for computing approximate solutions to mathematical problems that have no convenient closed form — or whose exact solution is too expensive to obtain. Instead of an exact formula they produce a number, correct to a controllable tolerance, using only the four arithmetic operations a computer can perform. Root finding (solve f(x)=0), interpolation (fit a curve through data), numerical integration (quadrature), numerical differentiation, and the numerical solution of differential equations are the core tasks. They are the computational backbone of every engineering analysis package, from structural finite-element solvers to hydraulic and traffic simulations.

Intuition

Most real engineering equations cannot be solved with pencil and paper: the head-loss equation for turbulent pipe flow, the bearing-capacity equation for a footing, or the deflection of an irregular beam have no tidy algebraic answer. Numerical methods trade the exact-but-impossible for the approximate-but-computable, and they do it by iteration and local approximation — replace a hard curve by a straight line or a parabola over a small interval, solve that easy version, then refine. Two ideas run through everything: convergence (does the sequence of approximations home in on the true answer, and how fast?) and error (how far off am I, and is it from the method itself — truncation error — or from the computer's finite precision — round-off error?).

Formal Definition

Definition

Representative core methods: the bisection and Newton–Raphson root finders, the trapezoidal rule for integration, and the finite-difference approximation to a derivative.

x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}

Newton–Raphson iteration for f(x)=0

c = \dfrac{a+b}{2}

Bisection midpoint (bracket [a,b] with sign change)

\int_a^b f(x)\,dx \approx \dfrac{h}{2}\Big[f_0 + 2\!\sum_{i=1}^{n-1} f_i + f_n\Big],\; h=\tfrac{b-a}{n}

Composite trapezoidal rule

f'(x) \approx \dfrac{f(x+h)-f(x-h)}{2h}

Central finite-difference derivative

Notation

Notation	Meaning
$x_n$	The n-th iterate — the approximation to the solution at step n
$h$	Step size / grid spacing; smaller h means less truncation error but more work and more round-off
$\varepsilon$	Tolerance — the iteration stops when the estimated error drops below ε
$O(h^n)$	Order of accuracy: the truncation error shrinks like hⁿ as the step size h is refined

Derivation

Newton–Raphson comes straight from a first-order Taylor expansion: approximate f near the current guess xₙ by its tangent line and solve where that line crosses zero.

f(x) \approx f(x_n) + f'(x_n)(x - x_n)

Tangent-line (first-order Taylor) approximation of f at xₙ.

0 = f(x_n) + f'(x_n)(x_{n+1}-x_n)

Set the tangent to zero and call the crossing point xₙ₊₁.

x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}

Solve for xₙ₊₁ — each step replaces the curve with its tangent.

|x_{n+1}-r| \le M\,|x_n - r|^2

Near a simple root r the error squares each step — 'quadratic convergence', roughly doubling correct digits per iteration.

Properties

Bisection convergence

|x_n - r| \le \dfrac{b-a}{2^{\,n}}

Condition: f continuous on [a,b] with f(a)f(b) < 0

Example: Guaranteed but slow (linear): each step halves the bracket, gaining about one bit of accuracy.

Newton quadratic convergence

\text{error}_{n+1} \propto (\text{error}_n)^2

Condition: Simple root, good initial guess, f′ ≠ 0 nearby

Example: Fast but can diverge if the guess is poor or f′ is near zero.

Trapezoidal accuracy

E \le \dfrac{(b-a)h^2}{12}\,\max|f''|

Condition: f twice continuously differentiable

Example: Error is O(h²): halving the step quarters the error.

Round-off vs truncation trade-off

\text{total error} = \text{truncation}(h) + \text{round-off}(1/h)

Condition: Finite-precision arithmetic

Example: Shrinking h forever backfires — below an optimal h round-off dominates and accuracy worsens.

Applications

Pipe-network design: the Colebrook–White equation for the turbulent friction factor is implicit and is solved for f by Newton–Raphson or fixed-point iteration inside every hydraulics package.

Worked Examples

f(x)=x²−2 and f′(x)=2x, so the iteration is xₙ₊₁ = xₙ − (xₙ²−2)/(2xₙ).
$x_{n+1} = x_n - \dfrac{x_n^2-2}{2x_n} = \dfrac{x_n}{2} + \dfrac{1}{x_n}$
Step 1 from x₀ = 1.5.
$x_1 = \dfrac{1.5}{2} + \dfrac{1}{1.5} = 0.75 + 0.6667 = 1.41667$
Step 2 from x₁ = 1.41667.
$x_2 = \dfrac{1.41667}{2} + \dfrac{1}{1.41667} = 0.70833 + 0.70588 = 1.41422$
Compare to the true value √2 = 1.41421… — two steps already give five correct digits (quadratic convergence).
$\sqrt{2} = 1.41421356\ldots$

Answer: x₂ ≈ 1.41422, accurate to 5 digits after just two iterations.

Practice Problems

Difficulty 5/10

A settlement calculation requires the cube root of 50. Solve f(x)=x³−50=0 by Newton–Raphson from x₀=3.7. Give x₁.

Difficulty 5/10

Bisection starts on the bracket [1, 2] (width 1). How many iterations guarantee the root to within a tolerance of 0.001?

Difficulty 7/10

The friction factor f in a pipe satisfies the implicit Colebrook equation 1/√f = −2 log₁₀(ε/(3.7D) + 2.51/(Re√f)). Why can't f be found algebraically, and what numerical approach is standard?

Difficulty 5/10

For a smooth curve, halving the step size h in the trapezoidal rule changes the error by roughly a factor of:

1/21/41/8no change

Difficulty 4/10

Replacing a derivative by (f(x+h)−f(x))/h introduces an error that vanishes as h→0. This error is:

Round-off errorTruncation errorHuman errorModelling error

Difficulty 6/10

Cross-sectional areas (m²) at 20 m stations along a road cut are 12, 18, 25, 20, 14. Use the trapezoidal rule to estimate the cut volume.

Common Mistakes

Common Mistake

Assuming smaller step size h always means a more accurate answer.

Only truncation error falls with h; round-off error rises as h shrinks (subtracting nearly-equal numbers). There is an optimal h — past it, accuracy gets worse.

Common Mistake

Applying Newton–Raphson from any starting point and expecting convergence.

Newton is only locally convergent: a poor initial guess, or an f′ near zero, can make it diverge or cycle. Bracketing methods like bisection are slower but guaranteed to converge.

Common Mistake

Confusing the order of a method with its actual error.

O(h²) describes how fast the error shrinks under refinement, not its size at a given h. A second-order method can still be inaccurate on a coarse grid.

Quiz

Flashcards

1 / 5

Summary

Numerical methods compute controllable-accuracy approximations using only arithmetic, for problems with no closed form.
Core tasks: root finding (bisection, Newton–Raphson), interpolation, quadrature (trapezoidal, Simpson), differentiation, and ODE/PDE solvers.
Newton–Raphson converges quadratically from a good guess; bisection is slow but always converges once the root is bracketed.
Every method carries truncation error (from the approximation, O(hⁿ)) and round-off error (from finite precision) — refining h trades one for the other.
They power real engineering tools: implicit friction-factor solutions, stiffness-matrix linear solves, earthwork volumes, and time-history dynamic analysis.

References

BookBurden, R. L. & Faires, J. D. Numerical Analysis, 10th ed. Cengage, 2015.
BookChapra, S. C. & Canale, R. P. Numerical Methods for Engineers, 7th ed. McGraw-Hill, 2015.
BookPress, W. H. et al. Numerical Recipes: The Art of Scientific Computing, 3rd ed. Cambridge, 2007.
WebsiteWikipedia — Numerical analysis