Truncation Error (Discretization Error)
Goal
Understand the definition of truncation error, learn how to estimate errors from truncating Taylor series and from discretization in finite-difference approximation, numerical integration, and ordinary differential equations, and clearly distinguish truncation error from round-off error.
Prerequisites
- Ch. 6: Absolute Error and Ch. 7: Relative Error
- Ch. 8: Round-off Error (for comparison)
- Basic knowledge of Taylor series
Table of Contents
1. Definition
Truncation error arises when an infinite mathematical process is replaced by a finite approximation.
Sources of Truncation Error
- Truncating an infinite series to finitely many terms (Taylor series truncation)
- Approximating derivatives by finite differences
- Approximating integrals by weighted sums at finitely many points (numerical quadrature)
- Approximating continuous ODE solutions by discrete time steps
In the context of numerical differentiation, numerical integration, and differential equations, truncation error often appears as discretization error. Truncation error is fundamentally different from round-off error: it arises from replacing exact operations by approximate procedures when solving a mathematical model, while round-off error arises from finite-precision floating-point arithmetic.
2. Taylor Series and Truncation Error
Expanding $f(x)$ in a Taylor series about $a$ up to order $n$ gives
$$f(x) = \displaystyle\sum_{k=0}^{n} \dfrac{f^{(k)}(a)}{k!}(x-a)^k + R_n(x)$$where $R_n(x)$ is the truncation error (remainder term). In Lagrange form:
$$R_n(x) = \dfrac{f^{(n+1)}(\xi)}{(n+1)!}(x-a)^{n+1}$$(when $f$ is $n+1$ times differentiable; $\xi$ is a point between $a$ and $x$).
Example: Taylor series for $e^x$
Truncating $e^x = 1 + x + \dfrac{x^2}{2!} + \dfrac{x^3}{3!} + \cdots$ up to the degree-$n$ term (the $n+1$ terms of degrees $0$ to $n$) gives truncation error
$$R_n = \dfrac{e^\xi}{(n+1)!} x^{n+1}$$At $x = 1$ with $n = 4$: $R_4 \le e/120 \approx 0.0227$. The 4th-order approximation $1 + 1 + 0.5 + 1/6 + 1/24 = 2.7083\ldots$ vs. true value $e \approx 2.7183$, so the actual error $\approx 0.010$ is within the bound.
3. Truncation Error in Numerical Differentiation
The truncation error of the forward difference formula is $O(h)$ (see Round-off Error, §4 for the derivation). The choice of difference formula determines the order of accuracy.
| Difference Formula | Approximation | Truncation Error |
|---|---|---|
| Forward difference | $\dfrac{f(x+h) - f(x)}{h}$ | $O(h)$ |
| Backward difference | $\dfrac{f(x) - f(x-h)}{h}$ | $O(h)$ |
| Central difference | $\dfrac{f(x+h) - f(x-h)}{2h}$ | $O(h^2)$ |
Central differences achieve one order higher accuracy than forward/backward differences because, when the Taylor expansions of $f(x+h)$ and $f(x-h)$ are subtracted, the even-order terms (such as $h^2 f''(x)$) cancel, leaving the leading error term $\dfrac{h^2}{6}f'''(x)$.
4. Truncation Error in Numerical Integration
Truncation errors of common numerical integration formulas, as orders of the composite rules that divide the interval into subintervals of width $h$ (local: per subinterval; global: over the whole interval):
| Formula | Local Truncation Error | Global Truncation Error |
|---|---|---|
| Trapezoidal rule | $O(h^3)$ | $O(h^2)$ |
| Simpson's rule | $O(h^5)$ | $O(h^4)$ |
| Gaussian quadrature ($n$ points) | $O(h^{2n+1})$ | $O(h^{2n})$ |
Halving $h$ reduces the trapezoidal rule error by a factor of about $1/4$ and the Simpson's rule error by about $1/16$.
5. Local vs Global Truncation Error
In numerical methods for ordinary differential equations, two types of truncation error are distinguished.
Local Truncation Error (LTE)
The error per step, assuming the previous step's value is exact.
Global Truncation Error (GTE)
The error at the final time that results from the local errors of all steps propagating and accumulating through later steps (excluding round-off error).
For Euler's method, LTE is $O(h^2)$ and GTE is $O(h)$. Over the interval $[a, b]$ with $N = (b-a)/h$ steps, accumulating LTE gives GTE $= O(h^2) \times O(1/h) = O(h)$ (when the method is stable, so that the propagation of errors is bounded by about $e^{L(b-a)}$, where $L$ is the Lipschitz constant; if $h$ is too large, errors can be amplified and blow up). See the error of the Euler method for a detailed analysis with numerical experiments.
Truncation errors of representative ODE solvers
| Method | LTE | GTE | Order |
|---|---|---|---|
| Euler's method | $O(h^2)$ | $O(h)$ | 1st |
| Improved Euler (Heun's method) | $O(h^3)$ | $O(h^2)$ | 2nd |
| Runge-Kutta (RK4) | $O(h^5)$ | $O(h^4)$ | 4th |
6. Trade-off with Round-off Error
Total numerical error is the sum of truncation error and round-off error. Decreasing $h$ reduces truncation error but increases round-off error, so in typical cases such as numerical differentiation, an optimal $h$ exists that minimizes total error.
For quantitative analysis (derivation of $O(h)$ vs $O(\varepsilon/h)$ for numerical differentiation, a log-log plot, optimal $h^* \sim \sqrt{\varepsilon}$, and the series truncation case), see Round-off Error, §4.
7. Frequently Asked Questions
Q1. What is truncation error?
Error arising when an infinite mathematical process is replaced by a finite approximation. Examples include truncating Taylor series, approximating derivatives by finite differences, and approximating integrals by numerical quadrature. In the context of finite differences, numerical integration, and differential equations, it is also called discretization error.
Q2. How does truncation error differ from round-off error?
Truncation error comes from replacing exact operations by approximate procedures; it decreases as the step size decreases when discretization is involved, or as the truncation degree increases for truncated series. Round-off error comes from finite-precision arithmetic and increases when step size is made too small. The balance between the two determines optimal accuracy.
Q3. Local vs global truncation error?
Local truncation error (LTE) is the error per step; global truncation error (GTE) is the error at the final time resulting from the local errors propagating and accumulating (for a stable method, its order is one lower than that of the local error). For Euler's method, LTE is $O(h^2)$ and GTE is $O(h)$.
8. References
- Wikipedia, "Truncation error"
- Wikipedia, "Discretization error"
- R. L. Burden & J. D. Faires, Numerical Analysis, 10th ed., Cengage, 2016.
- A. Quarteroni, R. Sacco & F. Saleri, Numerical Mathematics, 2nd ed., Springer, 2007.