Chapter 1: What is Numerical Computation?
What is Numerical Computation?
Numerical computation is the method of solving mathematical problems numerically using computers.
In mathematics, it is sometimes possible to express the solution of an equation using a formula or to compute an integral symbolically. However, in many real-world problems, such "clean answers" are rarely obtainable. Numerical computation provides methods for finding "approximate numerical answers" to such problems.
Example: Quadratic vs. Quintic Equations
The solutions of a quadratic equation $ax^2 + bx + c = 0$ can be found using the quadratic formula:
$$x = \dfrac{-b \pm \sqrt{b^2 - 4ac}}{2a}$$However, for equations of degree five or higher there is no general formula that uses only the four arithmetic operations and radicals on the coefficients (Abel-Ruffini theorem). Except for special forms such as $x^5 - 2 = 0$, solving such an equation means resorting to numerical computation. Even for a quadratic, turning $\sqrt{b^2 - 4ac}$ into a concrete number is itself numerical computation.
Analytical Solutions vs Numerical Solutions
Example: Solving $x^2 = 2$
Analytical solution (exact solution): $x = \sqrt{2}$ — an exact solution expressed as a formula
Numerical solution (approximate solution): $x \approx 1.41421356237...$ — a concrete number
Analytical solutions are exact, but in practice there are many situations where concrete numerical values are needed. Numerical analysis is the discipline of efficiently finding such numerical solutions.
| Aspect | Analytical Solution | Numerical Solution |
|---|---|---|
| Form | Formulas / Symbols | Concrete numbers |
| Precision | Exact | Approximate (with error) |
| Applicability | Limited | Broad |
| Advantage | Reveals general properties | Provides practical values |
The Concept of Approximation
The core of numerical computation is "approximation." Rather than seeking a perfect answer, the goal is to efficiently find a "sufficiently good answer."
Why Approximation is Necessary
- Many problems cannot be solved analytically
- Computers can only handle a finite number of digits
- In practice, a certain level of precision is sufficient
Example: Approximation of Pi
Pi $\pi = 3.14159265358979...$ is an infinite decimal. However, in practice:
- Elementary school: $\pi \approx 3.14$
- Engineering calculations: $\pi \approx 3.14159$
- High-precision calculations: $\pi \approx 3.141592653589793$ (double precision)
Approximate values are used at a level of precision appropriate to the application.
Applications of Numerical Computation
Solving Equations
Finding $x$ such that $f(x) = 0$. Example: solving $x^5 - x - 1 = 0$
Numerical Integration
Computing the value of $\displaystyle\int_a^b f(x)\,dx$. Example: $\displaystyle\int_0^1 e^{-x^2}\,dx$
Differential Equations
Weather forecasting, fluid simulations, etc.
Matrix Computation
Systems of linear equations, data analysis, machine learning
Importance of Algorithms
In numerical computation, "how the computation is carried out" is extremely important. Even for the same problem, the choice of algorithm affects:
- Speed can vary dramatically
- Accuracy can differ
- Stability (accumulation of errors) can be different
Example: Finding $\sqrt{2}$ with Two Algorithms
The positive solution of $x^2 = 2$ ($\sqrt{2} = 1.41421356237...$) is computed using a slow algorithm and a fast algorithm for comparison.
Method 1: Bisection Method (slow)
Since $1^2 = 1 < 2$ and $2^2 = 4 > 2$, the solution lies in the interval $[1, 2]$. The midpoint of the interval is evaluated, and the interval is narrowed to the half containing the solution. This process is repeated.
| Step | Approximation | Error |
|---|---|---|
| 1 | 1.5 | $8.6 \times 10^{-2}$ |
| 5 | 1.40625 | $8.0 \times 10^{-3}$ |
| 10 | 1.4150390625 | $8.3 \times 10^{-4}$ |
Even after 10 steps the error is $8.3 \times 10^{-4}$ (on the order of $10^{-3}$). The interval width is exactly halved at every step (about 1 bit), but the error of the midpoint does not decrease monotonically: it first drops below $10^{-4}$ at step 12.
Method 2: Newton's Method (fast)
Starting with initial value $x_0 = 1$, the following formula is iterated:
$$x_{n+1} = \dfrac{1}{2}\left(x_n + \dfrac{2}{x_n}\right)$$| Step | Approximation | Error |
|---|---|---|
| 1 | 1.5 | $8.6 \times 10^{-2}$ |
| 2 | 1.41666... | $2.5 \times 10^{-3}$ |
| 3 | 1.41421568... | $2.1 \times 10^{-6}$ |
| 4 | 1.41421356237... | $1.6 \times 10^{-12}$ |
Only 4 steps to reach error $10^{-12}$. The number of correct digits roughly doubles with each step.
Even for the same problem, the convergence speed differs dramatically depending on the choice of algorithm. Newton's method, however, requires a computable derivative and a starting point close to the solution: the iteration above divides by zero at $x_0 = 0$, and from $x_0 = -1$ it converges to $-\sqrt{2}$ rather than the positive solution we wanted. The bisection method always converges once a bracketing interval is given, but it is slow — it needs 38 steps to reach an error of $10^{-12}$. Which method to choose depends on whether speed, accuracy or stability matters most.
Summary
- Numerical computation is the method of solving mathematical problems numerically
- Analytical solutions are formulas; numerical solutions are concrete numbers
- Many problems cannot be solved analytically, so approximation is necessary
- The choice of algorithm affects both accuracy and speed
- Numerical computation is used in various fields of science and engineering
Frequently asked questions
What is numerical computation (numerical analysis)?
Numerical computation is a general term for methods that approximately solve mathematical problems as concrete numerical values that computers can handle. It is used to solve, to practical accuracy, problems such as differential equations, integrals and large systems of equations for which no exact analytical solution can be obtained, or for which a closed-form solution exists but is not practical to evaluate by hand.
What is the difference between analytical solutions and numerical solutions?
An analytical solution is an exact solution expressed in formulas and symbols, such as $\sqrt{2}$ or $\pi$. A numerical solution is an approximate solution expressed as concrete numbers like $1.41421356\ldots$, which contains error but can be used directly in practical calculations.
In what fields is numerical computation used?
It is used across a wide range of science and engineering fields: root-finding, numerical integration, differential equations (weather forecasting, fluid simulation), and matrix computation (systems of equations, data analysis, machine learning). Many problems that cannot be solved analytically, or that require large amounts of computation, rely on numerical computation.
How much does computation speed differ depending on the algorithm?
When finding $\sqrt{2}$, the bisection method achieves an error of $8.3 \times 10^{-4}$ (on the order of $10^{-3}$) after 10 steps, while Newton's method reaches an error of $10^{-12}$ in only 4 steps. The choice of algorithm has an order-of-magnitude impact on computational efficiency.
Do numerical solutions always contain error?
In most cases, round-off error (from representing values with a finite number of digits) and truncation error (from approximating an infinite process with finitely many steps) arise. However, by choosing an appropriate algorithm, the error can be made small enough for practical purposes.