Loss of Significance (Absorption)

1. Definition

Loss of significance (also called absorption) occurs when adding or subtracting two floating-point numbers of vastly different magnitudes. The trailing digits (some or all significant digits) of the smaller number are lost.

2. Mechanism

Consider $1.000 \times 10^4 + 5.678 \times 10^0$ in 4-digit decimal floating-point:

The smaller number $5.678$ is completely absorbed, and the addition has no effect.

3. Loss of Significance in Summation

When computing the sum of many numbers, loss of significance worsens as the running total grows. For example, computing $\displaystyle\sum_{k=1}^{n} 1/k$ from $k = 1$ forward, the running sum eventually dwarfs $1/k$ for large $k$. Summing in reverse ($k = n$ to $1$) is more accurate because small values are added together first.

4. Countermeasures

Sorted Summation

Sum values in order of increasing absolute value to minimize the scale difference between the running total and each term. Simple to implement but not optimal.

Kahan Compensated Summation

Accumulate the lost trailing digits in a compensation variable $c$ and apply it in the next addition:

function kahanSum(values):
    sum = 0.0
    c   = 0.0
    for v in values:
        y = v - c         // apply previous compensation
        t = sum + y       // sum is large, so y's low digits are lost
        c = (t - sum) - y // record what was lost in c
        sum = t
    return sum

The compensation variable $c$ records the low-order digits of $y$ that were absorbed by the running total when computing $\text{sum} + y$. In the next iteration, $y = v - c$ subtracts back the lost portion before adding, recovering the precision. Kahan summation has $O(\varepsilon_{\text{mach}})$ error, independent of the number of terms $n$.

Pairwise Summation

Split the array in half and recursively sum each half, then add the two sums. Error is $O(\log n \cdot \varepsilon_{\text{mach}})$ with good cache efficiency. NumPy's sum uses pairwise summation.

Sign-Separated Summation

When the data contains both positive and negative values, any of the methods above can still produce catastrophic cancellation whenever the running sum crosses near zero. A better approach is to separate positives from negatives, sum each group in increasing absolute value, and subtract once at the end:

1. Accumulate positive values in increasing absolute value → $P$
2. Accumulate negative values in increasing absolute value → $N$
3. Compute $P + N$ (a single subtraction)

Each partial sum has a fixed sign, so cancellation between nearly equal values cannot occur during accumulation; cancellation is concentrated in the final subtraction alone. Loss of significance (small low-order digits being absorbed by a growing running total) can still occur within each partial sum. Kahan and pairwise summation are effective against loss of significance but powerless against cancellation when the running sum crosses zero; sign separation covers exactly that gap, so combining them yields complementary protection:

• Sign-separated + Kahan: compute $P$ and $N$ each via Kahan summation. Loss of significance within each partial sum is compensated, leaving an error of $O(\varepsilon_{\text{mach}})$; only the final subtraction contributes a single cancellation event.
• Sign-separated + pairwise: compute $P$ and $N$ each by recursively halving the sequence. The error of each partial sum is $O(\log n \cdot \varepsilon_{\text{mach}})$, faster than the Kahan variant (no compensation arithmetic).

5. Comparison with Catastrophic Cancellation

6. Worked Examples

a = 1.0e16
b = 1.0
print(a + b)      // Output: 10000000000000000.0 (b absorbed)
print(a + b == a) // Output: true

7. Frequently Asked Questions

8. References

• Wikipedia, "Catastrophic cancellation" (English Wikipedia merges "Loss of significance" into this article)
• Wikipedia, "Kahan summation algorithm"
• D. Goldberg, "What Every Computer Scientist Should Know About Floating-Point Arithmetic," ACM Computing Surveys, 23(1), 1991.
• N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., SIAM, 2002.