Matteo Foglieni https://foglienimatteo.com/blog/ Personal website of Matteo Foglieni, posting every now and then about physics and computer science (especially Julia) quarto-1.11.9 Sun, 01 Nov 2026 00:00:00 GMT Why 0.1 isn’t 0.1: floating-point rounding error (part 1) https://foglienimatteo.com/blog/floating-point-rounding-error/ References:

  • [1] D. Goldberg: “What Every Computer Scientist Should Know About Floating-Point Arithmetic”, ACM Computing Surveys, Vol. 23 Num. 1 pag. 5–48 (1991), DOI 10.1145/103162.103163

TL;DR

Notation: , etc are the computed operations; , , , etc are the computed functions. For a real number approximated by , is the absolute error and the relative error (Section 2.1).

A floating-point number with base and precision can be written normalized (i.e. with first significant digit non-zero) as:

Ulps: rounding a real number to the nearest floating-point number commits an absolute error of at most half a unit in the last place (ulp), . Relative to , the exponent cancels, and the relative error is at most the machine epsilon :

Guard digits: computing with only digits can give a relative error as large as (Theorem 1); a single extra digit brings it below (Theorem 2):

Cancellation: subtracting two nearby numbers that already carry rounding errors is catastrophic; subtracting exactly known numbers is benign. Many formulas can be rewritten so that only benign cancellations occur:

quantity naive form rewritten form
root of , , Eq.(4.2)
area of a triangle Heron’s formula, Eq.(4.4) Kahan’s formula, Eq.(4.5)
, with , Eq.(4.8)

Exact rounding (compute exactly, then round to the nearest) with round half to even keeps from drifting (Theorem 5), allows to split a number into two half-precision parts (Theorem 6), and guarantees for (Theorem 7).

1. Floating-point formats

1.1 Representation

Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation: the result of most computations with real numbers cannot be represented exactly, and must be rounded to fit back into the finite representation. The resulting rounding error is the characteristic feature of floating-point computation.

A floating-point representation has a base (always assumed to be even) and a precision . A number is written as , where the -digit number is the significand (formerly called mantissa) and is the exponent. More precisely, represents the number

The term floating-point number means a real number that can be represented exactly in the format under discussion. The exponent is bounded, .

Example: the number .

  • With and , is represented exactly as .
  • With and , cannot be represented exactly: it has a finite decimal representation, but an infinite binary one. Doubling the fractional part over and over gives its binary digits one by one:

Keeping digits (one before the binary point and 23 after it, which is what a single-precision number in IEEE 754 standard does), the first discarded digits are , more than half a unit of the last kept digit, so we round up:

where is the 24-digit significand read as an integer, and is the weight of its last digit. In this single-precision representation, lies strictly between the two consecutive floating-point numbers and

so it is exactly representable by neither of them: the closest one, , is in error by , i.e. by units of the last digit ().

1.2 Encoding

Given a base , a precision and max/min integer exponents , there are possible significands and possible exponents. So, a floating-point number can be encoded in a number of bits equal to

and where the final is the sign bit. The precise encoding is not important for now. Two examples:

exponent significand sign total
1 6
1 33
1 65

The second and the third ones have the exponent range and the precision of IEEE single and double precision respectively, which nevertheless fit in 32 and 64 bits: the leading digit of a normalized binary significand (Section 1.4) is always 1, so it does not need to be stored. This will be discussed in the next article.

1.3 Why a real number may not be representable

There are two reasons:

  1. the most common one: a real number with a finite representation in one base can have an infinite representation in another, like in Eq.(1.2). Then it lies strictly between two floating-point numbers;
  2. the number is out of range: its absolute value is larger than or smaller than .

This article deals with the first reason only; out-of-range numbers are discussed in the next article.

1.4 Normalized numbers

Floating-point representations are not unique: both and represent .

Normalized representation: if the leading digit of a floating-point representation of a number is nonzero, i.e.  in Eq.(1.1), the representation is said to be normalized

E.g.: is normalized, is not. Requiring normalization makes the representation unique.

In a normalized significand can take values and each of can take values, so the number of positive normalized floating-point numbers is

For , , , , this gives numbers. The significand is , and the spacing between consecutive numbers with exponent is one unit of the last digit, :

spacing

Normalized floating-point numbers for beta = 2, p = 3, e_min = -1, e_max = 2

Each time the exponent grows by one, the spacing doubles: floating-point numbers are not uniformly distributed on the real line, it is their relative spacing that stays (roughly) constant. The bold marks are the numbers with significand .

Normalization has a drawback: zero cannot be represented, since . A natural way to represent it is : this preserves the fact that the numerical ordering of nonnegative real numbers corresponds to the lexicographic ordering of their floating-point representations (assuming the exponent is stored to the left of the significand). It also means that, when the exponent is stored in a -bit field, only values are available for use as exponents, since one must be reserved to represent 0.

Notation: the in a floating-point number is part of the notation, and different from a floating-point multiplication. For example, involves only a single floating-point multiplication:

where the last step renormalizes the significand.

2. Relative error and ulps

Assumption: from now on in this section, unless stated otherwise, and .

2.1 Units in the last place, absolute and relative error

Ulps: if the result of a floating-point computation is e.g.  and the answer computed with infinite precision is , the result is in error by 2 units in the last place (ulps), i.e. by 2 units of its last digit.

Note that what an ulp is exactly depends on the base , the precision and the exponent of the floating-point number: it is the weight of its last digit ,

For example, with and , one ulp of is , while one ulp of is .

Absolute error: if the floating-point number is used to represent the real number , it is in error by

It is often convenient to measure the absolute error in ulps, i.e. to use as the unit of measure: ulps means . Dividing Eq.(2.1b) by Eq.(2.1a), and using :

as long as . Note that the exponent is the one of the floating-point number , not of :

If is the floating-point number nearest to , it can still be in error by as much as half an ulp.

Relative error: another way to measure the difference between a floating-point number and the real number it approximates is the relative error:

where, in the last fraction, is expressed in the same units as , not in ulps. For example:

Notation: square brackets list the parameters of the floating-point format, as in or below; round brackets list the exact value and its approximation, as in and . The errors depend on and only through .

2.2 Half an ulp and the machine epsilon

When a real number is rounded to the closest floating-point number , the absolute error can be as large as half an ulp:

That is, the number whose only nonzero digit is , in the -th position after the point; for , this is . This bound on the absolute error depends on the exponent : the larger the number, the larger the absolute error committed when rounding it.

The relative error, however, divides the absolute error by , which grows with too. All the numbers whose closest floating-point number has the form lie between and ,

so, by Eq.(2.2), an absolute error of half an ulp corresponds to the relative error

which is smallest for the largest of Eq.(2.3b), and largest for the smallest one:

The exponent cancels: in Eq.(2.3c), appears both in the numerator (the half ulp) and in the denominator (), so the bounds of Eq.(2.3e) depend only on and . What matters is only where lies between and (for : whether its significand is close to or to ), not how large is. [1] writes Eq.(2.3e) as , where the middle term is a shorthand for “the relative error corresponding to half an ulp”: it is not the half ulp of Eq.(2.3a), which does contain .

With and , the relative size of half an ulp for numbers of very different magnitude:

Half an ulp grows by a factor from to , while its relative size does not change at all; for a fixed , it varies by a factor between the smallest and the largest significand.

Machine epsilon: the relative error corresponding to half an ulp can thus vary by a factor of , which is called the wobble. Setting the machine epsilon to the largest of the bounds of Eq.(2.3e),

and since rounding to the closest floating-point number always gives , Eq.(2.3c)–(2.3e) give, for any real number in range and whatever its exponent,

i.e. when a real number is rounded to the closest floating-point number, the relative error is always bounded by . Some values:

To avoid such small numbers, relative errors are normally written as a factor times . For the approximation of of Section 2.1:

2.3 Ulps versus relative error

Consider the real number . It is approximated by ( is halfway between and ; both rounding rules of Section 5.2 give ). Next consider the computation : the exact value is , while the computed value is , which is exactly representable. Using Eq.(2.1c) and Eq.(2.2):

The error measured in ulps is eight times larger, even though the relative error is the same: the absolute error has been multiplied by 8, but the ulp has not, since and have the same exponent. In general, when the base is , a fixed relative error expressed in ulps can wobble by a factor of up to . Conversely, as Eq.(2.3e) shows, a fixed error of half an ulp results in a relative error that can wobble by .

2.4 Contaminated digits

When only the order of magnitude of the rounding error matters, ulps and can be used interchangeably, since they differ by at most a factor of . If ulps, or if , then

For example, an error of ulps (Section 4.1) contaminates digits, i.e. about 2 of the 3 digits.

The most natural way to measure the rounding error of a single operation is in ulps: rounding to the nearest floating-point number corresponds to an error of at most half an ulp. When analysing the rounding error caused by a formula, relative error is a better measure. Since can overestimate the effect of rounding to the nearest floating-point number by the wobble factor , error estimates of formulas are tighter on machines with a small .

2.5 Try it yourself

Everything above is computed right here in your browser. Type a real number , pick a base and a precision , and the box rounds to the nearest number the format can hold and reports the error four ways. The defaults reproduce the example of Section 2.3; switch to , and type 0.1 to watch a number that looks exact in decimal fail to be representable in binary. Turn off nearest to compare against an you choose yourself (e.g. , 0.0312, the two-ulp example of Section 2.1).

3. Guard digits

Notation: to avoid confusion between exact and computed values, denotes the exact difference of and , while denotes the computed difference (i.e. with rounding error). Similarly, , and denote computed addition, multiplication and division. All caps indicate the computed value of a function, as in or , while lowercase functions and the traditional notation, as in and , denote their exact values. The error of a computed difference is then, e.g., .

3.1 Exact difference, then rounding

One method of computing the difference between two floating-point numbers is to compute it exactly, then round it to the nearest floating-point number. With , is computed as

which is correct, but expensive when the operands differ greatly in size: here we needed 20 digits to produce a 3-digit result.

3.2 No guard digit

Floating-point hardware normally operates on a fixed number of digits. Suppose that only digits are kept, and that when the smaller operand is shifted right, the digits that fall off are simply discarded (truncation, not rounding). Then the example above becomes

and the answer is the same as if the difference had been computed exactly, then rounded. But take :

The correct answer is , so by Eq.(2.1c) and Eq.(2.2) the computed difference is off by

and it is wrong in every digit!

3.3 Theorem 1

How bad can the error be?

Theorem 1: Using a floating-point format with parameters and and computing differences using digits, the relative error of the result, , can be as large as .

Proof: take and , where is the largest digit and has digits, all equal to (i.e. ). Summing the geometric series:

To subtract, is aligned to the exponent of : has digits (the leading 0 and the digits ), but only are kept, so the rightmost digit of gets shifted off:

This example is also the worst case, so is an upper bound too: without guard digit, only the digits of shifted beyond the last digit of are lost; they are at most (one digit, relative to ’s exponent), while a nonzero exact difference is at least .

Two instances of the proof:

, ,
exact

When , the absolute error can be as large as the exact result itself, and when it can be nine times larger. To put it another way, for we have , so means by Eq.(2.5) contaminated digits: all of the digits of the result are wrong!

3.4 One guard digit

Suppose one extra digit is added to guard against this situation (a guard digit): the smaller number is truncated to digits, then the result of the subtraction is rounded to digits. With a guard digit, the previous example becomes

and the answer is exact. With a single guard digit, however, the relative error of the result may still be greater than , as in :

( is a halfway case; both rounding rules of Section 5.2 give ). Compared with the correct answer :

The exactly rounded result would have been , with ulps only: the truncated digit of is the one that would have decided the rounding.

3.5 Theorem 2

In general, the relative error of the result can be only slightly larger than :

Theorem 2: If and are floating-point numbers in a format with parameters and , and if subtraction is done with digits (i.e. one guard digit), then the relative rounding error in the result is less than , i.e. .

The theorem is proven in Section 4.1 of [1]. Addition is included in it, since and can be positive or negative. In the example above, .

4. Cancellation

Section 3 can be summarized by saying that without a guard digit, the relative error committed when subtracting two nearby quantities can be very large. In other words, the evaluation of any expression containing a subtraction (or an addition of quantities with opposite signs) could result in a relative error so large that all the digits are meaningless (Theorem 1). When subtracting nearby quantities, the most significant digits of the operands match and cancel each other.

4.1 Catastrophic and benign cancellation

There are two kinds of cancellation.

Catastrophic cancellation occurs when the operands are subject to rounding errors. For example, the quadratic formula contains the expression . The quantities and are subject to rounding errors, since they are the results of floating-point multiplications: even if they are rounded to the nearest floating-point number, and so accurate to within half an ulp, their subtraction can make many of the accurate digits disappear, leaving behind mainly digits contaminated by rounding error. Example with , , :

( gives the same). Calling the exact value and the computed one, by Eq.(2.1c) and Eq.(2.2):

The subtraction itself did not introduce any error, is exactly : it rather exposed the errors introduced in the earlier multiplications,

Both products are within half an ulp () of the exact values, and , i.e. their relative errors are below ; but these absolute errors are not small compared with the small difference .

Benign cancellation occurs when subtracting exactly known quantities. If and have no rounding error, then by Theorem 2, if the subtraction is done with a guard digit, the difference has a very small relative error, .

A formula that exhibits catastrophic cancellation can sometimes be rearranged to eliminate the problem. The next sections show four examples.

4.2 The quadratic formula

The roots of are

When , does not involve a cancellation and . But then the other addition (subtraction) in one of the two formulas, , subtracts two nearly equal numbers, and the square root carries a rounding error: a catastrophic cancellation. To avoid it, we multiply the numerator and denominator of by :

and similarly for , multiplying by :

If and , computing with Eq.(4.1) involves a cancellation, so we use Eq.(4.2) for and Eq.(4.1) for . If , we use Eq.(4.1) for and Eq.(4.2) for . In both cases the sum that is actually computed is , whose two terms have the same sign.

Example, not in [1]: , , .

The exact roots are and . Eq.(4.1) gives a meaningless , (), while from Eq.(4.2) and from Eq.(4.1) are both correct to all 3 digits: .

4.3 Difference of squares

The expression is another formula that exhibits catastrophic cancellation. It is more accurate to evaluate it as

This form still has a subtraction, but it is a benign cancellation of quantities without rounding error. Example, not in [1], with and (exact ):

The first result has ulps and ; the second one is exact.

By Theorem 2, the relative error of both and is at most , and multiplying two quantities with a small relative error results in a product with a small relative error (Section 4.1 of [1]): with the (signed) relative errors of the subtraction and of the addition, and the one of the multiplication, so that , , and by Eq.(2.4b),

Although does not cause a catastrophic cancellation, it is slightly less accurate than if or : in this case has three rounding errors, but has only two, since the rounding error committed when computing the smaller of and does not affect the final subtraction (footnote 5 of [1]).

Inexact data: the floating-point numbers and might themselves be approximations of some true quantities and , e.g. exactly known decimal numbers that cannot be expressed exactly in binary. Then, even though is a good approximation of , it can have a huge relative error compared with the true . For example, if and are stored as and , then is exact, but , so . So the advantage of over is not as dramatic with inexact data; since computing it is about the same amount of work, it is still clearly the preferred form. In general, replacing a catastrophic cancellation by a benign one is not worthwhile if the expense is large, because the input is often (not always) an approximation. But eliminating a cancellation entirely, as in the quadratic formula, is worthwhile even if the data are not exact.

Assumption: from now on, as in [1], the floating-point inputs to an algorithm are exact, and the results are computed as accurately as possible.

4.4 Area of a triangle

The area of a triangle can be expressed directly in terms of the lengths of its sides , and with Heron’s formula:

Suppose the triangle is very flat, i.e. . Then , and the term subtracts two nearby numbers, one of which () may have a rounding error. Example with , :

Even though the computed value is in error by only ulps, the computed area is in error by ulps, i.e. . (Rounding the halfway case up to instead gives , , and again .)

Misprint in [1]: the ACM version states “an error of 60 ulps”; the error is ulps.

The order of the additions in matters too: gives , , and .

There is a way to rewrite formula (4.4) so that it returns accurate results even for flat triangles (Kahan, 1986):

If , and do not satisfy , simply rename them before applying Eq.(4.5). Eq.(4.4) and Eq.(4.5) are algebraically identical:

and the square root of , divided by 4, is . The parentheses do not matter in exact arithmetic, but they decide the order of the floating-point operations. With the same values as above:

The computed area is in error by ulps, about 1 ulp, much more accurate than Heron’s formula: . The difference is in the small factor: in Heron’s formula, cancels , which already carries a rounding error, while in Kahan’s formula the corresponding factor is computed exactly from exactly known data: both cancellations are benign.

Although formula (4.5) is much more accurate than (4.4) for this example, it would be nice to know how well (4.5) performs in general:

Theorem 3: The rounding error incurred when using Eq.(4.5) to compute the area of a triangle is at most , i.e. , provided subtraction is performed with a guard digit, , and square roots are computed to within half an ulp.

The condition is met in virtually every actual floating-point system:

In statements like Theorem 3, the relative error is that of the expression computed with floating-point arithmetic, i.e. of

Because of the cumbersome nature of Eq.(4.6), [1] usually says “the computed value of ” rather than writing out with circle notation.

Error bounds are usually too pessimistic: in the example above , much less than . The main reason for computing error bounds is not to get precise bounds, but rather to verify that the formula does not contain numerical problems.

4.5 Compound interest and

A final example of an expression that can be rewritten to use benign cancellation is , where . This expression arises in financial calculations. Consider depositing 100 USD every day into a bank account that earns an annual interest rate of , compounded daily, so that and the daily rate is . A deposit made days before the end of the year has grown to , so the amount of money accumulated at the end of one year (deposits at the end of each day, ) is the geometric sum

Computed exactly:

Computed with and (single precision), the result is 37615.45 USD, a discrepancy of 1.40 USD. The reason is easy to see: involves adding 1 to , so the low-order bits of are lost, and this rounding error is amplified when is raised to the -th power. In detail:

Step 1: near 1, the floating-point numbers are spaced by , so

The computation actually uses the daily rate : since , only 11 of the 24 bits of survive, and

where .

Step 2: raising to the -th power multiplies the relative error by :

Step 3: subtracting 1 amplifies the relative error again, by a factor , giving , i.e. an absolute error on the amount of USD. Evaluating Eq.(4.7) exactly, but with instead of in the power, gives 37615.47 USD; the remaining single-precision roundings give the 37615.45 USD of [1].

The troublesome expression can be rewritten as , where now the problem is to compute for small .

  • One approach is to use the approximation , in which case the payment becomes 37617.26 USD, with an absolute error of 3.21 USD: even less accurate than the obvious formula. The approximation drops the term of , i.e.  in the exponent, again amplified by the factor 17.2 of Step 3. (In exact arithmetic, ; single precision gives , which [1] reports as 37617.26.)
  • There is a way to compute accurately, as Theorem 4 shows (Hewlett-Packard, 1982). This formula yields 37614.07 USD, accurate to within 2 cents!

Theorem 4: If is computed using formula (4.8), the relative error is at most , i.e. , when , provided subtraction is performed with a guard digit, , and is computed to within half an ulp.

Misprint in [1]: the ACM version states “If is computed using the formula”; it is .

In exact arithmetic the second line of Eq.(4.8) is just ; what matters is that it is computed as . Theorem 4 assumes that approximates to within half an ulp. The problem it solves is that when is small, is not close to , because has lost the information in the low-order bits of . That is, the computed value of is not close to its actual value when .

The formula works for any value of , but it is only interesting for , which is where catastrophic cancellation occurs in the naive formula . Although it may seem mysterious, there is a simple explanation for why it works. Write

  • The left factor can be computed exactly, but the right factor will suffer a large rounding error when adding 1 to .
  • However, is almost constant, since : by Eq.(4.9), . So changing slightly will not introduce much error: if , computing is a good approximation of .
  • Is there a value for which and can be computed accurately? There is: , because then is exactly equal to (and the subtraction is a benign cancellation, exact with a guard digit). So is an accurate value of .

In the compound interest example (single precision):

quantity value vs exact
, exact
, exact
, exact

The error of that ruins changes by only .

The same mechanism with , , and (not in [1]):

The naive result has , the one of Theorem 4 : differs from by 19%, but differs from by only 0.11%.

4.6 The price of a guard digit

The results of this section can be summarized by saying that a guard digit guarantees accuracy when nearby precisely known quantities are subtracted (benign cancellation). Sometimes a formula that gives inaccurate results can be rewritten to have much higher numerical accuracy by using benign cancellation; however, the procedure only works if subtraction is performed using a guard digit.

The price of a guard digit is not high, because it merely requires making the adder 1 bit wider: for a 54-bit double precision adder, the additional cost is . For this price, you gain the ability to run many algorithms, such as Kahan’s formula (4.5) for computing the area of a triangle ([1] says formula (6), but it is the one that requires the guard digit, i.e. their (7)) and the expression of Theorem 4 for computing . Although most modern computers (in 1991) have a guard digit, there are a few, such as Crays, that do not.

5. Exactly rounded operations

5.1 Definition

When floating-point operations are done with a guard digit, they are not as accurate as if they were computed exactly, then rounded to the nearest floating-point number. Operations performed in this manner are called exactly rounded. The example of Section 3.4 shows that a single guard digit does not always give exactly rounded results ( instead of ). Section 4 gave several examples of algorithms that require a guard digit in order to work properly; this section gives examples of algorithms that require exact rounding.

5.2 Halfway cases: round half up or round half to even

Rounding is straightforward, with the exception of how to round halfway cases: should round to or ?

  • Round half up: one school of thought divides the 10 digits in half, letting round down and round up; thus rounds to . This is how rounding works on Digital Equipment Corporation’s VAX computers.
  • Round half to even: since numbers ending in 5 are halfway between two possible roundings, they should round down half of the time and round up the other half. One way of obtaining this behaviour is to require the rounded result to have its least significant digit even: rounds to , because 2 is even.

Reiser and Knuth (1975) offer the following reason for preferring round to even:

Theorem 5: Let and be floating-point numbers, and define , , …, . If and are exactly rounded using round to even, then either for all , or for all .

[1] does not include a proof. To clarify the result, take , , , :

All the exact intermediate results have 4 digits ending in 5, i.e. they are halfway cases: round half up rounds all of them up, so each successive value of increases by , while round half to even rounds up once and down once. How far does the drift go? (not in [1]): steps, up to . There, is no longer a halfway case (the result has exponent 1, and its last digit has weight ), and .

This example suggests that when using the round up rule, computations can gradually drift upward, whereas when using round to even the theorem says this cannot happen. Throughout the rest of [1], round to even is used.

5.3 Multiple precision: splitting a number

One application of exact rounding occurs in multiple precision arithmetic. There are two basic approaches to higher precision:

  1. represent floating-point numbers using a very large significand, stored in an array of words, and code the routines for manipulating these numbers in assembly language;
  2. represent higher precision floating-point numbers as an array of ordinary floating-point numbers, where adding the elements of the array in infinite precision recovers the high precision floating-point number.

[1] discusses the second approach: it can be coded portably in a high-level language, but it requires exactly rounded arithmetic.

The key to multiplication in this system is representing a product as a sum, where each summand has the same precision as and . This can be done by splitting and : writing and ,

If and have -bit significands, the summands will also have -bit significands, provided , , , can be represented using bits (the product of two numbers with bits has at most bits). When is even, it is easy to find a splitting: the number can be written as the sum

When is odd, this simple splitting method will not work, since one of the two halves has digits. An extra bit can, however, be gained by using negative numbers. For example, with , and :

where has 2 significant bits and has 1, both at most , while the digit splitting would need 3 significant bits for the second part.

There is more than one way to split a number. A splitting method that is easy to compute is due to Dekker (1971), but it requires more than a single guard digit:

Theorem 6: Let be the floating-point precision, with the restriction that is even when , and assume that floating-point operations are exactly rounded. Then if is half the precision (rounded up) and , can be split as , where and are given by Eq.(5.2), and each of them is representable using bits of precision.

To see how this theorem works, take , , , , , and compute :

By Eq.(2.1c) with , the naive result is in error by ulps. Now split , and with Eq.(5.2): and .

So , , , and with Eq.(5.1) each product is exact with digits:

where the sums are left unevaluated. Finally, subtracting these two series term by term:

which is identical to the exactly rounded result.

Misprint in [1]: the ACM version prints “”; the last term is .

To show that Theorem 6 really requires exact rounding, take , and . Then , , and

  • with exact rounding: , so and : both representable with bit;
  • with a single guard digit: is truncated to digits, , and , exactly representable. So , and , which is not representable with bit.

5.4 Theorem 7

As a final example of exact rounding, consider dividing by 10. The result is a floating-point number that will in general not be equal to . When , however, multiplying by 10 will miraculously restore , provided exact rounding is being used. Actually, a more general fact (due to Kahan) is true:

Theorem 7: When , if and are integers with and has the special form , then , provided floating-point operations are exactly rounded.

The proof is ingenious (see [1], pp. 14–15), and skipped here. For the example above, . The theorem holds true for any base , as long as is replaced by . As gets larger, however, there are fewer and fewer denominators of this form. Up to 100:

count
2 2, 3, 4, 5, 6, 8, 9, 10, 12, 16, 17, 18, 20, 24, 32, 33, 34, 36, 40, 48, 64, 65, 66, 68, 72, 80, 96 27
10 2, 11, 20 3

Numerical check in IEEE double precision (, ), where m/n*n in most programming languages computes exactly (not in [1]):

since the double precision numbers between 2 and 4 are spaced , and the exact product is only below 3. Counting the failures of for :

binary form failures
3 0
10 0
12 0
7 74892
49 160502
100 146800

For example, .

5.5 Does it matter?

We are now in a position to answer the question: does it matter if the basic arithmetic operations introduce a little more rounding error than necessary? The answer is that it does matter, because accurate basic operations enable us to prove that formulas are “correct”, in the sense that they have a small relative error. Section 4 discussed several algorithms that require guard digits to produce correct results in this sense. If the inputs to those formulas are numbers representing imprecise measurements, however, the bounds of Theorems 3 and 4 become less interesting. The reason is that the benign cancellation can become catastrophic if and are only approximations of some measured quantity. But accurate operations are useful even in the face of inexact data, because they enable us to establish exact relationships like those of Theorems 6 and 7. These are useful even if every floating-point variable is only an approximation of some actual value.

6. Reproducing the computations

All the base-10 examples above can be reproduced with Python’s decimal module: a Context(prec=p) performs exactly rounded , , , , and with significant digits, and the rounding of halfway cases can be chosen (ROUND_HALF_EVEN or ROUND_HALF_UP). The truncating subtraction of Section 3 (with or without guard digit) has to be written by hand. The single precision examples can be reproduced with numpy.float32, computing and in double precision and rounding them to single, so that they are accurate to within half an ulp as assumed by Theorem 4 (numpy’s own float32 exponential is off by 2 ulps in this example, which changes the result by about 7 cents).

from decimal import Decimal as D, Context, ROUND_HALF_EVEN
c3 = Context(prec=3, rounding=ROUND_HALF_EVEN)      # beta = 10, p = 3, exactly rounded

# catastrophic cancellation, Section 4.1
b, a, c = D("3.34"), D("1.22"), D("2.28")
bb  = c3.multiply(b, b)                             # 11.1556 -> 11.2
ac4 = c3.multiply(D(4), c3.multiply(a, c))          # 4 * 2.78 = 11.12 -> 11.1
print(c3.subtract(bb, ac4), b*b - 4*a*c)            # 0.1   0.0292

# Kahan's formula (4.5), Section 4.4
a, b, c = D("9.0"), D("4.53"), D("4.53")
amb  = c3.subtract(a, b)                            # 4.47, exact
prod = c3.multiply(c3.multiply(c3.multiply(
           c3.add(a, c3.add(b, c)), c3.subtract(c, amb)),
           c3.add(c, amb)), c3.add(a, c3.subtract(b, c)))
print(c3.divide(c3.sqrt(prod), D(4)))               # 2.35

# compound interest, Section 4.5: beta = 2, p = 24
import numpy as np
f = np.float32
j = f(0.06) / f(365)
w = f(1) + j                                        # 1 + 1379 * 2^-23
P = f(1)
for _ in range(365):
    P = P * w                                       # (1 (+) j)^n
print(f(100) * ((P - f(1)) / j))                    # 37615.453

LN  = lambda v: f(np.log(np.float64(v)))            # ln, exp correctly rounded to float32
EXP = lambda v: f(np.exp(np.float64(v)))
ln1p = j * LN(w) / (w - f(1))                       # Theorem 4
P4 = EXP(f(365) * ln1p)
print(f(100) * ((P4 - f(1)) / j))                   # 37614.074
]]> math computer science https://foglienimatteo.com/blog/floating-point-rounding-error/ Sun, 01 Nov 2026 00:00:00 GMT The algebra of a loan: where the amortization formulas come from https://foglienimatteo.com/blog/bank-loan-interest-derivation/ Notation:

  • : loan amount
  • : monthly installment
  • : nominal annual interest rate
  • : monthly interest rate, 30/360 convention
  • : total number of installments
  • : total paid (loan + interests)
  • : total interests paid

For the -th installment:

  • : debt left after the -th installment ()
  • : principal part of the -th installment
  • : interest part of the -th installment

Here a summary of all the equations derived in this article:

Check: amortization condition

The debt decreases iff:

If , then for all and the loan is never repaid.

Debt left to be payed after installments

Each month the bank adds the interest on the current balance, then subtracts the installment:

With simple algebra:

From this equation, we can easily derive the relation between the total number of installments , the loan , the interest rate and the installment by imposing the condition (i.e. the debt must be repaid at the end of last installment).

We’ll derive all the expressions later.

Equivalent derivation from fixed-point theorem

The fixed point of is , and the deviation from it grows geometrically:

Total number of installments

To get the number of installments needed to repay a debt, we can set and get the value :

In the previous expression, is real-valued => the (integer) number of installments is . However, the last installment, that pays the remaining debt plus one month of interest on it, is smaller than (or equal to) the standard :

Installment and Loan from the other terms

Recalling the equation we derived by setting , we can get all the inverses:

Installment from , ,

Loan from , ,

Interest rate from , , : no closed form

Substitute into the present-value condition:

This is a polynomial of degree in . For there is no general solution in radicals (Abel–Ruffini), and is usually in the hundreds. The same equation defines the APR / internal rate of return.

Existence and uniqueness of a positive rate: is strictly increasing on , with and . So there is exactly one positive root . It satisfies , i.e. , iff

Numerically, in terms of : the present value

is strictly decreasing in . Bisection on therefore converges to the unique root, in steps for width and tolerance .

For small , evaluate as -expm1(-N*log1p(i)) to avoid cancellation.

Principal part and Interest part

Recalling the definitions of the principal part and interest part of each installment

we can get their expressions by inserting the fixed-point form of into :

The principal part grows geometrically, so the interest part shrinks.

Total paid and total paid interest

We can get the total paied and the total interest paid easily (the total principal paid is trivial):

Treating as continuous gives , with an error below .

Function definitions

Analytical definitions

Note: for numerical reasons, it’s better to rewrite some terms as follows and use the numerical functions and :

]]> math https://foglienimatteo.com/blog/bank-loan-interest-derivation/ Sun, 18 Oct 2026 00:00:00 GMT What a loan really costs you https://foglienimatteo.com/blog/bank-loan-interest-plots/ Here are some interactive plots of what a loan actually costs you. Most of a loan’s early payments are almost all interest, with only a trickle going to the principal; the balance tips the other way only later on. Pick a loan and a rate below, then either set the monthly installment and read off how long you will be paying, or set the term and get the installment it implies. The curves and the summary follow as you drag, and hovering a curve reads off its values.

Notation:

  • : loan amount
  • : monthly installment
  • : nominal annual interest rate
  • : monthly interest rate, 30/360 convention
  • : total number of installments
  • : total paid (loan + interests)
  • : total interests paid

For the -th installment:

  • : debt left after the -th installment ()
  • : principal part of the -th installment
  • : interest part of the -th installment

The formulas used here are derived in Bank interest functions: analytical derivation.

Your loan

Interest part of each installment

How much of the -th installment is interest, , for a loan of repaid with a month. The blue curves are fixed annual rates from 1% to 6%, the orange one is your rate. A rate whose first month of interest exceeds the installment has no curve: that loan is never repaid.

Total interest paid

Total interest as a fraction of the loan, against the monthly installment, for a loan of . The curves leave the plot at 100%: beyond that you pay back more than twice what you borrowed.

The numbers, year by year

Your loan, one row per year.

The same two plots as images, for a loan of 100000 €.

Interest part of each installment for a monthly installment of 500 € (left) and 1000 € (right)

Total interest paid as a fraction of the loan, against the monthly installment
]]> math https://foglienimatteo.com/blog/bank-loan-interest-plots/ Sun, 18 Oct 2026 00:00:00 GMT Numerov’s method: solving y″ = f without the first derivative https://foglienimatteo.com/blog/numerov-method/ References:

  • [1] J. L. M. Quiroz González, D. Thompson: “Getting started with Numerov’s method”, Computers in Physics, Vol. 11 Num. 5 pag. 514–515 (1997), DOI 10.1063/1.168593

TL;DR

Equation to be solved numerically

Assumptions: we suppose the following:

  • to have a set of values EQUALLY SPACED by
  • to know all the and
  • defining , to know and

For the first step

The methods has a global error of

1. Introduction

Numerov’s method is an efficient algorithm for solving second-order differential equations of the following form:

Numerov’s method relies on two properties of Eq.(1.1):

  1. the first derivative is absent;
  2. is linear wrt .

Examples in physics of this type of equations:

  • 1D time-independent Schrödinger equation. From :

  • Undamped forced harmonic oscillator ( plays the role of time):

  • Poisson equation with spherically symmetric distribution in SI units:

Cast into the form of Eq.(1.1):

equation of Eq.(1.1)
(1.2) Schrödinger
(1.3) forced oscillator
(1.4) Poisson, spherical

2. Derivation

We now want to apply the central difference scheme to Eq.(1.1).

Assumption: We suppose to have a set of values EQUALLY SPACED by , and we define .

We can then apply the Taylor expansion to , forward () and backward ():

Add the two expansions. All odd powers of cancel (symmetric stencil), so the first neglected term is :

Defining the right-hand side of Eq.(1.1), which is linear in , as

and on the grid , with and .

Unfortunately, Eq.(2.4) is not yet a usable scheme, we cannot compute analytically from its definition:

because that requires derivatives of and and the slope , which the scheme does not carry.

However, is itself smooth, so Eq.(2.1)–(2.2b) hold with :

We then substitute Eq.(2.5c) into Eq.(2.4):

We didn’t finish yet, because Eq.(2.6) is implicit: contains the unknown . Splitting according to its definition Eq.(2.3a), and collecting the terms on the left:

Divide by , which is non-zero for and leaves the remainder :

  • The right-hand side needs only , (hence , , already computed in the previous steps) and , : one evaluation of and per step, for a local error .
  • Linearity in is used only in this step: for a general , Eq.(2.6) is a nonlinear equation for (root finding at every step).
  • Eq. (2.8) is a two-step recursion: it needs and to start, which is the subject of Section 3 (the paper’s Eqs. (8)–(15)).

2.1 Leading local error term

We can make one step further to get the leading error term of Eq.(2.8). Expanding Eq.(2.1) and Eq.(2.5a) two orders further,

using the definition of in Eq.(2.3b),

rederiving Eq.(2.5c) from Eq.(2.5a) but keeping one step further,

and comparing Eq.(2.11b) with (2.9c) we get that the in Eq.(2.6) and Eq.(2.8) is:

2.2 Global Error

At first glance, one could say that the global error of Numerov’s method is :

  • to reach a fixed , you take steps;
  • if each step adds an error ,
  • and if the errors just pile up over all the points,
  • the total error is

This counting rule works for one-step methods for first-order ODEs, like Euler or Runge–Kutta. There the step error lands directly on the value , so the global error is the local error divided by .

However, Numerov is a two-step recursion for a second-order equation. Its local error doesn’t land on , but on the second difference, the discrete . It’s like an error in acceleration rather than in position, and such an error in acceleration gets integrated twice before it shows up in position. In other words, one factor of h is lost per “integration”: becomes for the slope, then for the value.

Redoing the original argument to get Numerov’s method global error s :

  • to reach a fixed , you take steps;
  • if each step adds an error ,
  • and if the errors influence quadratically over all the points
  • the total error is

3. Initial conditions

Eq.(2.8) is a two-step recursion: to compute we need the two previous values and . Tha means, to start the algorithm we need both and .

Assumption: As initial conditions we know the value , which gives , and the slope . Without loss of generality, .

How accurate must be? To compute at with Eq.(2.8), we would need with the same accuracy. However, it is sufficient to compute with a precision : is computed only once, and a single error, injected once, grows only linearly, which means that its globally propagated error would be still of the same order of the global error.

The best we can do from scratch, without any analytical differentiation, is to Taylor-expand about , using , and :

The problem is then: how to estimate and ?

3.1 A naive first step

A first attempt is to replace with the forward difference and to drop the term by including it into the error (which goes from to ):

As in Eq.(2.6), contains the unknown , and we collect it on the left:

Eq.(3.3) is one order short: at costs a factor in the global error, which becomes . In the literature, three main alternatives can be found:

  1. compute analytically,which is not always practical;
  2. compute from Eq.(3.1) without and , get a first estimate of with Eq.(2.8), use it to estimate , and iterate until and stabilise;
  3. perform the first step with a different self-starting algorithm (e.g. Runge–Kutta), then switch to Numerov.

None of them is necessary.

3.2 The starting formula

In [1], the authors find a better approach.

Suppose for a moment that we somehow obtained an estimation for , hence also for . Using , and we can estimate, via e.g. forward difference, . That means, we look for in the form:

The constants , , are fixed by requiring that Eq.(3.4) agrees with Eq.(3.1).

We expand via Taylor series and about ,

which means that

This gives from Eq.(3.1)

and comparing term by term Eq.(3.4b) with Eq.(3.1):

needs , which is given by the standard Numerov step Eq.(2.8) with :

Eqs.(3.5) and (3.6) are two linear equations in the two unknowns and . Splitting and , we get

i.e. we have to solve the linear system

Solving with Cramer’s rule:

where:

We get then the final expression:

Eq.(3.8) gives at from the initial conditions , and from , at and only: no iteration and no analytical derivatives are needed.

Then, follows from Eq.(3.6), and Eq.(2.8) continues for .

3.3 Numerical check

Test problem: and chosen so that the exact solution is , integrated on with Eq.(2.8). Observed orders of the errors , for :

starting value error on global error at
paper’s Eq.(10), as printed
Eq.(3.3), sign corrected
Eq.(3.8)

4. See the orders for yourself

The table above is really a set of claims about how fast the error shrinks as the grid is refined. Here they are, live — the method runs in your browser. Pick a test problem, choose how the first step is seeded, and slide the number of intervals (so the step is ). The orange dots are the Numerov solution, the grey line the exact one.

The real payoff is the next plot. It runs each of the three starting formulas over a whole range of step sizes and plots the error against on log–log axes, where a method of order shows up as a straight line of slope . The naive first step drags the whole scheme down to first order; Eq. (3.3) gives third order; only Eq. (3.8) keeps the full fourth order the recursion is capable of — the three claims of the table. At the smallest steps the fourth-order line bends and flattens: that is rounding error, the subject of another post, hitting its floor — not the method failing.

]]> math physics https://foglienimatteo.com/blog/numerov-method/ Wed, 14 Oct 2026 00:00:00 GMT Basics of Numerical Analysis https://foglienimatteo.com/blog/basics-numerical-analysis/ References:

[1] Wikipedia page “Finite Difference” [2] Wikipedia page “Finite difference coefficient”

TL;DR

We summarise the important equations derived in this article, for a function that is times differentiable:

Rewritten using the index notation and and assuming that all the elements are equally spaced by , for the first derivative:

and for the second:

Reminder: big- and little- notation

We remind the definitions of big- and little-. In this article, we focus only on real numbers with the limit , so we will show their definitions in this context.

Big- Given two real functions , we define as big- of if, in the limit , there exists a positive real number such that, for

In simpler words: “ is something that, divided by , is like a constant ”.

Little- Given two real functions , we define as little- of if, in the limit , their ratio goes to zero:

In simpler words: “ is something that, divided by , goes to ”.

Their relation is not always trivial: you cannot replace mechanically with ; e.g., if we consider a quantity :

The first statement follows because ; the second is false because .

For finite differences, however, the leading error is often proportional to integer powers of (like , , etc), so the relation holds in this context.

Taylor’s theorem

Taylor’s theorem Given a function that is times differentiable at the point , there exists a function such that

The polynomial appearing in Taylor’s theorem is the -th order Taylor polynomial of the function at the point

Using the little- and big- notation, the statement in Taylor’s theorem reads as

NOTE: the big- notation in (1.3b) has slightly stronger assumptions than the little- one in (1.3a): the former requires that is differentiable times at instead of times.

We can rewrite the theorem in a slightly different notation, with the substitutions and

Rewritten using the index notation and and assuming that all the elements are equally spaced by

First order finite differences

The derivative of a function at a point is defined by the limit:

More precicely, it is said that the derivative at a point is the limit in (2.1a) if both the right and the limit exists, are finite and coincide, and such value is the limit of (2.1a)

We define the forward difference , backward difference and central difference as:

We can approximate the derivative of a function (assuming is twice differentiable, three times for centered-difference) using each of the previous three differences. Applying Taylor’s theorem, we can get the error of each approximation:

To summarize:

Rewritten using the index notation and and assuming that all the elements are equally spaced by

Unfortunately, the central difference has (and will for every odd ) multiplied by the non-integer . This is often a problem because it amounts to changing the interval of discretization, but we can remedy by applying the Taylor theorem of Eq.(1.4):

which for the sum leads to:

and for the difference leads to:

Second order finite differences

In an analogous way, one can obtain finite difference approximations to higher order derivatives and differential operators.

Analogously for backward and forward differences:

Note that we can also rewrite it in the following way:

So, to summarize:

Rewritten using the index notation and and assuming that all the elements are equally spaced by

Higher order finite differences

More generally, the -th order forward, backward, and central differences are given by the following expressions, which we will not derive here:

And this is just the top of the iceberg: Ref.[2] shows the whole set of coefficients to approximate the -th derivative with an error of order , for both forward, backward and central differences:

]]> math physics https://foglienimatteo.com/blog/basics-numerical-analysis/ Mon, 12 Oct 2026 00:00:00 GMT Rolling my own cubic spline (because Dierckx wouldn’t run on the GPU) https://foglienimatteo.com/blog/spline-gapse/

MySpline

As a side project, I was working together with my colleague Salvatore Cielo on the parallelisation of the Julia code I wrote in my Master thesis, GaPSE.jl.

After some research, I find out a package, KernelAbstractions.jl, that allows to write kernels architecture agnostic, i.e. that work on CPUs and on every major GPU backend (Metal.jl, CUDA.jl, oneAPI.jl and AMDGPU.jl)

As a MWE:

# Equivalent sintazes are available in oneAPI, Metal and AMDGPU
using CUDA, KernelAbstractions, Test
const backend = CUDA.CUDABackend()
const DevArray = CuArray

a = ones(1024)
dev_a = DevArray(a)
dev_b = similar(dev_a)

@kernel function my_kernel!(b, a, num)
    i = @index(Global, Linear)
    b[i] = a[i] + num
end

compiled_kernel! = my_kernel!(backend, 32)   # 32 is the group size
compiled_kernel!(dev_b, dev_a, 2; ndrange=sizeof(dev_a))

b = Array(dev_b)    
@test all(b.≈3)

In doing so, I discovered that I couldn’t offload to the GPU the Dierckx splines I was using. As an interesting exercise and with the help of my wonderful colleague Ivan Pribec, I decided to try to implement the code for 1D Splines from scratch.

The mathematical procedure I implemented in GaPSE.jl is based on: Parviz Moin, “Fundamentals of Engineering Numerical Analysis” (2010), Cambridge University Press, Second edition, Chapter 1.2 (“Cubic Spline Interpolation”)

Derivation of the equation system for the cubic spline

Let us suppose to have a set of data points that we want to use to create our cubic spline. Let be the cubic in the interval and let denote the collection of all the cubics for the entire range of x. Since is piecewise cubic, its second derivative is piecewise linear. For the interval , we can then write the equation for the corresponding straight line as

Integrating two times and imposing to match the known values at the endpoints and , we get:

where .

The are the unknowns, and applying the continuity of the first derivative condition , we get

We have then equations for unknowns. The remaining ones are the “end conditions” that one may want to apply. The most common ones are:

  • Free run-out or Natural spline:
    • most commonly used;
    • this spline is the smoothest interpolant (i.e.  is the smallest possible)
  • Parabolic run-out:
    • the interpolating polynomials in the first and last intervals are parabolas rather than cubics
  • Combination of the first two:
  • Third Derivative continuity: for this look at the appropriate section

Using the coefficients , , , , and for this Initial Conditions (IC), we can write the complete set of equation in the matrix form:

Apply the TDMA

The equations are in tridiagonal form and diagonally dominant, so we can easily apply the TriDiagonal Matrix Algorithm (TDMA) to this system

with the following coefficients:

and the corresponding IC values:

  • Free run-out:

  • Parabolic run-out:

  • Combination of first two run-out:

  • Third Derivative continuity:

Then, we can compute the new coefficients , , and , that give us a new simplified matrix:

with:

Even better, we can divide everything for , and obtain:

with

The unknowns are given from:

Final spline polynomial coefficients

The final expression for is:

where:

NOTE: the coefficients , and are tuples! We defined just to get rid of and write and in function of only. However, that has no importance after the recursive computation, and you can get rid of it.

]]> math physics https://foglienimatteo.com/blog/spline-gapse/ Sat, 26 Apr 2025 00:00:00 GMT