# [R] Interesting quirk with fractions and rounding / using == for floating point

J C Nash profjcnash at gmail.com
Sun Apr 23 14:49:59 CEST 2017

```For over 4 decades I've had to put up with people changing my codes because
I use equalities of floating point numbers in tests for convergence. (Note that
tests of convergence are a subset of tests for termination -- I'll be happy to
explain that if requested.) Then I get "your program isn't working" and find
that is is NOT my program, but a crippled version thereof.

But I don't use equality tests (i.e., ==) blindly. My tests are of the form

if ( (x_new + offset) == (x_old + offset) ) { # we cannot get more progress

Now it is possible to imagine some weird cases where this can fail, so an
additional test is needed on, say, maximum iterations to avoid trouble. But
I've not seen such cases (or perhaps never noticed one, though I run with
very large maximum counters). The test works by having offset as some
modest value. For single precision I use 10 or 16, for double around 100.

When x_new and x_old are near zero, the bit pattern is dominated by offset and
we get convergence. When x_new is big, then it dominates.

Why do I do this? The reason is that in the early 1970s there were many, many
different floating point arithmetics with all sorts of choices of radix and
length of mantissa. (Are "radix" and "mantissa" taught to computer science
students any more?). If my programs were ported between machines there was a
good chance any tolerances would be wrong. And users -- for some reason at the
time engineers in particular -- would say "I only need 2 digits, I'll use 1e-3
as my tolerance". And "my" program would then not work. Sigh. For some reason,
the nicely scaled offset did not attract the attention of the compulsive fiddlers.

So equality in floating point is not always "wrong", though it should be used
with some attention to what is going on.

Apologies to those (e.g., Peter D.) who have heard this all before. I suspect
there are many to whom it is new.

John Nash

On 2017-04-23 12:52 AM, Paul Johnson wrote:
>
> 1 Don't use == for floating point numbers.
>

```