[R] comparison operator, decimals, and signif()

[(Ted Harding)]

On 22-May-05 Nick Drew wrote:
> Hi, I recently spent quite a bit of time trouble
> shooting a function that I had written only to
> discover that the problem I was having was with the
> comparison operator. I assumed that the following
> would return TRUE:
> 
>> testMean <- 82.8 + 0.1
>> testMean
> [1] 82.9
>> testMean == 82.9
> [1] FALSE
> 
> 
> Apparently this has to do with deciml places. Look:
> 
>> newTest <- 82.0
>> newTest
> [1] 82
>> newTest == 82
> [1] TRUE
>> newTest == 82.0
> [1] TRUE
>> 
> 
> What does signif() do to my object called "testMean"
> so that the comparison now evaluates to TRUE?
> 
>> signif(testMean, 3) == 82.9
> [1] TRUE

While not an exact explanation, the following strongly hints
at the underlying answer:

  > print(82.9,digits=20)
  [1] 82.9

  > print(82.8,digits=20)
  [1] 82.8

  > print(0.1,digits=20)
  [1] 0.1

  > print(82.8+0.1,digits=20)
  [1] 82.89999999999999

  > (82.8+0.1)-82.9
  [1] -1.421085e-14

So decimal arithmetic in a binary computer is not as exact
as decimal arithmetic done with pencil and paper.

The underlying reason (in part) for the above apparent anomalies
is that when 82.8 is added to 0.1, the smaller of these numbers
(in its binary "floating point" representation) is shifted along
until corresponding digits line up (the shift being accounted for
in the "exponent"). In performing the shift, binary digits are
dropped from the bottom end. Since "0.1" in binary in fact has
an infinite number of significant decimal places, inaccuracy
inevitably creeps in!

A close neighbour of the above calculations, such that there are
no discrepancies in the binary representations, illustrates the
above explanation:

  > print((82.0 + 7/8),digits=20)
  [1] 82.875

  > print((82.0 + 7/8)+1/16,digits=20)
  [1] 82.9375

  > print(82.9375,digits=20)
  [1] 82.9375

  > ((82.0 + 7/8)+1/16)-82.9375
  [1] 0

No discrepancies here! The reason is that the fractional parts
of these numbers all have exact finite (and short) binary
representations. Compare:

  > ((82.0 + 7/8)+1/16)==82.9375
  [1] TRUE

  > ((82.0 + 8/10)+1/10)==82.9
  [1] FALSE

Hoping this helps,
Best wishes,
Ted.


--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at nessie.mcc.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 22-May-05                                       Time: 09:58:00
------------------------------ XFMail ------------------------------