From bbolker at gmail.com  Sat Feb  5 21:58:09 2011
From: bbolker at gmail.com (Ben Bolker)
Date: Sat, 5 Feb 2011 15:58:09 -0500
Subject: [Rd] print(...,digits=2) behavior
Message-ID: <4D4DB9E1.4000306@mcmaster.ca>


  A bug was recently posted to the R bug database (which probably would
better have been posted as a query here) as to why this happens:

> print(7.921,digits=2)
[1] 8
> print(7.92,digits=2)
[1] 7.9

  Two things I *haven't* done to help make sense of this for myself are
(1) writing out the binary representations to see if something obvious
pops out about why this would be a breakpoint and (2) poking in the
source code (I did a little bit of this but gave up).

  I know that confusion over rounding etc. is very common, and my first
reaction to this sort of question is always that there must be some sort
of confusion (either (1) in a FAQ 7.31-ish sort of way that floating
point values have finite precision in the first place, or (2) a
confusion over the difference between the value and the representation
of the number, or (3) more subtly, about the differences among various
choices of rounding conventions).

  However, in this case I am a bit stumped: I don't see that any of the
standard confusions apply.  I grepped the R manuals for "rounding" and
didn't find anything useful ...  I have a very strong prior that such a
core part of R must be correct, and that therefore I (and the original
bug reporter) must be misunderstanding something.

  Thoughts/references?

  cheers
    Ben Bolker



[Rd] print(...,digits=2) behavior
Ben Bolker