From pmc1 at cornell.edu  Tue Jun  3 17:43:05 2008
From: pmc1 at cornell.edu (Patrick Carr)
Date: Tue, 3 Jun 2008 11:43:05 -0400
Subject: [Rd] significant digits (PR#9682)
In-Reply-To: <48451EBC.1000207@stats.uwo.ca>
References: <20080602234017.CB1FD282C786@mail.pubhealth.ku.dk>
	<48451EBC.1000207@stats.uwo.ca>
Message-ID: <a696b2f20806030843i5c290c87u1125b280b8a7672e@mail.gmail.com>

On 6/3/08, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
>
>  because signif(0.90, digits=2) == 0.9.  Those two objects are identical.

My text above that is poorly worded. They're identical internally,
yes. But in terms of the number of significant digits, 0.9 and 0.90
are different. And that matters when the number is printed, say as an
annotation on a graph. Passing it through sprintf() or format() later
requires you to specify the number of digits after the decimal, which
is different than the number of significant digits, and requires case
testing for numbers of different orders of magnitude.

The original complainant (and I) expected this behavior from signif(),
not merely rounding. As I said before, I wrote my own workaround so
this is somewhat academic, but I don't think we're alone.

>  As far as I know, rounding is fine in Windows:
>
>  > round(1:10 + 0.5)
>  [1]  2  2  4  4  6  6  8  8 10 10
>

It might not be the rounding, then. (windows xp sp3)

   > signif(12345,digits=4)
   [1] 12340
   > signif(0.12345,digits=4)
   [1] 0.1235

OS X (10.5.2/intel) does not have that problem. But (on both windows and OS X):

   > signif(12345.12345,digits=10)
   [1] 12345.12

Pat Carr



[Rd] significant digits (PR#9682)
Patrick Carr