From pmc1 at cornell.edu  Tue Jun  3 22:07:10 2008
From: pmc1 at cornell.edu (Patrick Carr)
Date: Tue, 3 Jun 2008 16:07:10 -0400
Subject: [Rd] significant digits (PR#9682)
In-Reply-To: <48459200.20401@stats.uwo.ca>
References: <20080602234017.CB1FD282C786@mail.pubhealth.ku.dk>
	<48451EBC.1000207@stats.uwo.ca>
	<a696b2f20806030843i5c290c87u1125b280b8a7672e@mail.gmail.com>
	<48459200.20401@stats.uwo.ca>
Message-ID: <a696b2f20806031307m4b38c2a7o85c619b5b34dbc1c@mail.gmail.com>

On 6/3/08, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
>
>  It's easy to make mistakes in this, but a little outside-of-R
> experimentation suggests those are the right answers.  The number 12345 is
> exactly representable, so it is exactly half-way between 12340 and 12350, so
> 12340 is the right answer by the unbiased round-to-even rule.  The number
> 0.12345 is not exactly representable, but (I think) it is represented by
> something slightly closer to 0.1235 than to 0.1234.  So it looks as though
> Windows gets it right.

Well, right within the limitations of binary floating-point
arithmetic. Not right right.

In the grander scheme, this is a nicety which is largely
inconsequential--if I need a real measure precision (precise
precision?) I'll use a +/- notation of a propagated error and/or edit
the typography of the numbers by hand immediately before the final
output. But again, final printed output of the number is basically the
useful use I see for a function that returns significant digits. And
for that purpose I think it should be right right, and actually output
the number of significant digits requested.

> >   > signif(12345.12345,digits=10)
> >   [1] 12345.12
>
>  This is a different problem.  The number is correctly computed as
> 12345.12345 (or at least a representable number quite close to that), and
> then the default display rounds it some more.  Set options(digits=19) to see
> it in its full glory.

Aha, my mistake; I missed that setting.

Pat Carr



[Rd] significant digits (PR#9682)
Patrick Carr