From btyner at gmail.com  Wed Mar 11 02:09:22 2009
From: btyner at gmail.com (Benjamin Tyner)
Date: Tue, 10 Mar 2009 21:09:22 -0400
Subject: [Rd] bug (PR#13570)
In-Reply-To: <alpine.LFD.2.00.0903090910590.25123@gannet.stats.ox.ac.uk>
References: <20090304211010.A588D2832180@mail.pubhealth.ku.dk>
	<49AF88B7.1000002@biostat.ku.dk>
	<20090305174301.20099b11@berwin-nus1>
	<49AFB412.2060807@statistik.tu-dortmund.de>
	<alpine.LFD.2.00.0903051121300.2646@gannet.stats.ox.ac.uk>
	<49AFC711.1030902@biostat.ku.dk>
	<alpine.LFD.2.00.0903051237560.3864@gannet.stats.ox.ac.uk>
	<E0F590EA-8F52-44A6-AD5D-E050BD6740D7@stat.purdue.edu>
	<B37C0A15B8FB3C468B5BC7EBC7DA14CC61CC034A04@LP-EXMBVS10.CO.IHC.COM>
	<7A1AA352-D2E8-48A2-BDE3-FF48B71E28D9@stat.purdue.edu>
	<49B07B45.3040308@gmail.com>
	<alpine.LFD.2.00.0903060718570.24770@auk.stats.ox.ac.uk>
	<alpine.LFD.2.00.0903090910590.25123@gannet.stats.ox.ac.uk>
Message-ID: <49B70F42.3040206@gmail.com>

Many thanks Brian for tracking this down. Was it fixed by

      c next line is not in current dloess
	        	  goto 7

in ehg136? If this needs to be in the netlib version as well, we should 
inform Eric Grosse.

While we're at it, there are a few more inconsistencies (not nearly as 
serious as PR#13570 so I hesitate to call them bugs) regarding the 
definition of leaf cell membership (certain .lt. should be .le. ) in 
ehg128, ehg137, and ehg138 (not currently used); it seems I neglected to 
mention these to Eric. If you are interested in these I can submit a 
patch and will notify Eric as well.

Finally, perhaps now is as good a time as any to point out that in the 
documentation, the bit about cross-terms in

    \item{drop.square}{for fits with more than one predictor and
        \code{degree=2}, should the quadratic term (and cross-terms) be
        dropped for particular predictors?

is incorrect -- cross terms are not dropped in this implementation of 
loess.

Thanks again,
Ben

Prof Brian Ripley wrote:
> I've found the discrepancy, so the patched code from current dloess is 
> now available in R-patched and R-devel.
>
> On Fri, 6 Mar 2009, Prof Brian Ripley wrote:
>
>> On Thu, 5 Mar 2009, Benjamin Tyner wrote:
>>
>>> Hi
>>>
>>> Nice to hear from you Ryan. I also do not have the capability to 
>>> debug on windows; however, there is a chance that the behavior you 
>>> are seeing is caused by the following bug noted in my thesis 
>>> (available on ProQuest; email me if you don't have access):
>>>
>>> "When lambda = 0 there are no local slopes to aid the blending 
>>> algorithm, yet the
>>> interpolator would still assume they were available, and thus use 
>>> arbitrary values
>>> from memory. This had implications for both fit and tr[L] 
>>> computation. In the
>>> updated code these are set equal to zero which seems the best 
>>> automatic rule when
>>> lambda = 0." [lambda refers to degree]
>>>
>>> I submitted a bug fix to Eric Grosse, the maintainer of the netlib 
>>> routines; the fixed lines of fortran are identified in the comments 
>>> at (just search for my email address):
>>>
>>> http://www.netlib.org/a/loess
>>>
>>> These fixes would be relatively simple to incorporate into R's 
>>> version of loessf.f
>>
>> The fixes from dloess even more simply, since R's code is based on 
>> dloess. Thank you for the suggestion.
>>
>> Given how tricky this is to reproduce, I went back to my example 
>> under valgrind.  If I use the latest dloess code, it crashes, but by 
>> selectively importing some of the differences I can get it to work.
>>
>> So it looks as if we are on the road to a solution, but something in 
>> the current version (not necessarily in these changes) is 
>> incompatible with the current R code and I need to dig further (not 
>> for a few days).
>>
>>> Alternatively, a quick check would be for someone to compile the 
>>> source package at 
>>> https://centauri.stat.purdue.edu:98/loess/loess_0.4-1.tar.gz and 
>>> test it on windows. Though this package incorporates this and a few 
>>> other fixes, please be aware that it the routines are converted to C 
>>> and thus there is a slight performance hit compared to the fortran.
>>>
>>> Hope this helps,
>>> Ben
>>
>> [...]
>>
>> -- 
>> Brian D. Ripley,                  ripley at stats.ox.ac.uk
>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford,             Tel:  +44 1865 272861 (self)
>> 1 South Parks Road,                     +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>>
>



[Rd] bug (PR#13570)
Benjamin Tyner