From hb at maths.lth.se  Sat Aug 27 12:38:18 2005
From: hb at maths.lth.se (Henrik Bengtsson)
Date: Sat, 27 Aug 2005 12:38:18 +0200
Subject: [Rd] Memory leakage/violation?
In-Reply-To: <43103F47.3000902@maths.lth.se>
References: <430F3B7A.5020807@maths.lth.se>	<Pine.A41.4.61b.0508261146090.87310@homer03.u.washington.edu>	<431034D8.5030008@statistik.uni-dortmund.de>	<Pine.LNX.4.61.0508271046220.27340@gannet.stats>
	<43103F47.3000902@maths.lth.se>
Message-ID: <4310429A.1020703@maths.lth.se>

And... hopefully my final admission that I did not do my background 
research; the system I'm working with and borrowed from a colleague 
(since my own is on service) is not a Windows XP Pro _SP2_, but 
apparently _SP1_.  It's not worth troubleshooting much until I get an 
update here.  Thank you Brian for making me aware of this by mentioning 
a possible broken DLL.

Thanks you all for taking the time on this one

Henrik


Henrik Bengtsson wrote:
> Prof Brian Ripley wrote:
> 
>>On Sat, 27 Aug 2005, Uwe Ligges wrote:
>>
>>
>>>Thomas Lumley wrote:
>>>
>>>
>>>>I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred
>>>>tries), or running under Valgrind on AMD64 Linux (in four or five 
>>>>tries).
>>>
>>>
>>>Cannot reproduce either (using R-2.1.1 and an older version of R-devel,
>>>though). Maybe a compiler issue?
>>>Henrik, do you use exactly the compiler set up mentioned in the manuals?
>>>Which version of gcc? Did your emember to replace the f771.exe?
>>
>>
>>It would be better to use gcc-3.4.4 as recommended in the current manual.
>>However, I read this that Henrik was using the binary from CRAN which I 
>>think Duncan M is now building with that.
>>
>>Another possibility is a broken version of a Windows DLL, such as 
>>msvcrt.dll.
> 
> 
> Thank you for this.
> 
> I scanned my computer for possible msvcrt.dll (Microsoft (R) C Runtime 
> Library) files and found:
> 
> C:\Windows\system32\msvcrt.dll
>     v7.0.2600.1106 (xpsp1.020828-1920), 323 072 bytes
> 
> C:\WINDOWS\$NtServicePackUninstall$\msvcrt.dll
>     v7.0.2600.0 (xpclient.010817-1148), 322 560 bytes
> 
> (plus a few versions of the same in "temporary" directories).
> 
> As I mentioned in the email I sent recently, I cannot reproduce the 
> error today after a fresh reboot (and letting the computer rest for 12 
> hours).
> 
> Best
> 
> Henrik
> 
> 
> 
>>>Uwe
>>>
>>>
>>>
>>>
>>>>     -thomas
>>>>
>>>>
>>>>On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
>>>>
>>>>
>>>>
>>>>>Hi,
>>>>>
>>>>>I've spotted a possible memory leakage/violation in the latest R v2.1.1
>>>>>patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
>>>>>
>>>>>I first caught it deep down in a nested svd algorithm when 
>>>>>subtracting a
>>>>>double 'c' from a integer vector 'a' where both had finite values but
>>>>>when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
>>>>>not.  Different runs with the identical data would introduce NaNs at
>>>>>random positions, but not all the time.
>>>>>
>>>>>Troubleshooting is after a couple of hours still at v0.5, but here is a
>>>>>script that generates the strange behavior on the above R setups.  I 
>>>>>let
>>>>>the script speak for itself.  Note that both the script 'strange.R' and
>>>>>the data 'strange.RData' is online too, see code below.
>>>>>
>>>>>People on other systems (but also on Windows), could you please try it
>>>>>and see if you can reproduce what I get.
>>>>>
>>>>>Cheers
>>>>>
>>>>>Henrik
>>>>>
>>>>>
>>>>># The following was tested on: Windows XP Pro SP2 Eng with
>>>>>#   i) R Version 2.1.1 Patched (2005-08-25)
>>>>>#  ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
>>>>>
>>>>># Start 'R --vanilla' and source() this script, i.e.
>>>>>#  source("http://www.maths.lth.se/help/R/strange.R")
>>>>># If you do not get any errors, retry a few times.
>>>>>
>>>>>foo <- function(x) {
>>>>> print(list(
>>>>>   name=as.character(substitute(x)),
>>>>>   storage.mode=storage.mode(x),
>>>>>   na=any(is.na(x)),
>>>>>   nan=any(is.nan(x)),
>>>>>   inf=any(is.infinite(x)),
>>>>>   ok=all(is.finite(a))
>>>>> ))
>>>>> print(length(x))
>>>>> print(summary(x))
>>>>>}
>>>>>
>>>>># Load data from a complicated "non-reproducible" algorithm.
>>>>># The below errors occur also when data is not
>>>>># saved and then reloaded from file.  Data was generated in
>>>>># R v2.1.1 patched (see above).
>>>>>if (file.exists("strange.RData")) {
>>>>> load("strange.RData")
>>>>>} else {
>>>>> load(url("http://www.maths.lth.se/help/R/strange.RData"))
>>>>>}
>>>>>
>>>>># First glance at data...
>>>>>foo(a)
>>>>>foo(c)
>>>>>
>>>>>## $name
>>>>>## [1] "a"
>>>>>##
>>>>>## $storage.mode
>>>>>## [1] "integer"
>>>>>##
>>>>>## $na
>>>>>## [1] FALSE
>>>>>##
>>>>>## $nan
>>>>>## [1] FALSE
>>>>>##
>>>>>## $inf
>>>>>## [1] FALSE
>>>>>##
>>>>>## $ok
>>>>>## [1] TRUE
>>>>>##
>>>>>## [1] 15000
>>>>>##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>>>>>##    41.0    51.0    63.0   292.2   111.0 65170.0
>>>>>## $name
>>>>>## [1] "c"
>>>>>##
>>>>>## $storage.mode
>>>>>## [1] "double"
>>>>>##
>>>>>## $na
>>>>>## [1] FALSE
>>>>>##
>>>>>## $nan
>>>>>## [1] FALSE
>>>>>##
>>>>>## $inf
>>>>>## [1] FALSE
>>>>>##
>>>>>## $ok
>>>>>## [1] TRUE
>>>>>##
>>>>>## [1] 1
>>>>>##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>>>>>##   53.43   53.43   53.43   53.43   53.43   53.43
>>>>>##
>>>>>
>>>>># But, trying the following, will result in
>>>>># no-reproducible error messages. Sometimes
>>>>># it errors at kk==1, sometimes at kk >> 1.
>>>>># Also, look at the different output for
>>>>># different kk:s.
>>>>>for (kk in 1:100) {
>>>>> cat("kk=",kk, "\n")
>>>>> print(summary(a-c))
>>>>>}
>>>>>
>>>>>## kk= 1
>>>>>##       Min.     1st Qu.      Median        Mean     3rd Qu.        
>>>>>Max.
>>>>>## -7.741e+307  -2.431e+00   9.569e+00               5.757e+01
>>>>>## kk= 2
>>>>>##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>>>##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>>>## kk= 3
>>>>>##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>>>##   -12.430    -2.431     9.569              57.570 65120.000
>>>>>## kk= 4
>>>>>##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>>>##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>>>## kk= 5
>>>>>##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>>>##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>>>## kk= 6
>>>>>## Error in quantile.default(object) : missing values and NaN's
>>>>>## not allowed if 'na.rm' is FALSE
>>>>>
>>>>>
>>>>>## Comments: If you shorten down 'a', the bug occurs less frequently.
>>>>>
>>>>>______________________________________________
>>>>>R-devel at r-project.org mailing list
>>>>>https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>>
>>>>
>>>>
>>>>Thomas Lumley            Assoc. Professor, Biostatistics
>>>>tlumley at u.washington.edu    University of Washington, Seattle
>>>>
>>>>______________________________________________
>>>>R-devel at r-project.org mailing list
>>>>https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>>>______________________________________________
>>>R-devel at r-project.org mailing list
>>>https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>>
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
>



[Rd] Memory leakage/violation?
Henrik Bengtsson