From hb at maths.lth.se  Sat Aug 27 12:24:07 2005
From: hb at maths.lth.se (Henrik Bengtsson)
Date: Sat, 27 Aug 2005 12:24:07 +0200
Subject: [Rd] Memory leakage/violation?
In-Reply-To: <Pine.LNX.4.61.0508271046220.27340@gannet.stats>
References: <430F3B7A.5020807@maths.lth.se>
	<Pine.A41.4.61b.0508261146090.87310@homer03.u.washington.edu>
	<431034D8.5030008@statistik.uni-dortmund.de>
	<Pine.LNX.4.61.0508271046220.27340@gannet.stats>
Message-ID: <43103F47.3000902@maths.lth.se>

Prof Brian Ripley wrote:
> On Sat, 27 Aug 2005, Uwe Ligges wrote:
> 
>> Thomas Lumley wrote:
>>
>>> I can't reproduce this on R2.2.0dev on Windows XP (in a few hundred
>>> tries), or running under Valgrind on AMD64 Linux (in four or five 
>>> tries).
>>
>>
>> Cannot reproduce either (using R-2.1.1 and an older version of R-devel,
>> though). Maybe a compiler issue?
>> Henrik, do you use exactly the compiler set up mentioned in the manuals?
>> Which version of gcc? Did your emember to replace the f771.exe?
> 
> 
> It would be better to use gcc-3.4.4 as recommended in the current manual.
> However, I read this that Henrik was using the binary from CRAN which I 
> think Duncan M is now building with that.
> 
> Another possibility is a broken version of a Windows DLL, such as 
> msvcrt.dll.

Thank you for this.

I scanned my computer for possible msvcrt.dll (Microsoft (R) C Runtime 
Library) files and found:

C:\Windows\system32\msvcrt.dll
    v7.0.2600.1106 (xpsp1.020828-1920), 323 072 bytes

C:\WINDOWS\$NtServicePackUninstall$\msvcrt.dll
    v7.0.2600.0 (xpclient.010817-1148), 322 560 bytes

(plus a few versions of the same in "temporary" directories).

As I mentioned in the email I sent recently, I cannot reproduce the 
error today after a fresh reboot (and letting the computer rest for 12 
hours).

Best

Henrik


>>
>> Uwe
>>
>>
>>
>>>      -thomas
>>>
>>>
>>> On Fri, 26 Aug 2005, Henrik Bengtsson wrote:
>>>
>>>
>>>> Hi,
>>>>
>>>> I've spotted a possible memory leakage/violation in the latest R v2.1.1
>>>> patched and R v2.2.0dev on Windows XP Pro SP2 Eng.
>>>>
>>>> I first caught it deep down in a nested svd algorithm when 
>>>> subtracting a
>>>> double 'c' from a integer vector 'a' where both had finite values but
>>>> when assigning 'a <- a - c' would report NaNs whereas (a-c) alone would
>>>> not.  Different runs with the identical data would introduce NaNs at
>>>> random positions, but not all the time.
>>>>
>>>> Troubleshooting is after a couple of hours still at v0.5, but here is a
>>>> script that generates the strange behavior on the above R setups.  I 
>>>> let
>>>> the script speak for itself.  Note that both the script 'strange.R' and
>>>> the data 'strange.RData' is online too, see code below.
>>>>
>>>> People on other systems (but also on Windows), could you please try it
>>>> and see if you can reproduce what I get.
>>>>
>>>> Cheers
>>>>
>>>> Henrik
>>>>
>>>>
>>>> # The following was tested on: Windows XP Pro SP2 Eng with
>>>> #   i) R Version 2.1.1 Patched (2005-08-25)
>>>> #  ii) R 2.2.0 Under development (unstable) (2005-08-25 r35394M)
>>>>
>>>> # Start 'R --vanilla' and source() this script, i.e.
>>>> #  source("http://www.maths.lth.se/help/R/strange.R")
>>>> # If you do not get any errors, retry a few times.
>>>>
>>>> foo <- function(x) {
>>>>  print(list(
>>>>    name=as.character(substitute(x)),
>>>>    storage.mode=storage.mode(x),
>>>>    na=any(is.na(x)),
>>>>    nan=any(is.nan(x)),
>>>>    inf=any(is.infinite(x)),
>>>>    ok=all(is.finite(a))
>>>>  ))
>>>>  print(length(x))
>>>>  print(summary(x))
>>>> }
>>>>
>>>> # Load data from a complicated "non-reproducible" algorithm.
>>>> # The below errors occur also when data is not
>>>> # saved and then reloaded from file.  Data was generated in
>>>> # R v2.1.1 patched (see above).
>>>> if (file.exists("strange.RData")) {
>>>>  load("strange.RData")
>>>> } else {
>>>>  load(url("http://www.maths.lth.se/help/R/strange.RData"))
>>>> }
>>>>
>>>> # First glance at data...
>>>> foo(a)
>>>> foo(c)
>>>>
>>>> ## $name
>>>> ## [1] "a"
>>>> ##
>>>> ## $storage.mode
>>>> ## [1] "integer"
>>>> ##
>>>> ## $na
>>>> ## [1] FALSE
>>>> ##
>>>> ## $nan
>>>> ## [1] FALSE
>>>> ##
>>>> ## $inf
>>>> ## [1] FALSE
>>>> ##
>>>> ## $ok
>>>> ## [1] TRUE
>>>> ##
>>>> ## [1] 15000
>>>> ##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>>>> ##    41.0    51.0    63.0   292.2   111.0 65170.0
>>>> ## $name
>>>> ## [1] "c"
>>>> ##
>>>> ## $storage.mode
>>>> ## [1] "double"
>>>> ##
>>>> ## $na
>>>> ## [1] FALSE
>>>> ##
>>>> ## $nan
>>>> ## [1] FALSE
>>>> ##
>>>> ## $inf
>>>> ## [1] FALSE
>>>> ##
>>>> ## $ok
>>>> ## [1] TRUE
>>>> ##
>>>> ## [1] 1
>>>> ##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>>>> ##   53.43   53.43   53.43   53.43   53.43   53.43
>>>> ##
>>>>
>>>> # But, trying the following, will result in
>>>> # no-reproducible error messages. Sometimes
>>>> # it errors at kk==1, sometimes at kk >> 1.
>>>> # Also, look at the different output for
>>>> # different kk:s.
>>>> for (kk in 1:100) {
>>>>  cat("kk=",kk, "\n")
>>>>  print(summary(a-c))
>>>> }
>>>>
>>>> ## kk= 1
>>>> ##       Min.     1st Qu.      Median        Mean     3rd Qu.        
>>>> Max.
>>>> ## -7.741e+307  -2.431e+00   9.569e+00               5.757e+01
>>>> ## kk= 2
>>>> ##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>> ##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>> ## kk= 3
>>>> ##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>> ##   -12.430    -2.431     9.569              57.570 65120.000
>>>> ## kk= 4
>>>> ##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>> ##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>> ## kk= 5
>>>> ##     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
>>>> ##   -12.430    -2.431     9.569   238.700    57.570 65120.000
>>>> ## kk= 6
>>>> ## Error in quantile.default(object) : missing values and NaN's
>>>> ## not allowed if 'na.rm' is FALSE
>>>>
>>>>
>>>> ## Comments: If you shorten down 'a', the bug occurs less frequently.
>>>>
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>
>>>
>>> Thomas Lumley            Assoc. Professor, Biostatistics
>>> tlumley at u.washington.edu    University of Washington, Seattle
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>



[Rd] Memory leakage/violation?
Henrik Bengtsson