From simon.urbanek at r-project.org  Thu Aug 30 21:54:25 2012
From: simon.urbanek at r-project.org (Simon Urbanek)
Date: Thu, 30 Aug 2012 15:54:25 -0400
Subject: [Rd] stack overflow? was Re: [R] segfault in gplots::heatmap.2
In-Reply-To: <503FC17F.9090902@fhcrc.org>
References: <87harkhm5p.fsf@med.uni-goettingen.de>
	<CAAmySGPC8bYUdv40=AKThKKURnMk_yW+3dbV05mn_GzpTmhjiQ@mail.gmail.com>
	<503FC17F.9090902@fhcrc.org>
Message-ID: <E2576FF5-1B67-4E88-9448-27B835DC0359@r-project.org>


On Aug 30, 2012, at 3:39 PM, Martin Morgan <mtmorgan at fhcrc.org> wrote:

> I think this belongs on R-devel and I'm forwarding there.
> 
> Here's a more refined example
> 
>  library("XLConnect")
>  load(file="ddr.Rda")
> 
>  oV <- function(x) {
>      if (is.leaf(x)) {
>          return(x)
>      }
>      for (j in seq_along(x)) {
>          b <- oV(x[[j]])
>      }
>      x
>  }
> 
>  res <- oV(ddr)                      # segfault for me under 2.15.1
>  str(ddr)                            # segfault more reliably
> 
> 'ddr' is a dendrogram produced from Andreas' original object at the point where heatmap.2 calls 'reorder'. 'oV' is a reduced version of the internal function in reorder.dendrogram.
> 
> Simple changes to oV above have interesting effects, e.g., removing {} in the 'if' and 'for' mean that the segfault labelled at 2.15.1 does not occur.
> 
> Under R-devel, valgrind complains about Java (I think this is a red herring) and then
> 
> 
> 14604== Can't extend stack to 0x7fef03370 during signal delivery for thread 1:
> ==14604==   too small or bad protection modes
> 
> which likely indicates stack overflow.
> 
> gdb shows that we're several 1000's of calls down when the segfault occurs.
> 
> I think R overflows its stack, and that rJava just makes that more likely to occur. I don't really know how to investigate a stack overflow further.
> 

In general, the moment you load Java, stack checking is disabled, because that turns the R process into a multi-threaded application so the stack is no longer guaranteed at a fixed location. Guarding the stack is messy enough without threads, but it becomes close to impossible with threads.

If you have stack checking enabled, it gets caught:


>  str(ddr)                            # segfault more reliably
[...]
                                                                                           |--leaf "124" 
     |  |  |                                |                 |                                                                                                                                                                                                                                                                                                                                                                                                                                                        `--Error: C stack usage is too close to the limit

Cheers,
Simon



> Martin
> 
> R version 2.15.1 Patched (2012-08-26 r60438)
> Platform: x86_64-unknown-linux-gnu (64-bit)
> 
> locale:
> [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=C                 LC_NAME=C
> [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> 
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
> 
> 
> > sessionInfo()
> R Under development (unstable) (2012-08-20 r60336)
> Platform: x86_64-unknown-linux-gnu (64-bit)
> 
> locale:
> [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=C                 LC_NAME=C
> [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> 
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
> 
> 
> On 08/30/2012 12:04 PM, R. Michael Weylandt wrote:
>> On Thu, Aug 30, 2012 at 2:30 AM, Andreas Leha
>> <andreas.leha at med.uni-goettingen.de> wrote:
>>> Hi all,
>>> 
>>> I experience a segfault when calling gplots::heatmap.2(), but only when
>>> certain other packages are loaded.
>>> 
>>> I am not sure for the correct place to send this bug report.  Should I send
>>> it to the package maintainers directly?  If R-help is the wrong place,
>>> please feel free to direct me to the correct one.
>>> 
>>> I am on debian (testing) linux 64 with the binary R distribution
>>> from the repositories (version 2.15.1).
>>> 
>>> Below follows a simple reproducible example causing the segfault on my
>>> machine.
>>> The offending dataset is quite big, so instead of posting it here I put
>>> it here: https://gist.github.com/3523761.  Please put it into offending.txt to
>>> make the code below working.
>>> 
>>> This is the example.  Note, that without loading 'XLConnect' this works
>>> nicely.
>>> #+begin_src R
>>>   library("gplots")
>>>   library("XLConnect") # any of XLConnect, venneuler, xlsx case a segfault
>>> 
>>>   offending <- dget("offending.txt")
>>>   heatmap.2(x=offending)
>>> #+end_src
>>> 
>>> Interestingly, I get a segfault when loading any of c("XLConnect",
>>> "venneuler", "xlsx"), which all depend on rJava.  But loading rJava on
>>> its own did not produce a segfault.
>> 
>> Hi Andreas,
>> 
>> Thanks for the nicely reproducible example. Unfortunately, I can't
>> reproduce the segfault on my Mac OS X 10.6 running R 2.15.0. I could
>> only test with rJava + venneuler because xlsx and XLConnect fall
>> victim to Mac's Java "infelicities." It's something of a formality,
>> but are you sure you are up-to-date with your packages as well as with
>> R itself. Something like
>> 
>> update.packages(checkBuilt = TRUE)
>> 
>> will ensure you've got the most current release of all your packages.
>> (Note that I'm not sure that's the right way to do it on Debian)
>> 
>> Do you happen to know if this happens with other versions of R? e.g.,
>> 2.15.0 or the not-yet-released R-devel or R-patched (maintenance
>> branch of 2.15.z which will become 2.15.2 eventually)
>> 
>> Consequently, I'd suspect that there's something going on in the
>> intersection of Java + R + Deb Testing, so three places you might seek
>> more advanced help, as this is likely deeper than the day-to-day of
>> this list. i) The rJava mailing list
>> (http://mailman.rz.uni-augsburg.de/mailman/listinfo/stats-rosuda-devel);
>> ii) the R-SIG-Debian list; iii) the R Devel list.
>> 
>> I'm not sure which one makes the most sense to try, but I'd think the
>> third should be of last resort, because it seems least likely to be a
>> problem in base-R if it requires rJava being around to reproduce. The
>> R-SIG-Debian list most likely has someone who can reproduce your exact
>> config.
>> 
>> Cheers,
>> Michael
>> 
>>> 
>>> Regards,
>>> Andreas
>>> 
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>> 
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>> 
> 
> 
> -- 
> Computational Biology / Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N.
> PO Box 19024 Seattle, WA 98109
> 
> Location: Arnold Building M1 B861
> Phone: (206) 667-2793
> <ddr.Rda.tar>______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



[Rd] stack overflow? was Re: [R] segfault in gplots::heatmap.2
Simon Urbanek