From P.Dalgaard at biostat.ku.dk  Fri Jan  5 18:10:34 2007
From: P.Dalgaard at biostat.ku.dk (Peter Dalgaard)
Date: Fri, 05 Jan 2007 18:10:34 +0100
Subject: [Rd] help for memory problem with 64-bit machines
In-Reply-To: <459E81DD.6060707@cimr.cam.ac.uk>
References: <E1H2rmC-0005nz-00@DMjcadima> <459E81DD.6060707@cimr.cam.ac.uk>
Message-ID: <459E868A.2080302@biostat.ku.dk>

Hin-Tak Leung wrote:
> I got the same error with 64-bit R 2.4.1 on FC6 x86_64, and 32-bit
> R 2.4.1 on the same machine is okay. There is definitely something wrong
> with your code.
>
> I would suggest fixing all the compier warnings - there are piles of
> them about uninitialized variables, and about doing comparison
> between signed and unsigned expressions, etc first. Put -Wall in
> CFLAGS CXXFLAGS and FFLAGS and you'll see.
>
> good luck.
>
> Hin-Tak Leung
>
>   
Good advice. Also, the most common culprit for 64/32 problem is pointers
stored as integers so watch out for any of those. And notice that you
can set a breakpoint at randsk1_ and start poking around to see what is
inside various variables and singlestep to the point of the crash (it's
a bit painful and confusing in Fortran code, though.)

    -pd
> Jorge Cadima wrote:
>   
>> Hello,
>>
>> I would appreciate *any* ideas on this problem. I'm the maintainer of a 
>> package ("subselect"), which on CRAN's Daily Package Checks is OK on all 
>> flavours of R, except  r-devel Linux x86_64, where there is a "memory not 
>> mapped" segfault with the very first example that is tried out (output below). 
>> Additionally, a user with an AMD64 machine has just reported a similar 
>> problem, and kindly ran a couple of tests identifying the culprit (output 
>> below):a Fortran routine. Unfortunately, I don't have 64-bit machines 
>> available and the segfault has not occured with other machines, so I am having 
>> trouble trying to spot the problem.
>>
>> I suspect the problem lies in  one of two things:
>>
>> 1) The Fortran subroutine where the crash occurs has variable-length arrays. 
>> Could there be a problem with this (the user who spotted the problem is 
>> running the gfortran gcc compiler - I presume r-devel Linux x86_64 does too)?
>> 3) the guilty routine ("randsk1") calls a second routine ("randint") which, in 
>> turn, calls R's Random Number Generator. Can the problem arise in the 
>> Fortran/R interface?
>>
>> As I said, any idea of what the problem may be, or where I can look for help, 
>> would be much appreciated.
>>
>> In despair, but with best regards,
>> Jorge Cadima
>>
>> -------------------------------------------------------------------------------
>> ---
>> --- Cran Daily Package check output -------------------------------------------
>> ---
>> -------------------------------------------------------------------------------
>> ---
>>
>>     * using R version 2.5.0 Under development (unstable) (2007-01-03 r40349)
>>     * checking for file 'subselect/DESCRIPTION' ... OK
>>     * this is package 'subselect' version '0.9-999'
>>     * checking package dependencies ... OK
>>  (...)
>>     * creating subselect-Ex.R ... OK
>>     * checking examples ... ERROR
>>       Running examples in 'subselect-Ex.R' failed.
>>       The error most likely occurred in:
>>
>>       > ### * anneal
>>       >
>>       > flush(stderr()); flush(stdout())
>>       >
>>       > ### Name: anneal
>>       > ### Title: Simulated Annealing Search for an optimal k-variable subset
>>       > ### Aliases: anneal
>>       > ### Keywords: manip
>>       >
>>       > ### ** Examples
>>       >
>>       >
>>       > ## --------------------------------------------------------------------
>>       >
>>       > ##
>>       > ## (1) For illustration of use, a small data set with very few 
>> iterations
>>       > ## of the algorithm, using the RM criterion.
>>       > ##
>>       >
>>       > data(swiss)
>>       > anneal(cor(swiss),2,3,nsol=4,niter=10,criterion="RM")
>>
>>       *** caught segfault ***
>>       address 0x47cd37c, cause 'memory not mapped'
>>
>>       Traceback:
>>       1: .Fortran("anneal", as.integer(criterio), as.integer(p), 
>> as.double(as.vector(mat)), as.integer(kmin), as.integer(kmax), 
>> as.double(valores), as.integer(vars), as.double(bestval), as.integer(bestvar), 
>> as.integer(nexclude), as.integer(exc), as.integer(ninclude), as.integer(inc), 
>> as.integer(nsol), as.integer(niter), as.logical(improvement), 
>> as.double(cooling), as.double(temp), as.integer(coolfreq), 
>> as.integer(length(pcindices)), as.integer(pcindices), as.logical(esp), 
>> as.logical(silog), as.integer(as.vector(initialsol)), as.double(valp), 
>> as.double(as.vector(vecp)), as.double(as.vector(H)), as.integer(r), PACKAGE = 
>> "subselect")
>>       2: anneal(cor(swiss), 2, 3, nsol = 4, niter = 10, criterion = "RM")
>>       aborting ...
>>
>> -------------------------------------------------------------------------------
>> -----
>> ------- user's tests output ---------------------------------------------------
>> --
>> -------------------------------------------------------------------------------
>> --
>>
>> The install
>> ===========
>>
>> work:/home/jimmy/R # R CMD INSTALL subselect_0.9-999.tar.gz
>> * Installing *source* package 'subselect' ...
>> ** libs
>> gfortran   -fpic   -c anneal.f -o anneal.o
>> gfortran   -fpic   -c auxiliar.f -o auxiliar.o
>> g++ -I/usr/lib64/R/include -I/usr/lib64/R/include  -I/usr/local/include   
>> -fpic  -g -O2 -c CCRcrt.cpp -o CCRcrt.o
>>
>> (...)
>>
>> g++ -I/usr/lib64/R/include -I/usr/lib64/R/include  -I/usr/local/include   
>> -fpic  -g -O2 -c Wrkf1.cpp -o Wrkf1.o
>> g++ -shared -L/usr/local/lib64 -o subselect.so anneal.o auxiliar.o CCRcrt.o 
>> GCDcrt.o genetic.o Gstatma.o improve.o MStcrt.o NewtonRp.o Qforms.o RMcrt.o 
>> Rnk3CCRcrt.o RVcrt.o SpecialArrays.o SR_datatrnsf.o SR_intf.o SRI_sscma.o 
>> Sscma.o Subsets.o Varsmbo.o VSQforms.o wrap.o Wrkf1.o -L/usr/lib64/R/lib 
>> -lRlapack  -lgfortran -lm -lgcc_s -lgfortran -lm -lgcc_s -L/usr/lib64/R/lib -lR
>> ** R
>> ** inst
>> ** preparing package for lazy loading
>> ** help
>>  >>> Building/Updating help pages for package 'subselect'
>>      Formats: text html latex example
>>   anneal                            text    html    latex   example
>>
>> (...)
>>
>>   zeta2                             text    html    latex   example
>> ** building package indices ...
>> * DONE (subselect)
>> work:/home/jimmy/R #                   
>>
>>
>>
>> The library is put into:
>> /usr/lib64/R/library/subselect/libs/subselect.so
>>
>> and nm -s give me (in part):
>>
>> work:/home/jimmy/R # nm -s /usr/lib64/R/library/subselect/libs/subselect.so
>> 0000000000012a4c T anneal_
>> ....
>> 0000000000014580 T randint_
>> 000000000001446a T randsk1_
>>                  U Rf_error
>> 000000000003470f T rndend_
>> 0000000000034704 T rndstart_
>>                  U Rprintf
>> ....
>> etc
>>
>> So randsk1 is there.
>>
>>
>> Running the subselect example from the manual:
>>
>> jimmy at work:~> R
>>
>> R : Copyright 2006, The R Foundation for Statistical Computing
>> Version 2.3.1 (2006-06-01)
>> ISBN 3-900051-07-0
>>
>> R is free software and comes with ABSOLUTELY NO WARRANTY.
>> You are welcome to redistribute it under certain conditions.
>> Type 'license()' or 'licence()' for distribution details.
>>
>>   Natural language support but running in an English locale
>>
>> R is a collaborative project with many contributors.
>> Type 'contributors()' for more information and
>> 'citation()' on how to cite R or R packages in publications.
>>
>> Type 'demo()' for some demos, 'help()' for on-line help, or
>> 'help.start()' for an HTML browser interface to help.
>> Type 'q()' to quit R.
>>
>>     
>>> library(subselect)
>>> data(swiss)
>>> anneal(cor(swiss),2,3,nsol=4,niter=10,criterion="RM")
>>>       
>>  *** caught segfault ***
>> address 0xffffffffb03f38dc, cause 'memory not mapped'
>>
>> Traceback:
>>  1: .Fortran("anneal", as.integer(criterio), as.integer(p), 
>> as.double(as.vector(mat)),     as.integer(kmin), as.integer(kmax), 
>> as.double(valores), as.integer(vars),     as.double(bestval), 
>> as.integer(bestvar), as.integer(nexclude),     as.integer(exc), 
>> as.integer(ninclude), as.integer(inc), as.integer(nsol),     
>> as.integer(niter), as.logical(improvement), as.double(cooling),     
>> as.double(temp), as.integer(coolfreq), as.integer(length(pcindices)),     
>> as.integer(pcindices), as.logical(esp), as.logical(silog),     
>> as.integer(as.vector(initialsol)), as.double(valp), 
>> as.double(as.vector(vecp)),     as.double(as.vector(H)), as.integer(r), 
>> PACKAGE = "subselect")
>>  2: anneal(cor(swiss), 2, 3, nsol = 4, niter = 10, criterion = "RM")
>>
>> Possible actions:
>> 1: abort (with core dump)
>> 2: normal R exit
>> 3: exit R without saving workspace
>> 4: exit R saving workspace
>> Selection:                  
>>
>>
>> Running R in the debugger with this input file:
>>
>> library(subselect)
>> data(swiss)
>> anneal(cor(swiss),2,3,nsol=4,niter=10,criterion="RM")
>>
>> (gdb) run
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00002b86c2ed9526 in randsk1_ () from /usr/lib64/R/library/subselect/libs/subs
>> elect.so
>> (gdb) 
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>     
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>   


-- 
   O__  ---- Peter Dalgaard             ?ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45) 35327907



[Rd] help for memory problem with 64-bit machines
Peter Dalgaard