From tlumley at u.washington.edu  Mon May 15 16:48:29 2006
From: tlumley at u.washington.edu (Thomas Lumley)
Date: Mon, 15 May 2006 07:48:29 -0700 (PDT)
Subject: [Rd] Memory allocation fails in R 2.2.1 and R 2.3.0 on SGI Irix,
 while plenty of memory available (PR#8861)
In-Reply-To: <20060515130956.5B71619B2D@slim.kubism.ku.dk>
References: <20060515130956.5B71619B2D@slim.kubism.ku.dk>
Message-ID: <Pine.LNX.4.64.0605150740250.29777@homer23.u.washington.edu>

On Mon, 15 May 2006, m.vroonhoven at erasmusmc.nl wrote:

> Dear R developers,
>
> We have a big SGI Origin computation server with 32 cpu's and 64 Gb of
> RAM. In R 2.0.0 we could run large jobs, allocating 8 Gb of RAM was not
> a problem, for example by running:
>  > v1 <- seq(1,2^29)
>  > v2 <- seq(1,2^29)
>  > v3 <- seq(1,2^29)
>  > v4 <- seq(1,2^29)
> This yields an R process, consuming about 8 Gb of RAM:
>       PID       PGRP USERNAME PRI  SIZE   RES STATE    TIME WCPU% CPU% COMMAND
>    177484     177484 mirjam    20 8225M 8217M sleep    1:18  29.3 0.00 R
>
> After upgrading from R 2.0.0 to R 2.2.1, we cannot allocate more than
> about 1300 M of memory, as shown below:
>  > v1 <- seq(1,2^29)
>  Error: cannot allocate vector of size 2097152 Kb
>  > v1 <- seq(1,2^28)
>  > v2 <- seq(1,2^27)
>  Error: cannot allocate vector of size 524288 Kb
>  > v2 <- seq(1,2^25)
>  > v3 <- seq(1,2^24)
>  > v4 <- seq(1,2^23)
>  > v5 <- seq(1,2^22)
>  Error: cannot allocate vector of size 16384 Kb
>  > v5 <- seq(1,2^21)
>  > v6 <- seq(1,2^20)
>  > v7 <- seq(1,2^19)
>  > v8 <- seq(1,2^18)
>  > q()
>  Save workspace image? [y/n/c]: n
> Upgrading to R 2.3.0 yields the same results.
> This yields an R executable taking 1284M of RAM, refusing to allocate
> more RAM, with about 30Gb free on the machine.


You can tell if you have a 64bit build of R by looking at
.Machine$sizeof.pointer in R, which should be 8.

If not, then you need to set whatever C and Fortran compilation flags give 
a 64bit system. It doesn't look to me as if R's configure script has any 
special handling for C compiler flags on SGI.

If you have a 64bit build then something strange is happening. The message 
you quote happens only when malloc() returns NULL, so it is hard to see 
how R could be causing it, though.

 	-thomas

Thomas Lumley			Assoc. Professor, Biostatistics
tlumley at u.washington.edu	University of Washington, Seattle



[Rd] Memory allocation fails in R 2.2.1 and R 2.3.0 on SGI Irix, while plenty of memory available (PR#8861)
Thomas Lumley