From suharto_anggono at yahoo.com  Sat Oct 29 17:32:00 2016
From: suharto_anggono at yahoo.com (Suharto Anggono Suharto Anggono)
Date: Sat, 29 Oct 2016 15:32:00 +0000 (UTC)
Subject: [Rd] lapply on long vector fails
References: <1007285085.282204.1477755120056.ref@mail.yahoo.com>
Message-ID: <1007285085.282204.1477755120056@mail.yahoo.com>

I report here that, in RStudio in Data Scientist Workbench,
lapply(raw(2^31), function(x) NULL)
failed after not so long time.

> res <- lapply(raw(2^31), function(x) NULL)
Error in FUN(X[[i]], ...) : long vectors not supported yet: memory.c:1652
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
 [1] LC_CTYPE=en_US.UTF-8      
 [2] LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                 
 [9] LC_ADDRESS=C              
[10] LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
[1] SparkR_1.6.1

loaded via a namespace (and not attached):
[1] tools_3.3.1


However, the code that implements 'lapply', function 'do_lapply' in apply.c, seems to support long vectors.

The error message points to memory.c:1652. I don't understand the code there.


A different case:
gc()
after
system.time(vector("list", 2^31))
gave an error with message pointing to memory.c at different line. Subsequent
gc()
didn't give error.

> system.time(vector("list", 2^31))
   user  system elapsed 
  3.104  15.608  18.711 
> gc()
Error in gc() : long vectors not supported yet: memory.c:1121
> gc()
         used (Mb) gc trigger    (Mb)   max used    (Mb)
Ncells 445496 23.8     750400    40.1     592000    31.7
Vcells 667725  5.1 2062440028 15735.2 2148153146 16389.2



[Rd] lapply on long vector fails
Suharto Anggono Suharto Anggono