From brod|e@g@@|@m @end|ng |rom y@hoo@com  Sun Dec 13 02:12:31 2020
From: brod|e@g@@|@m @end|ng |rom y@hoo@com (brodie gaslam)
Date: Sun, 13 Dec 2020 01:12:31 +0000 (UTC)
Subject: [Rd] R crashes when using huge data sets with character string
 variables
In-Reply-To: <56b2d17e-35e1-e5af-54e2-7bd8303c4436@gmail.com>
References: <CAMTWbJgN-GW9Zp21awbAmXpor9Rqd7H00gZa10K-dFVjrN-ysQ@mail.gmail.com>
 <56b2d17e-35e1-e5af-54e2-7bd8303c4436@gmail.com>
Message-ID: <1511796618.83974.1607821951891@mail.yahoo.com>

 > On Saturday, December 12, 2020, 6:33:33 PM EST, Ben Bolker <bbolker at gmail.com> wrote:
>
>? On Windows you can use memory.limit.
>
> https://stackoverflow.com/questions/12582793/limiting-memory-usage-in-r-under-linux
>
>??? Not sure how much that helps.
>
>On 12/12/20 6:19 PM, Arne Henningsen wrote:
>> When working with a huge data set with character string variables, I
>> experienced that various commands let R crash. When I run R in a
>> Linux/bash console, R terminates with the message "Killed". When I use
>> RStudio, I get the message "R Session Aborted. R encountered a fatal
>> error. The session was terminated. Start New Session". If an object in
>> the R workspace needs too much memory, I would expect that R would not
>> crash but issue an error message "Error: cannot allocate vector of
>> size ...".? A minimal reproducible example (at least on my computer)
>> is:
>>
>> nObs <- 1e9
>>
>> date <- paste( round( runif( nObs, 1981, 2015 ) ), round( runif( nObs,
>> 1, 12 ) ), round( runif( nObs, 1, 31 ) ), sep = "-" )
>>
>> Is this a bug or a feature of R?

On OS X I see:

??? > nObs <- 1e9
??? >? date <- paste( round( runif( nObs, 1981, 2015 ) ), round( runif( nObs,1, 12 ) ), round( runif( nObs, 1, 31 ) ), sep = "-" )
??? Error: vector memory exhausted (limit reached?)
??? > sessionInfo()
??? R version 4.0.3 (2020-10-10)
??? Platform: x86_64-apple-darwin17.0 (64-bit)
??? Running under: macOS Catalina 10.15.7

Which is what I would expect.? I don't doubt the error you've seen, just
providing a data point for whoever ends up looking into this further.

Best,

Brodie.



[Rd] R crashes when using huge data sets with character string variables
brodie gaslam