From bri@n @ending from br@verock@com  Tue Sep 18 17:30:55 2018
From: bri@n @ending from br@verock@com (Brian G. Peterson)
Date: Tue, 18 Sep 2018 10:30:55 -0500
Subject: [Rd] memory footprint of readRDS()
In-Reply-To: <CAO1zAVYNvp6ykmEAqdXx9rGZyniu=3ih8MfgWD1VZ5ty+OB6zw@mail.gmail.com>
References: <CAO1zAVYNvp6ykmEAqdXx9rGZyniu=3ih8MfgWD1VZ5ty+OB6zw@mail.gmail.com>
Message-ID: <1537284655.21577.22.camel@braverock.com>

Your RDS file is likely compressed, and could have compression of 10x
or more depending on the composition of the data that is in it and the
compression method used. 'gzip' compression is used by default.

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

On Tue, 2018-09-18 at 17:28 +0200, Joris Meys wrote:
> Dear all,
> 
> I tried to read in a 3.8Gb RDS file on a computer with 16Gb available
> memory. To my astonishment, the memory footprint of R rises quickly
> to over 13Gb and the attempt ends with an error that says "cannot
> allocate vector of size 5.8Gb".
> 
> I would expect that 3 times the memory would be enough to read in
> that file, but apparently I was wrong. I checked the memory.limit()
> and that one gave me a value of more than 13Gb. So I wondered if this
> was to be expected, or if there could be an underlying reason why
> this file doesn't want to open.
> 
> Thank you in advance
> Joris
>



[Rd] memory footprint of readRDS()
Brian G. Peterson