From dhinds at sonic.net  Thu Nov 10 07:12:11 2011
From: dhinds at sonic.net (dhinds at sonic.net)
Date: Thu, 10 Nov 2011 06:12:11 +0000
Subject: [Rd] Moderating consequences of garbage collection when in C
References: <4E8BD4EB.60802@fhcrc.org>
Message-ID: <j9fpvr$ad$1@dough.gmane.org>

Martin Morgan <mtmorgan at fhcrc.org> wrote:
> Allocating many small objects triggers numerous garbage collections as R 
> grows its memory, seriously degrading performance. The specific use case 
> is in creating a STRSXP of several 1,000,000's of elements of 60-100 
> characters each; a simplified illustration understating the effects 
> (because there is initially little to garbage collect, in contrast to an 
> R session with several packages loaded) is below.

What a coincidence -- I was just going to post a question about why it
is so slow to create a STRSXP of ~10,000,000 unique elements, each ~10
characters long.  I had noticed that this seemed to show much worse
than linear scaling.  I had not thought of garbage collection as the
culprit -- but indeed it is.  By manipulating the GC trigger, I can
make this operation take as little as 3 seconds (with no GC) or as
long as 76 seconds (with 31 garbage collections).

-- Dave



[Rd] Moderating consequences of garbage collection when in C
dhinds at sonic.net