From brian at braverock.com  Wed Aug 17 14:56:53 2011
From: brian at braverock.com (Brian G. Peterson)
Date: Wed, 17 Aug 2011 07:56:53 -0500
Subject: [Rd] R cmd check and multicore foreach loop
In-Reply-To: <CAC+N9BXxfkOpCCRKFickMfFX+se6eq0P1dCOj2W9iafgh10wRA@mail.gmail.com>
References: <4E4B7FBF.1020109@cbio.uct.ac.za>
	<CAC+N9BX=Gt8VLufTQ67i_e_uTSthdYZL5e4NRVtb=aA2-gWVgA@mail.gmail.com>
	<4E4BA876.3060401@cbio.uct.ac.za>
	<CAC+N9BXxfkOpCCRKFickMfFX+se6eq0P1dCOj2W9iafgh10wRA@mail.gmail.com>
Message-ID: <1313585813.3760.13.camel@brian-rcg>

On Wed, 2011-08-17 at 04:50 -0700, Tim Triche, Jr. wrote:
> I'll see if I can put together a self-contained example.  Primarily,
> the times that I use multicore (and attempted to use doSMP, mostly
> because one of my users refuses to ditch Windows) are when I am
> reading a ton of binary files, none of which depend on the others.
> This is a blindingly obvious use-case for e.g. doMC and doSMP, yet
> what typically happens is that the entire operation wedges.  I'm told
> that doSMP really only works well with Revolution R, but per above, I
> will try to put together a working self-contained example to show
> how. 

Remember that physical I/O can bind up the processes too.  Having a
bunch of processes all trying to read from local disk at the same time
(especially when they are all trying to read the same file(s), a problem
it seems you may not have) is a recipe for I/O locks that can seize up
your processes.

So, if the problem only occurs with physical I/O, the first thing I
would try is to move that storage to a storage device on another machine
that is tuned for that level of disk I/O.

Regards,

   - Brian

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock



[Rd] R cmd check and multicore foreach loop
Brian G. Peterson