From r at erik.prutser.cx  Fri Jan  5 11:29:48 2007
From: r at erik.prutser.cx (Erik van Zijst)
Date: Fri, 05 Jan 2007 11:29:48 +0100
Subject: [Rd] How to execute R scripts simultaneously from multiple
	threads
In-Reply-To: <200701041237.30054.vdergachev@rcgardis.com>
References: <459B6DBA.3050207@erik.prutser.cx>
	<200701031337.45613.vdergachev@rcgardis.com>
	<459CCEDB.5040201@erik.prutser.cx>
	<200701041237.30054.vdergachev@rcgardis.com>
Message-ID: <459E289C.1080201@erik.prutser.cx>

Vladimir Dergachev wrote:
>>Using pipes or shared memory to pass things around to other processes on
>>the same box is very fast indeed, but if we base our design around
>>something like RServe which uses TCP it could be significantly slower.
>>Our R-based system will be running scripts in response to high-volume
>>real-time stock exchange data, so we expect lots of calls to many tiny
>>functions indeed.
> 
> Very interesting :) 
> 
> If you are running RServe on the other box you will need to send data over 
> ethernet anyway (and will probably use TCP). If it is on the same box and you 
> use "localhost" the packets will go over loopback - which would be 
> significantly faster.

I'd like to do as little IO as possible, so I'm looking at using a 
single, very powerfull box and keep everything local.

> At some point (years ago) there was even an argument on some mailiing list 
> (xfree86-devel ?) about whether Xserver should support shared memory as unix 
> socket was "fast enough" - with the other side arguing that when you pass 
> megabyte images around (as in DVD playback) there is non-negligible overhead.

We're currently doing performance tests with the RServe-approach where 
we measure the actual evaluation time of a function. I'm interested in 
the evaluation-time versus overhead ratio. Loopback TCP might work as 
long as this ratio is sufficiently high.

Erik
-- 
Practice is the best of all instructors.
		-- Publilius



[Rd] How to execute R scripts simultaneously from multiple threads
Erik van Zijst