From simon.urbanek at r-project.org  Wed Oct 24 22:07:16 2012
From: simon.urbanek at r-project.org (Simon Urbanek)
Date: Wed, 24 Oct 2012 16:07:16 -0400
Subject: [Rd] concurrent requests (Rook,
	but I think the question is more general)
In-Reply-To: <508845EB.2090007@rug.nl>
References: <50882FC1.40608@rug.nl>
	<F8B3AAEF-822A-4970-BD6A-AB88FF8E8B9B@r-project.org>
	<50883CD9.1090509@rug.nl>
	<FEE1BFBC-494F-4CA6-BBE8-5F3D4D8559CD@r-project.org>
	<508845EB.2090007@rug.nl>
Message-ID: <F1CBF6BC-B328-4298-9033-1859F04CD67B@r-project.org>


On Oct 24, 2012, at 3:47 PM, Richard D. Morey wrote:

> 
> Richard D. Morey
> Assistant Professor
> Psychometrics and Statistics
> Rijksuniversiteit Groningen / University of Groningen
> http://drsmorey.org/research/rdmorey
> 
> On 24/10/12 9:23 PM, Simon Urbanek wrote:
>> On Oct 24, 2012, at 3:09 PM, Richard D. Morey wrote:
>> 
>>> On 24/10/12 8:53 PM, Simon Urbanek wrote:
>>>> On Oct 24, 2012, at 2:13 PM, Richard D. Morey wrote:
>>>> 
>>>>> This question involves Rook, but I think the answer will be general enough that it pays to post here. At any rate, I don't know enough to know whether this is a Rook only issue or a general R issue.
>>>>> 
>>>>> Here's what I'd like to do (and indeed, have code that should do this):
>>>>> 
>>>>> 1. Start R, Rook
>>>>> 2. Start an analysis via a HTTP request to Rook. This analysis uses .Call() to some compiled C code, if that matters. The C code calls a callback function to update a variable with its progress.
>>>>> 3. While the analysis is happening, use Rook to obtain current status with an HTTP request
>>>>> 
>>>> You can't. R doesn't support threading so it's simply not possible to have an asynchronous eval. The R HTTP server works by simply enqueuing an eval to run while R is idle, it can't do that if R is busy. (Note that the HTTP server was *only* designed for the internal help).
>>>> 
>>>> What you can do is have your C code start another thread that reports the progress when asked e.g. on a socket, but that thread is not allowed to call any R API so you want that progress to be entirely in your C code.
>>> How can I start a new thread? By running R again from the command line, or is there a better way?
>>> 
>> No, you have to use the system thread API like pthreads, NSThread etc. If you have to ask about this, you probably don't want to go there ;) - threads can be quite dangerous if you are not familiar with them.
>> 
>> Another poor man's solution is to simply have your C code write out a file with the progress. Along the same lines you could use a shared object to store the progress (e.g. via bigmemory) ...
> 
> I'd be fine with the poor man's solution (maybe with tempfile()?) if I can get access to the local file via javascript. But I don't think I can, due to the security limitations of the browser. I may have to rethink this significantly.
> 

That should be no problem, you can have another R instance serving the monitoring page (or you can use Rserve's HTTP server and have just one instance with arbitrarily many connections as needed).

Cheers,
Simon


>> 
>> 
>>>> Note that if your C code is robust enough, it can call R_CheckUserInterrupt() to allow external events to happen, but a) your C code must in that case be prepared for early termination (clean up memory etc.) and b) I don't remember if the httpd is allowed to run during interrupt check on all platforms - you may want to check that first.
>>> I do use R_CheckUserInterrupt(), but if I understand what you're saying then given that it doesn't work, httpd must not run during the interrupt check. At least, on OSX, which is what I'm testing on.
>>> 
>> Yes, if your code calls R_CheckUserInterrupt() and httpd doesn't respond at that point then it may not be allowed to run. On OS X you can try your luck with R_ProcessEvents() as well.
>> 
>> Cheers,
>> Simon
>> 
>> 
>> 
>>>> Cheers,
>>>> Simon
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> The problem is that once the analysis starts, Rook does not respond to requests. All of the status requests to Rook pile up, and then are answered when the analysis (step 2) is done. Here is some example code to demonstrate what the issue:
>>>>> 
>>>>> ##########
>>>>> 
>>>>> library(Rook)
>>>>> s <- Rhttpd$new()
>>>>> s$add(
>>>>>  name="pingpong",
>>>>>  app=Rook::URLMap$new(
>>>>>    '/ping' = function(env){
>>>>>      req <- Rook::Request$new(env)
>>>>>      res <- Rook::Response$new()
>>>>>      res$write('This is ping.')
>>>>>      Sys.sleep(20)
>>>>>      res$finish()
>>>>>    },
>>>>>    '/pong' = function(env){
>>>>>      req <- Rook::Request$new(env)
>>>>>      res <- Rook::Response$new()
>>>>>      res$write("This is pong.")
>>>>>      res$finish()
>>>>>    },
>>>>>    '/?' = function(env){
>>>>>      req <- Rook::Request$new(env)
>>>>>      res <- Rook::Response$new()
>>>>>      res$redirect(req$to_url('/pong'))
>>>>>      res$finish()
>>>>>    }
>>>>>  )
>>>>> )
>>>>> 
>>>>> s$start(quiet=TRUE)
>>>>> s$browse('pingpong')
>>>>> 
>>>>> #############################
>>>>> 
>>>>> If you request /ping, R runs Sys.sleep() for 20 seconds. This is where my .Call() statement would be. While the .Call() (Sys.sleep()) function is doing its thing, I need to get Rook to respond on /pong (which would simply respond with the progress), but if you run this code, request /ping, then immediately request /pong, you'll see that the /pong request will not be answered until the Sys.sleep() is done.
>>>>> 
>>>>> Of course, for a progress report to be useful, the requests have to be answered immediately. Is this a Rook issue, or an R issue? Or am I asking something unreasonable?
>>>>> 
>>>>> ______________________________________________
>>>>> R-devel at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>> 
>>>>> 
>>> 
> 
> 



[Rd] concurrent requests (Rook, but I think the question is more general)
Simon Urbanek