From |uke-t|erney @end|ng |rom u|ow@@edu  Mon May 20 19:29:50 2019
From: |uke-t|erney @end|ng |rom u|ow@@edu (Tierney, Luke)
Date: Mon, 20 May 2019 17:29:50 +0000
Subject: [Rd] [External] most robust way to call R API functions from a
 secondary thread
In-Reply-To: <E1hSIAJ-0005FM-8P@rmmprod07.runbox>
References: <E1hSIAJ-0005FM-8P@rmmprod07.runbox>
Message-ID: <alpine.DEB.2.21.1905201213210.2805@luke-Latitude-7480>

Your analysis looks pretty complete to me and your solutions seems
plausible.  That said, I don't know that I would have the level of
confidence yet that we haven't missed an important point that I would
want before going down this route.

Losing stack checking is risky; it might be eventually possible to
provide some support for this to be handled via a thread-local
variable. Ensuring that R_ToplevelExec can't jump before entering the
body function would be a good idea; if you want to propose a patch we
can have a look.

Best,

luke

On Sun, 19 May 2019, Andreas Kersting wrote:

> Hi,
>
> As the subject suggests, I am looking for the most robust way to call an (arbitrary) function from the R API from another but the main POSIX thread in a package's code.
>
> I know that, "[c]alling any of the R API from threaded code is ?for experts only? and strongly discouraged. Many functions in the R API modify internal R data structures and might corrupt these data structures if called simultaneously from multiple threads. Most R API functions can signal errors, which must only happen on the R main thread." (https://cran.r-project.org/doc/manuals/r-release/R-exts.html#OpenMP-support)
>
> Let me start with my understanding of the related issues and possible solutions:
>
> 1) R API functions are generally not thread-safe and hence one must ensure, e.g. by using mutexes, that no two threads use the R API simultaneously
>
> 2) R uses longjmps on error and interrupts as well as for condition handling and it is undefined behaviour to do a longjmp from one thread to another; interrupts can be suspended before creating the threads by setting R_interrupts_suspended = TRUE; by wrapping the calls to functions from the R API with R_ToplevelExec(), longjmps across thread boundaries can be avoided; the only reason for R_ToplevelExec() itself to fail with an R-style error (longjmp) is a pointer protection stack overflow
>
> 3) R_CheckStack() might be executed (indirectly), which will (probably) signal a stack overflow because it only works correctly when called form the main thread (see https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Threading-issues); in particular, any function that does allocations, e.g. via allocVector3() might end up calling it via GC -> finalizer -> ... -> eval; the only way around this problem which I could find is to adjust R_CStackLimit, which is outside of the official API; it can be set to -1 to disable the check or be changed to a value appropriate for the current thread
>
> 4) R sets signal handlers for several signals and some of them make use of the R API; hence, issues 1) - 3) apply; signal masks can be used to block delivery of signals to secondary threads in general and to the main thread while other threads are using the R API
>
>
> I basically have the following questions:
>
> a) Is my understanding of the issues accurate?
> b) Are there more things to consider when calling the R API from secondary threads?
> c) Are the solutions proposed appropriate? Are there scenarios in which they will fail to solve the issue? Or might they even cause new problems?
> d) Are there alternative/better solutions?
>
> Any feedback on this is highly appreciated.
>
> Below you can find a template which, combines the proposed solutions (and skips all non-illustrative checks of return values). Additionally, R_CheckUserInterrupt() is used in combination with R_UnwindProtect() to regularly check for interrupts from the main thread, while still being able to cleanly cancel the threads before fun_running_in_main_thread() is left via a longjmp. This is e.g. required if the secondary threads use memory which was allocated in fun_running_in_main_thread() using e.g. R_alloc().
>
> Best regards,
> Andreas Kersting
>
>
>
> #include <Rinternals.h>
> #include <pthread.h>
> #include <signal.h>
> #include <stdint.h>
>
> extern uintptr_t R_CStackLimit;
> extern int R_PPStackTop;
> extern int R_PPStackSize;
>
> #include <R_ext/libextern.h>
> LibExtern Rboolean R_interrupts_suspended;
> LibExtern int R_interrupts_pending;
> extern void Rf_onintr(void);
>
> // mutex for exclusive access to the R API:
> static pthread_mutex_t r_api_mutex = PTHREAD_MUTEX_INITIALIZER;
>
> // a wrapper arround R_CheckUserInterrupt() which can be passed to R_UnwindProtect():
> SEXP check_interrupt(void *data) {
>  R_CheckUserInterrupt();
>  return R_NilValue;
> }
>
> // a wrapper arround Rf_onintr() which can be passed to R_UnwindProtect():
> SEXP my_onintr(void *data) {
>  Rf_onintr();
>  return R_NilValue;
> }
>
> // function called by R_UnwindProtect() to cleanup on interrupt
> void cleanfun(void *data, Rboolean jump) {
>  if (jump) {
>    // terminate threads cleanly ...
>  }
> }
>
> void fun_calling_R_API(void *data) {
>  // call some R API function, e.g. mkCharCE() ...
> }
>
> void *threaded_fun(void *td) {
>
>  // ...
>
>  pthread_mutex_lock(&r_api_mutex);
>
>  // avoid false stack overflow error:
>  intptr_t R_CStackLimit_old = R_CStackLimit;
>  R_CStackLimit = -1;
>
>
>  // R_ToplevelExec() below will call PROTECT 4x:
>  if (R_PPStackTop > R_PPStackSize - 4) {
>    // ppstack would overflow in R_ToplevelExec() -> handle this ...
>  }
>
>  // avoid longjmp to different thread:
>  Rboolean ok = R_ToplevelExec(fun_calling_R_API, (void *) &some_data);
>
>  // re-enable stack size checking:
>  R_CStackLimit = R_CStackLimit_old;
>  pthread_mutex_unlock(&r_api_mutex);
>
>  if (!ok) {
>    // handle error ...
>  }
>
>  // ...
> }
>
> SEXP fun_running_in_main_thread() {
>
>  // ...
>
>  /* create continuation token for R_UnwindProtect():
>   *
>   * do this explicitly here before the threads are created because this might
>   * fail in allocation or with pointer protection stack overflow
>   */
>  SEXP cont = PROTECT(R_MakeUnwindCont());
>
>  /* block all signals:
>   *
>   * do this before the threads are created such that they inherit the mask
>   */
>  sigset_t block_set, prev_mask;
>  sigfillset(&block_set);
>  pthread_sigmask(SIG_SETMASK, &block_set, &prev_mask);
>
>  // suspend interrupts:
>  Rboolean __oldsusp__ = R_interrupts_suspended;
>  R_interrupts_suspended = TRUE;
>
>  // create threads running threaded_fun() ...
>
>  for(;;) {
>    // timed blocking check if threads are done ...
>
>    // unblock signals, check for interrupts and run cleanfun if there is one:
>    pthread_mutex_lock(&r_api_mutex);
>    pthread_sigmask(SIG_SETMASK, &prev_mask, NULL);
>
>    R_interrupts_suspended = __oldsusp__;
>    if (R_interrupts_pending && ! R_interrupts_suspended) {
>      R_UnwindProtect(my_onintr, NULL, cleanfun, (void *) clean_data, cont);
>    }
>
>    R_UnwindProtect(check_interrupt, NULL, cleanfun, (void *) clean_data, cont);
>
>    R_interrupts_suspended = TRUE;
>
>    pthread_sigmask(SIG_SETMASK, &block_set, NULL);
>    pthread_mutex_unlock(&r_api_mutex);
>  }
>
>  // now all threads are dead
>
>  UNPROTECT(1);  // continuation token
>
>  // unblock signals:
>  pthread_sigmask(SIG_SETMASK, &prev_mask, NULL);
>
>  // reset interrupt-suspension:
>  R_interrupts_suspended = __oldsusp__;
>  if (R_interrupts_pending && ! R_interrupts_suspended) {
>    Rf_onintr();
>  }
>
>  // ...
> }
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu


[Rd] [External] most robust way to call R API functions from a secondary thread
Tierney, Luke