From olegfink at gmail.com  Wed Jan 27 16:41:11 2016
From: olegfink at gmail.com (Oleg Finkelshteyn)
Date: Wed, 27 Jan 2016 15:41:11 +0000
Subject: [Rd] symbol name conflict in grDevices on linux
Message-ID: <CAJe5o3e2eebMKUKYC0Sg5DPoZR8BzUJaL2LJtiBkPkeSvFLyBw@mail.gmail.com>

It appears that grDevices (and perhaps other R libraries) is built
without passing C_VISIBILITY to PKG_CFLAGS (-fvisibility=hidden),
while other libraries (e.g. utils) do use the flag.

This results in potential crashes on linux in situations like using
embedded R in another application that happens to export a symbol
named similarly to one defined in R's library, since the runtime
linker substitutes the address of the wrong function into the
library's CallEntries.

Using -fvisibility=hidden or -Wl,-Bsymbolic[-functions] fixes this
since then the addresses of functions in CallEntries are fixed at
library build time, which is probably the intended behaviour.

Here is a small example, using grDevices::colors. I compile it with
  gcc -O2 r.c -I/usr/include/R -L/usr/lib/R/lib -lR -DBUG -rdynamic
(remove -DBUG to disable triggering the bug) and run it with
  R_HOME=/usr/lib/R ./a.out

without -DBUG colors() from grDevices is called as expected, however,
with -DBUG adding a colors() function definition in the hosting
binary, this version ends up being called which is most likely not the
intent. I suspect this might not manifest on Debian as it has a global
-Wl,-Bsymbolic-functions which likely gets passed on during the R
build process.

Is there any particular reason why grDevices isn't compiled with
-fvisibility=hidden like utils (which doesn't exhibit the described
problem), and shouldn't that flag and/or -Wl,-Bsymbolic also make its
way to R CMD's *LIB_*FLAGS to aid in getting correct builds of custom
R libraries too?

Thanks

--- r.c ---
#include<Rinternals.h>
#include<Rdefines.h>
#include<Rembedded.h>
#include<R_ext/Parse.h>
#ifdef BUG
void colors(){puts("all colours of the rainbow, but not the expected ones");}
#endif
int cmd(char*x){SEXP c,e;ParseStatus s;int o;
 PROTECT(c=allocVector(STRSXP,1));SET_STRING_ELT(c,0,mkChar(x))
 PROTECT(e=R_ParseVector(c,1,&s,R_NilValue));
 PROTECT(R_tryEval(VECTOR_ELT(e,0),R_GlobalEnv,&o));
 UNPROTECT(3);return s;}
int main(){Rf_initEmbeddedR(2,(char*[]){"R","--slave"});cmd("grDevices::colors()");return
0;}
---



[Rd] symbol name conflict in grDevices on linux
Oleg Finkelshteyn