From tom@@@k@liber@ @ending from gm@il@com  Sat Jun  9 10:50:18 2018
From: tom@@@k@liber@ @ending from gm@il@com (Tomas Kalibera)
Date: Sat, 9 Jun 2018 10:50:18 +0200
Subject: [Rd] Byte-compilation failure on different architectures /
 low-memory systems
In-Reply-To: <23318.56725.212774.801149@rob.eddelbuettel.com>
References: <23317.22342.150301.768151@rob.eddelbuettel.com>
 <2ff56f05-bb12-3813-d7c7-347931e731c7@gmail.com>
 <23318.56725.212774.801149@rob.eddelbuettel.com>
Message-ID: <b5676c11-7255-a4fa-0d1c-afa5a1ad4307@gmail.com>

It turned out slightly more complicated, the fallback to AST is was 
already in place, but memory available to malloc got exhausted via heap 
expansion. Messages starting "Error: compilation failed" mean there was 
an error while compiling, including out of memory, and that the AST will 
be used - so these are worth looking at but not fatal. The error message 
"cannot allocate buffer" was the real problem.

We have addressed that with Luke in R-devel and R-patched (more general 
fix might come later to R-devel only). Now there is a GC on the fallback 
path which should release some memory from R heap to be usable by malloc 
again. Also, the compiler needs a bit less memory when analyzing similar 
code patterns as those causing trouble in fBasics (solves the fBasics 
problems with "ulimit -v 1000000")

Best
Tomas

On 06/05/2018 08:59 PM, Dirk Eddelbuettel wrote:
> On 4 June 2018 at 20:06, Tomas Kalibera wrote:
> | thanks for the report. Access to the test system is not necessary, the
> | memory requirements of the byte-code compiler are usually
> | platform-independent and specifically with this package I can reproduce
> | they are very high. We'll have a look what we can do, certainly there
> | should at least be a way to recover and use the uncompiled version when
> | memory allocation fails, this is already done by the JIT but not when
> | compiling during installation. Before R or the package is patched, the
> | only workaround for memory constrained systems is probably to disable
> | byte-compilation of this package, as I read you are doing already.
>
> Yes. And as a shortcut, we just turned it off unconditionally, ie on all
> build architectures.  Worked fine as per
>
>     https://buildd.debian.org/status/package.php?p=fbasics
>
> it has been built everywhere where we have R 3.5.0 (some 20 or so platforms).
>
> The fix you suggest sounds ideal: if possible recover, and maybe WARN.
>
> Dirk
>



[Rd] Byte-compilation failure on different architectures / low-memory systems
Tomas Kalibera