From tangoh at gmail.com  Mon Mar  9 03:02:30 2015
From: tangoh at gmail.com (Hsiu-Khuern Tang)
Date: Sun, 8 Mar 2015 19:02:30 -0700
Subject: [Rd] Notes on building a gcc toolchain for Rtools (but not multilib)
Message-ID: <CAPuMoLZAWUvcQ-ubBwuqzLONom3ZLGJzzUPbmzm2E_2HzCHWYg@mail.gmail.com>

Hi,

[This is a follow-up to the "New version of Rtools for Windows" thread
in January, but I just subscribed and don't know how to reply to an
old thread -- my apologies.]

I was able to use the nuwen distro to build a gcc 4.9.2 toolchain and
use it to build the latest R-patched with it.

Below are some notes about what I did; I hope they will be useful for
keeping Rtools up-to-date.


Note:

- This is 64-bit only; I tried but was unable to create a multilib toolchain

- I did not run any tests on the resulting R binary, other than
starting R and running some basic commands

- I don't necessarily know what I'm doing!


Outline of steps:

- The (nicely done!) nuwen website by Stephan Lavavej has made
available a MinGW distro and the scripts used to create it.

  + However, the gcc toolchain there is built with --disable-lib32 (so
no multilib) and with --disable-gomp (the default)

  + Moreover, the pthreads library is not included in the distro

  + I believe building R requires GOMP and pthreads, hence I tried to
modify the scripts to add these

- Installing your own toolchain

  + Read the instructions in the section "How To Build Your Own Distro"

  + You don't have to rebuild everything in components-12.2.7z: you
only need the original binutils-2.25.7z and your own build of
mingw-w64+gcc.7z to replace Rtools's gcc

  + You will need to run a modified version of Stephan's
mingw-w64+gcc.sh script.  Besides the gcc source code, you will also
need to download the pthreads-w32 source code from
https://www.sourceware.org/pthreads-win32/

  + Here are the changes I made to the mingw-w64+gcc.sh script:

--------------------------------------------------

diff --git a/mingw-w64+gcc.sh b/mingw-w64+gcc.sh
index 2402ffc..fd44e76 100644
--- a/mingw-w64+gcc.sh
+++ b/mingw-w64+gcc.sh
@@ -8,6 +8,7 @@ source 0_append_distro_path.sh
 7z x '-oC:\Temp\gcc' gmp-6.0.0a.tar > NUL || fail_with gmp-6.0.0a.tar
- EPIC FAIL
 7z x '-oC:\Temp\gcc' mpfr-3.1.2.tar > NUL || fail_with mpfr-3.1.2.tar
- EPIC FAIL
 7z x '-oC:\Temp\gcc' mpc-1.0.2.tar > NUL || fail_with mpc-1.0.2.tar - EPIC FAIL
+7z x '-oC:\Temp\gcc' pthreads-w32-2-9-1-release.tar > NUL ||
fail_with pthreads-w32-2-9-1-release.tar - EPIC FAIL

 patch -Z -d /c/temp/gcc/mpfr-3.1.2 -p1 < mpfr.patch

@@ -25,6 +26,14 @@ make all install "CFLAGS=-s -O3" || fail_with
mingw-w64 make - EPIC FAIL
 cd /c/temp/gcc
 rm -rf build src

+# Build pthreads-w32.
+cd pthreads-w32-2-9-1-release
+make clean GC
+
+cp libpthreadGC2.a ../dest/x86_64-w64-mingw32/lib/libpthread.a
+cp pthread.h sched.h semaphore.h ../dest/x86_64-w64-mingw32/include
+cp pthreadGC2.dll $X_DISTRO_ROOT/bin/
+
 # Prepare to build gcc - set up the in-tree builds of gmp, mpfr, and mpc.
 mv gcc-4.9.2 src
 mv gmp-6.0.0 src/gmp
@@ -40,7 +49,7 @@ cp -r dest/x86_64-w64-mingw32/include
src/gcc/winsup/mingw/include
 # Configure.
 mkdir build
 cd build
-../src/configure --enable-languages=c,c++ --build=x86_64-w64-mingw32
--host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32
--disable-multilib --prefix=/c/temp/gcc/dest
--with-sysroot=/c/temp/gcc/dest --disable-libstdcxx-pch --disable-lto
--disable-nls --disable-shared --disable-win32-registry
--enable-checking=release --with-tune=haswell || fail_with gcc
configure - EPIC FAIL
+../src/configure --enable-languages=c,c++,fortran
--build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32
--target=x86_64-w64-mingw32 --disable-multilib
--prefix=/c/temp/gcc/dest --with-sysroot=/c/temp/gcc/dest
--disable-libstdcxx-pch --disable-lto --disable-nls --disable-shared
--disable-win32-registry --enable-libgomp --enable-checking=release
--with-tune=haswell || fail_with gcc configure - EPIC FAIL

 # --enable-languages=c,c++        : I want C and C++ only.
 # --build=x86_64-w64-mingw32      : I want a native compiler.

--------------------------------------------------

  + After running this script, you'll get your own mingw-w64+gcc.7z.

    - You'll need some patience -- the full 3-stage bootstrap of gcc
took 10 hours for me

    - You can replace the "make bootstrap" in the script by "make
bootstrap2" to omit the last stage.

  + Install your toolchain:

    - If you haven't done so, install Rtools.  We will still use the
command line utilities in C:\Rtools\bin

    - Make a new directory, e.g., C:\Rtools\nuwen and unpack
binutils-2.25.7z and mingw-w64+gcc.7z there

    - Put the above directory in your PATH, in front of any other
toolchain locations such as C:\Rtools\gcc-4.6.3\bin (it may be better
to remove the latter from your PATH)

- Now for the installation of R:

  + Get the latest R-patched sources (rather than 3.1.2, because
Duncan (thanks!) have added some useful changes)

  + Make sure you have the prerequisites for building R (see the R
Installation and Administration Manual)

    - In particular, the source files for the recommended packages,
the support files for Tcl, and the "extsoft" headers and libraries

  + In src/gnuwin32, copy MkRules.dist to MkRules.local and apply the
following patch:

--------------------------------------------------

--- MkRules.local 2015-03-02 13:57:38.601903500 -0800
+++ MkRules.local 2015-03-06 10:43:44.708581800 -0800
@@ -52,6 +52,7 @@
 # BINPREF =
 # prefix for 64-bit: path or x86_64-w64-mingw32-
 # BINPREF64 = x86_64-w64-mingw32-
+BINPREF64 =

 # Others use a -m64 or -m32 option to select architectures
 # M_ARCH =
@@ -64,6 +65,7 @@

 # 32- or 64-bit Windows?
 # WIN = 32
+WIN = 64

 # The gcc 4.9.2 64 bit toolchain is set up for the 'medium code'
model and needs
 # to remove the .refptr entries from the exports list; this is the default
@@ -135,12 +137,12 @@
 # Full paths of extra DLLs that need to be shipped
 # e.g
 # DLLs32 = c:/R/bin/pthreadGC2-w32.dll
-# DLLs64 = c:/R/bin64/pthreadGC2-w64.dll
+DLLs64 = c:/MinGW/bin/pthreadGC2.dll
 # DLLs32 =
 # DLLs64 =

 # Define this to 1 if using the gcc 4.9.2 toolchain with dynamic linking
-# COPY_RUNTIME_DLLS =
+COPY_RUNTIME_DLLS = 1

 ## ====== configuration macros for building MSI installer ===========

--------------------------------------------------

  + (I don't know whether the DLLs64 and COPY_RUNTIME_DLLS changes
above are necessary or not.  C:/MinGW/bin is where I put the pthreads
DLL that was built earlier)

  + Run "make all recommended".  If this works, you should have a
working R, built using your new toolchain!



Additional notes:

- I tried using my new R to install Rcpp from source, but this failed
because the R build scripts was not able to determine the right set of
symbols to be exported in the Rcpp DLL.  To solve this, edit the file
etc/x64/Makeconf under your R source tree, replacing

NM_FILTER = | sed -e '/.refptr./d'

by

NM_FILTER = | sed -e '/.refptr./d; /\.weak\./d'


Hope this helps,

- Hsiu-Khuern



[Rd] Notes on building a gcc toolchain for Rtools (but not multilib)
Hsiu-Khuern Tang