From murdoch.duncan at gmail.com  Mon Mar  9 11:50:54 2015
From: murdoch.duncan at gmail.com (Duncan Murdoch)
Date: Mon, 09 Mar 2015 06:50:54 -0400
Subject: [Rd] Notes on building a gcc toolchain for Rtools (but not
	multilib)
In-Reply-To: <CAPuMoLZAWUvcQ-ubBwuqzLONom3ZLGJzzUPbmzm2E_2HzCHWYg@mail.gmail.com>
References: <CAPuMoLZAWUvcQ-ubBwuqzLONom3ZLGJzzUPbmzm2E_2HzCHWYg@mail.gmail.com>
Message-ID: <54FD7B0E.8040500@gmail.com>

On 08/03/2015 10:02 PM, Hsiu-Khuern Tang wrote:
> Hi,
> 
> [This is a follow-up to the "New version of Rtools for Windows" thread
> in January, but I just subscribed and don't know how to reply to an
> old thread -- my apologies.]

I am planning to put a new Rtools online today that uses a different
build of gcc 4.9.2.  I will be concentrating on getting it to work with
all the external libraries before the 3.2.0 release next month.  I'm not
planning to try to get it to work with R-patched, and I expect it won't:
 I needed to make a number of patches to R-devel for compatibility.

It is also not multilib.  It uses static linking of libraries, and SJLJ
exception handling in both 32 and 64 bit builds -- this is necessary for
compatibility with existing compiles of external libraries, but may not
be ideal in 64 bits.  However, I believe using SEH will mean static
linking is impossible, and that leads to problems distributing run-time
DLLs, and I'd rather not worry about those.

Testing of the new Rtools will be appreciated.  I expect the R-devel
builds will start using it tomorrow or the next day.

Duncan Murdoch

> 
> I was able to use the nuwen distro to build a gcc 4.9.2 toolchain and
> use it to build the latest R-patched with it.
> 
> Below are some notes about what I did; I hope they will be useful for
> keeping Rtools up-to-date.
> 
> 
> Note:
> 
> - This is 64-bit only; I tried but was unable to create a multilib toolchain
> 
> - I did not run any tests on the resulting R binary, other than
> starting R and running some basic commands
> 
> - I don't necessarily know what I'm doing!
> 
> 
> Outline of steps:
> 
> - The (nicely done!) nuwen website by Stephan Lavavej has made
> available a MinGW distro and the scripts used to create it.
> 
>   + However, the gcc toolchain there is built with --disable-lib32 (so
> no multilib) and with --disable-gomp (the default)
> 
>   + Moreover, the pthreads library is not included in the distro
> 
>   + I believe building R requires GOMP and pthreads, hence I tried to
> modify the scripts to add these
> 
> - Installing your own toolchain
> 
>   + Read the instructions in the section "How To Build Your Own Distro"
> 
>   + You don't have to rebuild everything in components-12.2.7z: you
> only need the original binutils-2.25.7z and your own build of
> mingw-w64+gcc.7z to replace Rtools's gcc
> 
>   + You will need to run a modified version of Stephan's
> mingw-w64+gcc.sh script.  Besides the gcc source code, you will also
> need to download the pthreads-w32 source code from
> https://www.sourceware.org/pthreads-win32/
> 
>   + Here are the changes I made to the mingw-w64+gcc.sh script:
> 
> --------------------------------------------------
> 
> diff --git a/mingw-w64+gcc.sh b/mingw-w64+gcc.sh
> index 2402ffc..fd44e76 100644
> --- a/mingw-w64+gcc.sh
> +++ b/mingw-w64+gcc.sh
> @@ -8,6 +8,7 @@ source 0_append_distro_path.sh
>  7z x '-oC:\Temp\gcc' gmp-6.0.0a.tar > NUL || fail_with gmp-6.0.0a.tar
> - EPIC FAIL
>  7z x '-oC:\Temp\gcc' mpfr-3.1.2.tar > NUL || fail_with mpfr-3.1.2.tar
> - EPIC FAIL
>  7z x '-oC:\Temp\gcc' mpc-1.0.2.tar > NUL || fail_with mpc-1.0.2.tar - EPIC FAIL
> +7z x '-oC:\Temp\gcc' pthreads-w32-2-9-1-release.tar > NUL ||
> fail_with pthreads-w32-2-9-1-release.tar - EPIC FAIL
> 
>  patch -Z -d /c/temp/gcc/mpfr-3.1.2 -p1 < mpfr.patch
> 
> @@ -25,6 +26,14 @@ make all install "CFLAGS=-s -O3" || fail_with
> mingw-w64 make - EPIC FAIL
>  cd /c/temp/gcc
>  rm -rf build src
> 
> +# Build pthreads-w32.
> +cd pthreads-w32-2-9-1-release
> +make clean GC
> +
> +cp libpthreadGC2.a ../dest/x86_64-w64-mingw32/lib/libpthread.a
> +cp pthread.h sched.h semaphore.h ../dest/x86_64-w64-mingw32/include
> +cp pthreadGC2.dll $X_DISTRO_ROOT/bin/
> +
>  # Prepare to build gcc - set up the in-tree builds of gmp, mpfr, and mpc.
>  mv gcc-4.9.2 src
>  mv gmp-6.0.0 src/gmp
> @@ -40,7 +49,7 @@ cp -r dest/x86_64-w64-mingw32/include
> src/gcc/winsup/mingw/include
>  # Configure.
>  mkdir build
>  cd build
> -../src/configure --enable-languages=c,c++ --build=x86_64-w64-mingw32
> --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32
> --disable-multilib --prefix=/c/temp/gcc/dest
> --with-sysroot=/c/temp/gcc/dest --disable-libstdcxx-pch --disable-lto
> --disable-nls --disable-shared --disable-win32-registry
> --enable-checking=release --with-tune=haswell || fail_with gcc
> configure - EPIC FAIL
> +../src/configure --enable-languages=c,c++,fortran
> --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32
> --target=x86_64-w64-mingw32 --disable-multilib
> --prefix=/c/temp/gcc/dest --with-sysroot=/c/temp/gcc/dest
> --disable-libstdcxx-pch --disable-lto --disable-nls --disable-shared
> --disable-win32-registry --enable-libgomp --enable-checking=release
> --with-tune=haswell || fail_with gcc configure - EPIC FAIL
> 
>  # --enable-languages=c,c++        : I want C and C++ only.
>  # --build=x86_64-w64-mingw32      : I want a native compiler.
> 
> --------------------------------------------------
> 
>   + After running this script, you'll get your own mingw-w64+gcc.7z.
> 
>     - You'll need some patience -- the full 3-stage bootstrap of gcc
> took 10 hours for me
> 
>     - You can replace the "make bootstrap" in the script by "make
> bootstrap2" to omit the last stage.
> 
>   + Install your toolchain:
> 
>     - If you haven't done so, install Rtools.  We will still use the
> command line utilities in C:\Rtools\bin
> 
>     - Make a new directory, e.g., C:\Rtools\nuwen and unpack
> binutils-2.25.7z and mingw-w64+gcc.7z there
> 
>     - Put the above directory in your PATH, in front of any other
> toolchain locations such as C:\Rtools\gcc-4.6.3\bin (it may be better
> to remove the latter from your PATH)
> 
> - Now for the installation of R:
> 
>   + Get the latest R-patched sources (rather than 3.1.2, because
> Duncan (thanks!) have added some useful changes)
> 
>   + Make sure you have the prerequisites for building R (see the R
> Installation and Administration Manual)
> 
>     - In particular, the source files for the recommended packages,
> the support files for Tcl, and the "extsoft" headers and libraries
> 
>   + In src/gnuwin32, copy MkRules.dist to MkRules.local and apply the
> following patch:
> 
> --------------------------------------------------
> 
> --- MkRules.local 2015-03-02 13:57:38.601903500 -0800
> +++ MkRules.local 2015-03-06 10:43:44.708581800 -0800
> @@ -52,6 +52,7 @@
>  # BINPREF =
>  # prefix for 64-bit: path or x86_64-w64-mingw32-
>  # BINPREF64 = x86_64-w64-mingw32-
> +BINPREF64 =
> 
>  # Others use a -m64 or -m32 option to select architectures
>  # M_ARCH =
> @@ -64,6 +65,7 @@
> 
>  # 32- or 64-bit Windows?
>  # WIN = 32
> +WIN = 64
> 
>  # The gcc 4.9.2 64 bit toolchain is set up for the 'medium code'
> model and needs
>  # to remove the .refptr entries from the exports list; this is the default
> @@ -135,12 +137,12 @@
>  # Full paths of extra DLLs that need to be shipped
>  # e.g
>  # DLLs32 = c:/R/bin/pthreadGC2-w32.dll
> -# DLLs64 = c:/R/bin64/pthreadGC2-w64.dll
> +DLLs64 = c:/MinGW/bin/pthreadGC2.dll
>  # DLLs32 =
>  # DLLs64 =
> 
>  # Define this to 1 if using the gcc 4.9.2 toolchain with dynamic linking
> -# COPY_RUNTIME_DLLS =
> +COPY_RUNTIME_DLLS = 1
> 
>  ## ====== configuration macros for building MSI installer ===========
> 
> --------------------------------------------------
> 
>   + (I don't know whether the DLLs64 and COPY_RUNTIME_DLLS changes
> above are necessary or not.  C:/MinGW/bin is where I put the pthreads
> DLL that was built earlier)
> 
>   + Run "make all recommended".  If this works, you should have a
> working R, built using your new toolchain!
> 
> 
> 
> Additional notes:
> 
> - I tried using my new R to install Rcpp from source, but this failed
> because the R build scripts was not able to determine the right set of
> symbols to be exported in the Rcpp DLL.  To solve this, edit the file
> etc/x64/Makeconf under your R source tree, replacing
> 
> NM_FILTER = | sed -e '/.refptr./d'
> 
> by
> 
> NM_FILTER = | sed -e '/.refptr./d; /\.weak\./d'
> 
> 
> Hope this helps,
> 
> - Hsiu-Khuern
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



[Rd] Notes on building a gcc toolchain for Rtools (but not multilib)
Duncan Murdoch