From aixtools at gmail.com  Fri Dec 18 10:58:05 2015
From: aixtools at gmail.com (Michael Felt)
Date: Fri, 18 Dec 2015 10:58:05 +0100
Subject: [Rd] Assistance much appreciated
In-Reply-To: <DCB5B35F-85DD-4078-8F02-940C368DCE95@r-project.org>
References: <5672EA93.104@gmail.com>
	<DCB5B35F-85DD-4078-8F02-940C368DCE95@r-project.org>
Message-ID: <5673D8AD.7030405@gmail.com>

On 2015-12-18 02:29, Simon Urbanek wrote:
> Michael,
>
> I got access to PDP AIX so I can try to replicate your problem. Can you, please, share exactly your setup - AIX version and well as how exactly you installed the compilers (=where from)? I can then try to replicate it. AFAICS there is no official binary for gfortran nor gcc 4.7 so it must be some 3rd party - which could also be a problem.
>
> Thanks,
> Simon
I built gcc and gfortran myself. Available via 
http://download.aixtools.net/test/aixtools.obj.gcc.4.7.4.0.aix537.I 
(still in the test directory because the fileset name should be 
aixtools.gnu.gcc) - other filesets and instructions are available at: 
http://download.aixtools.net/gnu/gcc/

Note, especially, I was not smart enough - then - to automatically run 
the mkheaders command (full command is either

  /opt/libexec/gcc/powerpc-ibm-aix5.3.7.0/4.7.4/install-tools/mkheaders or  /opt/libexec/gcc/powerpc-ibm-aix5.3.0.0/4.7.4/install-tools/mkheaders
depending on the version of libtool I had installed when packaging (the latest also looks at the TL level, hence 5.3.7.0 rather that 5.3.0.0)


Only looking at the rte (runtime) filesets - this is what I have added: 
All are available via http:://www.aixtools.net (wiki interface) or 
http://download.aixtools.net (self-service).

The AIXTOOLBOX approach uses rpm for packaging - which I did not 
understand when I got started several years back. And I am not switching 
to rpm - following in the footsteps of Perzl because:
a) Michael Perzl does an excellent job of keeping the AIXTOOLBOX where 
it should be - no need to replicate that
b) I hate the RPM dependency nightmare (e.g., installing three rpm -dev 
packages  just to get 5 include files)
c) The RPM.spec files often (well occaisionally, but once is once too 
many for me) overwrite AIX/IBM installp files - so updates of one damage 
the other
d) RPM and NIM did not work as well as NIM and INSTALLP

Almost end of soapbox --- In AIX 5.2 IBM included a tool called 
mkinstallp. I have a tool around that - that I call buildaix - that 
largely automates packaging OSS/FOSS, especially when the packaging 
follows GNU build guidelines.

root at x065:[/data/prj/cran/R-3.2.3/src/extra/tre] aixtools | grep rte <
   aixtools.32.R.rte          3.1.3.0    C     F    1548 1130 1457
   aixtools.bison.rte         3.0.4.0    C     F    bison 1504 0125 1445
   aixtools.buildaix.rte      2.0.3.1    C     F    1540 1007 2004
   aixtools.bzip2.rte         1.0.6.0    C     F    1543 1030 1351
   aixtools.curl.rte         7.45.0.0    C     F    1544 1106 1841
   aixtools.gmp.rte           5.0.5.0    C     F    gmp version 5.0.5 
for powerpc
   aixtools.gnu.bash.rte     4.3.42.0    C     F    1537 0913 1505
   aixtools.gnu.gettext.rte  0.19.5.0    C     F    1531 0807 0000
   aixtools.gnu.gzip.rte      1.6.0.0    C     F    1550 1216 1405
   aixtools.gnu.gzip.rte.64   1.6.0.0    C     F    1550 1216 1417
   aixtools.gnu.m4.rte       1.4.17.0    C     F    1523 0613 1221
   aixtools.gnu.make.rte      4.1.0.0    C     F    1523 0613 1158
   aixtools.libiconv.rte     1.14.0.0    C     F    libiconv 1447 1127 1728
   aixtools.libjpeg.rte       6.2.0.0    C     F    libjpeg version 
6.2.0 for
   aixtools.libpng.rte       1.5.12.0    C     F    libpng version 
1.5.12 for
   aixtools.mpc.rte           1.0.2.0    C     F    mpc 2014.Oct.31 
15.44 CUT
   aixtools.mpfr.rte          2.4.2.0    C     F    mpfr 2014.Oct.31 
15.19 CUT
   aixtools.obj.gcc.rte       4.7.4.0    C     F    1515 0416 2134
   aixtools.pcre.rte         8.37.0.0    C     F    1544 1106 1445
   aixtools.pkgconfig.gcc.rte
   aixtools.tiff.rte          4.0.6.0    C     F    1542 1024 2144
   aixtools.xz.rte            5.0.8.0    C     F    1522 0602 0809
   aixtools.zlib.rte          1.2.8.1    C     F    zlib 1505 0203 1627

In closing, I made the mistake of thinking people know installp as well 
as they know rpm (at command-line). If you do not know installp at the 
command line level
a) run rm -f /where/you/download/to/.toc after everytime you add a file
b) use "smit install" to install filesets

If you are comfortable with installp cli interface the standard commands 
to remember are:
a) inutoc /where/you/download/to
b) installp -d /where/you/download/to -L # to list filesets
c) installp -d /where/you/download/to -agX <what you want to install> | 
"all"

Part II

a) I download R-project files to /data/prj/cran/downloads
b) cd /data/prj/cran
     gzip -dc downloads/R-3.2.3.tar.gz | tar xf -
     cp -rp R-3.2.3 R-3.2.3.dist
c) ls -l 32/*.env 64/*.env

sdiff -w ??/aix.env
cat bin/my_shared.ksh

d) slibclean; rm -rf 32/R-3.2.3; rm -rf 64/R-3.2.3
     mkdir 32/R-3.2.3
     mkdir 64/R-3.2.3

e) note current diffs between R-3.2.3 and R-3.2.3.dist
root at x072:[/data/prj/cran]diff -ru R-3.2.3.dist R-3.2.3
diff -ru R-3.2.3.dist/configure R-3.2.3/configure
--- R-3.2.3.dist/configure      2015-12-02 23:16:46.000000000 +0000
+++ R-3.2.3/configure   2015-12-16 10:52:15.000000000 +0000
@@ -26339,11 +26339,11 @@
       ##ADD: However, for example, symbol in libc of memcpy is 
__memmove,__memmove64.
       ##ADD: This black magic puts lc before lR and pockets this.
       if test "x${OBJECT_MODE}" = "x64"; then
-       main_ldflags="${wl}-brtl ${wl}-bexpall ${wl}-bpT:0x100000000 
${wl}-bpD:0x110000000 -lc"
+       main_ldflags="${wl}-brtl ${wl}-bpT:0x100000000 
${wl}-bpD:0x110000000 -lc"
       else
-       main_ldflags="${wl}-brtl ${wl}-bexpall -lc"
+       main_ldflags="${wl}-brtl ${wl}-bmaxdata:0xd0000000/dsa -lc"
       fi
-     shlib_ldflags="${wl}-brtl ${wl}-G ${wl}-bexpall ${wl}-bnoentry -lc"
+     shlib_ldflags="${wl}-G ${wl}-bnoentry -lc"
       SHLIB_LIBADD="\$(LIBM)"
       shlib_cxxldflags="${shlib_ldflags}"
       if test "${GCC}" = yes; then
diff -ru R-3.2.3.dist/share/make/basepkg.mk R-3.2.3/share/make/basepkg.mk
--- R-3.2.3.dist/share/make/basepkg.mk  2015-03-18 23:04:12.000000000 +0000
+++ R-3.2.3/share/make/basepkg.mk       2015-12-17 12:01:02.000000000 +0000
@@ -148,7 +148,9 @@

   sysdata: $(srcdir)/R/sysdata.rda
         @$(ECHO) "installing 'sysdata.rda'"
-       @$(ECHO) 
"tools:::sysdata2LazyLoadDB(\"$(srcdir)/R/sysdata.rda\",\"$(top_builddir)/library/$(pkg)/R\")" 
| \
+       $(ECHO) "warnings()" | \
+         R_DEFAULT_PACKAGES=NULL LC_ALL=C $(R_EXE)
+       $(ECHO) 
"tools:::sysdata2LazyLoadDB(\"$(srcdir)/R/sysdata.rda\",\"$(top_builddir)/library/$(pkg)/R\")" 
| \
           R_DEFAULT_PACKAGES=NULL LC_ALL=C $(R_EXE)


diff -ru R-3.2.3.dist/src/include/Defn.h R-3.2.3/src/include/Defn.h
--- R-3.2.3.dist/src/include/Defn.h     2015-08-25 22:15:24.000000000 +0000
+++ R-3.2.3/src/include/Defn.h  2015-12-16 10:53:08.000000000 +0000
@@ -1271,7 +1271,7 @@
   void invalidate_cached_recodings(void);  /* from sysutils.c */
   void resetICUcollator(void); /* from util.c */
   void dt_invalidate_locale(); /* from Rstrptime.h */
- int R_OutputCon; /* from connections.c */
+ extern int R_OutputCon; /* from connections.c */
   extern int R_InitReadItemDepth, R_ReadItemDepth; /* from serialize.c */
   void get_current_mem(size_t *,size_t *,size_t *); /* from memory.c */
   unsigned long get_duplicate_counter(void);  /* from duplicate.c */
diff -ru R-3.2.3.dist/src/library/tools/R/makeLazyLoad.R 
R-3.2.3/src/library/tools/R/makeLazyLoad.R
--- R-3.2.3.dist/src/library/tools/R/makeLazyLoad.R     2015-08-25 
22:16:54.000000000 +0000
+++ R-3.2.3/src/library/tools/R/makeLazyLoad.R  2015-12-17 
06:53:34.000000000 +0000
@@ -38,9 +38,13 @@

   sysdata2LazyLoadDB <- function(srcFile, destDir, compress = TRUE)
   {
+ browser()
       e <- new.env(hash=TRUE)
+ # browser()
       load(srcFile, e)
+ # browser()
       makeLazyLoadDB(e, file.path(destDir, "sysdata"), compress = compress)
+ # browser()
   }

   list_data_in_pkg <- function(package, lib.loc = NULL, dataDir = NULL)
diff -ru R-3.2.3.dist/src/library/tools/src/gramLatex.c 
R-3.2.3/src/library/tools/src/gramLatex.c
--- R-3.2.3.dist/src/library/tools/src/gramLatex.c      2015-12-09 
23:20:01.000000000 +0000
+++ R-3.2.3/src/library/tools/src/gramLatex.c   2015-12-17 
14:50:14.000000000 +0000
@@ -1358,7 +1358,7 @@


   /* The lookahead symbol.  */
- int yychar;
+ extern int yychar;


   #ifndef YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN
@@ -1370,7 +1370,7 @@
   #endif

   /* The semantic value of the lookahead symbol.  */
- YYSTYPE yylval YY_INITIAL_VALUE(yyval_default);
+ extern YYSTYPE yylval YY_INITIAL_VALUE(yyval_default);

   /* Location data for the lookahead symbol.  */
   YYLTYPE yylloc
@@ -1381,7 +1381,7 @@


   /* Number of syntax errors so far.  */
- int yynerrs;
+ extern int yynerrs;


   /*----------.
diff -ru R-3.2.3.dist/src/library/tools/src/gramRd.c 
R-3.2.3/src/library/tools/src/gramRd.c
--- R-3.2.3.dist/src/library/tools/src/gramRd.c 2015-12-09 
23:20:01.000000000 +0000
+++ R-3.2.3/src/library/tools/src/gramRd.c      2015-12-17 
14:51:19.000000000 +0000
@@ -1972,7 +1972,7 @@


   /* The lookahead symbol.  */
- int yychar;
+ extern int yychar;


   #ifndef YY_IGNORE_MAYBE_UNINITIALIZED_BEGIN
@@ -1984,7 +1984,7 @@
   #endif

   /* The semantic value of the lookahead symbol.  */
- YYSTYPE yylval YY_INITIAL_VALUE(yyval_default);
+ extern YYSTYPE yylval YY_INITIAL_VALUE(yyval_default);

   /* Location data for the lookahead symbol.  */
   YYLTYPE yylloc
@@ -1995,7 +1995,7 @@


   /* Number of syntax errors so far.  */
- int yynerrs;
+ extern int yynerrs;


   /*----------.

+++++++++
The changes in /src/library/tools/R/makeLazyLoad.R are not relevant 
(initial learning...)

f) cd 32/R-3.2.3; ../../bin/my_shared.ksh aix R-3.2.3; export 
OBJECT_MODE=32; make -j4; cd -
g) cd 64/R-3.2.3; ../../bin/my_shared.ksh aix R-3.2.3; export 
OBJECT_MODE=64; make -j4 && cd -

Michael

p.s. please keep me posted on any difficulties - so I can add better 
instructions on the wiki.
p.p.s. - I am using R-3.2.3 to by-pass the removal of the src/extra packages



[Rd] Assistance much appreciated
Michael Felt