From simon.urbanek at r-project.org  Fri Jan  1 18:20:07 2016
From: simon.urbanek at r-project.org (Simon Urbanek)
Date: Fri, 1 Jan 2016 12:20:07 -0500
Subject: [Rd] R,
	AIX 64-bit builds - trying to understand root cause for message:
	"Error: Line starting 'Package: tools ...' is malformed!"
In-Reply-To: <5685466E.7000804@gmail.com>
References: <56825A4B.7010508@gmail.com> <56839C9E.2010401@gmail.com>
	<5685466E.7000804@gmail.com>
Message-ID: <4BED4A65-8EE9-49B8-8AFB-4E6CC3BB5BBF@r-project.org>

Michael,

thanks, I'll have a look once my PDP VMs are up again (later today). This may be a signedness issue although it's unclear why other platforms wouldn't be affected.

Cheers,
Simon


On Dec 31, 2015, at 10:14 AM, Michael Felt <aixtools at gmail.com> wrote:

> On 2015-12-30 09:58, Michael Felt wrote:
>> On 2015-12-29 11:02, Michael Felt wrote:
>>> This seems to be a problem that goes back a long time - and I hope someone who understands what tre is suppossed to be doing will look at this.
>>> 
>>> A short history of other people who have reported on this on different versions of AIX. I shall only add that I get the same results on AIX 5.3 TL7, AIX 6.1 TL9 and AIX 7.1 TL3.
>>> 
>>> Basically, with settings that work for AIX and 32-bit - the only changes being
>>> -maix32 becomes -maix64
>>> and
>>> export OBJECT_MODE=32 becomes export OBJECT_MODE=64
>>> 
>>> Then to shorten the 'make' bla bla, first run just make, then
>>> 
>>> cd src/library/tools
>>> make -s sysdata
>>> 
>>> http://article.gmane.org/gmane.comp.lang.r.devel/38817/match=package+tools+malformed 
>>> http://article.gmane.org/gmane.comp.lang.r.devel/36886/match=package+tools+malformed 
>>> http://article.gmane.org/gmane.comp.lang.r.devel/23372/match=package+tools+malformed Date: 2010-01-25 06:55:41 GMT (5 years, 48 weeks, 1 day, 20 hours and 30 minutes ago)
>>> 
>>> To that, to get debug data, I have
>>> 
>>> * added -DTRE_DUGUG to src/extra/tre/Makefile # ALL_CFLAGS = $(ALL_CFLAGS_LO) -DTRE_DEBUG
>>> * rm src/extra/tre/tre-match-parallel.o
>>> * find . -name \*.so -exec rm {} \;
>>> * make
>>> * cd src/library/tools
>>> * make -s sysdata
>>> 
>>> Attached are the two script files of the screen output. The 32-bit one is more verbose - and contains magically lines such as:
>>>   found match 3037fd14 (while "found" does not occur in the 64-bit output)
>>> 
>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]wc /tmp/sysdata.??.*
>>>     4730   14123  139916 /tmp/sysdata.32.text
>>>     1312    3688   40528 /tmp/sysdata.64.text
>>>     6042   17811  180444 total
>>> 
>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]grep -c found /tmp/sysdata.??.*
>>> /tmp/sysdata.32.text:19
>>> /tmp/sysdata.64.text:0
>>> 
>>> 
>>> Hope this brings us (or me), closer to a resolution to an old concern.
>>> 
>>> And, best wishes for the new year!
>>> 
>>> Michael
>>> 
>>> 
>> Still hoping for someones curiosity/willingness.
>> 
>> The differences show up in the first comparision that is made (of the string "3.2.3" it seems) - 32-bit is on the left, 64-bit on the right.
>> 
>> Script command is started on Tue Dec 29 08:39:16 UTC 2015.                     |  Script command is started on Tue Dec 29 08:39:56 UTC 2015.
>> root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make -s sysdata     |  root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make -s sysdata
>> installing 'sysdata.rda'                                                       |  installing 'sysdata.rda'
>> tre_tnfa_run_parallel, input type 1                                            |  tre_tnfa_run_parallel, input type 1
>> length: -1                                                                     |  length: -1
>> pos:chr/code | states and tags                                                 |  pos:chr/code | states and tags
>> -------------+------------------------------------------------                 |  -------------+------------------------------------------------
>> init > 30380200 3038014c 30380098                                             |   init > 110cc3040 110cc2f28 110cc2e10
>> match end offset = -1                                                          |  match end offset = -1
>> tre_tnfa_run_parallel, input type 1                                            |  tre_tnfa_run_parallel, input type 1
>> length: -1                                                                     |  length: -1
>> pos:chr/code | states and tags                                                 |  pos:chr/code | states and tags
>> -------------+------------------------------------------------                 |  -------------+------------------------------------------------
>> init > 3037fb88                                                               |   init > 110cc3310
>>  0: 3/00051 | 3037fb88/0:0                                                    |    0: 3/00051 | 110cc3310/0:0
>>  1: ./00046 | 3037fb88/0:0                                                    |    1: ./00046 | 110cc3310/0:0
>> init > 3037fb88                                                               |   init > 110cc3310
>>  1: ./00046 | 3037fb88/0:1                                                    |    1: ./00046 | 110cc3310/0:1
>>  2: 2/00050 | 3037fb88/0:1                                                    |    2: 2/00050 | 110cc3310/0:1
>> assertion failed                                                               |  assertion failed
>> init > 3037fb88                                                               |   init > 110cc3310
>>  2: 2/00050 | 3037fc18/0:1 3037fb88/0:2                                       |    2: 2/00050 | 110cc33f0/0:1 110cc3310/0:2
>>  3: ./00046 | 3037fc18/0:1 3037fb88/0:2                                       |    3: ./00046 | 110cc33f0/0:1 110cc3310/0:2
>> assertion failed                           *** DIFFERENCE ***                  |   init > 110cc3310
>> init > 3037fb88                                                               |    3: ./00046 | 110cc3310/0:3
>>  3: ./00046 | 3037fc18/0:1 3037fb88/0:3                                       |    4: 3/00051 | 110cc3310/0:3
>>  4: 3/00051 | 3037fc18/0:1 3037fb88/0:3                                       |  assertion failed
>> assertion failed                                                               |   init > 110cc3310
>> init > 3037fb88                                                               |    4: 3/00051 | 110cc33f0/0:3 110cc3310/0:4
>>  4: 3/00051 | 3037fc18/0:3 3037fb88/0:4                                       |    5: /00000 | 110cc33f0/0:3 110cc3310/0:4
>>  5: /00000 | 3037fc18/0:3 3037fb88/0:4 |   init > 110cc3310
>>  found match 3037fd14 *** DIFFERENCE ***                   |  match end offset = -1
>> match end offset = 5 *** DIFFERENCE ***                   |  tre_tnfa_run_parallel, input type 1
>> tre_tnfa_run_parallel, input type 1                                            |  length: -1
>> length: -1                                                                     |  pos:chr/code | states and tags
>> pos:chr/code | states and tags                                                 |  -------------+------------------------------------------------
>> -------------+------------------------------------------------                 |   init > 110cc4780 110cc4668 110cc4550
>> init > 303811c0 3038110c 30381058                                             |  match end offset = -1
>> match end offset = -1                                                          |  tre_tnfa_run_parallel, input type 1
>> tre_tnfa_run_parallel, input type 1                                            |  length: -1
>> length: -1                                                                     |  pos:chr/code | states and tags
>> pos:chr/code | states and tags                                                 |  -------------+------------------------------------------------
>> -------------+------------------------------------------------                 |   init > 110cc5700 110cc55e8 110cc54d0
>> 
> One day further - looks like tre_compile (or just before, after all).
> 
> With TRE_DEBUG switched on in tre-compile.c and tre-ast.c I see (snip)
> 
> --- /tmp/x.32   2015-12-31 15:09:44.000000000 +0000
> +++ /tmp/x.64   2015-12-31 15:09:30.000000000 +0000
> @@ -1,5 +1,5 @@
> - Script command is started on Thu Dec 31 15:04:39 2015.
> - root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make sysdata
> + Script command is started on Thu Dec 31 15:08:43 2015.
> + root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make sysdata
>  installing 'sysdata.rda'
>  echo "tools:::sysdata2LazyLoadDB(\"/data/prj/cran/R-3.2.3/src/library/tools/R/sysdata.rda\",\"../../../library/tools/R\")" | \
>    R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../../bin/R --vanilla --slave
> @@ -167,7 +167,7 @@
>  initial: 1/1,0, assert 0
>  initial: 0/0, assert 0
>  initial: 0/0, assert 0
> - final state 30370718
> + final state 110cba530
>  tre_compile: parsing '(^|[^%])(%%)*%V'
>  AST:
>  catenation, sub 0, 0 tags
> @@ -177,7 +177,7 @@
>          assertions: bol
>          union, sub -1, 0 tags
>            literal (, $) (0, 36), pos 0, sub -1, 0 tags
> -           literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
> +           literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>        iteration {0, -1}, sub -1, 0 tags, greedy
>          catenation, sub 2, 0 tags
>            literal (%, %) (37, 37), pos 1, sub -1, 0 tags
> @@ -197,7 +197,7 @@
>  Union
>  Literal 0-36
>  After union left
> - Literal 38-65535
> + Literal 38--1
>  After union right
>  After union right
>    num_tags += 2
> @@ -231,7 +231,7 @@
>          assertions: bol
>          union, sub -1, 0 tags
>            literal (, $) (0, 36), pos 0, sub -1, 0 tags
> -           literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
> +           literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>        iteration {0, -1}, sub -1, 2 tags, greedy
>          catenation, sub 2, 1 tags
>            literal (%, %) (37, 37), pos 1, sub -1, 1 tags
> @@ -255,7 +255,7 @@
>  Union
>  Literal 0-36
>  After union left
> - Literal 38-65535
> + Literal 38--1
>  After union right
>  After union right
>  tre_add_tag_right: tag 3
> @@ -342,7 +342,7 @@
>            catenation, sub -1, 0 tags
>              union, sub -1, 0 tags
>                literal (, $) (0, 36), pos 0, sub -1, 0 tags
> -               literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
> +               literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>              tag 4
> 
> It seems in 32-bit mode -1 is unsigned (65535) but -1 == -1 in 64-bit mode.
> 
> I suspect I will "find it" - but a proposed change is appreciated.
> 
> Happy New Year,
> Michael
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 



[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
Simon Urbanek