[Rd] R, AIX 64-bit builds - trying to understand root cause for message: "Error: Line starting 'Package: tools ...' is malformed!"
Michael Felt
aixtools at gmail.com
Sun Jan 3 17:28:05 CET 2016
On 2016-01-03 16:59, Michael Felt wrote:
> On 2016-01-01 23:48, peter dalgaard wrote:
>> Nice catch you two!!!
>>
>> Happy New Year
>> -pd
> I am much happier with this great start!
>
> Simon - which compiler)s) did you use: xlc and xlfortran, or
> gcc/gfortran?
>
> I have made some changes to configure(.ac) so maybe my problems are
> self-inflicted. But would be good to know what environment you are using.
>
Using the dist, plus the following change:
diff -ru R-3.2.3.dist/src/extra/tre/tre-internal.h
R-3.2.3.1/src/extra/tre/tre-internal.h
--- R-3.2.3.dist/src/extra/tre/tre-internal.h 2014-06-13
22:15:07.000000000 +0000
+++ R-3.2.3.1/src/extra/tre/tre-internal.h 2016-01-03
16:08:31.000000000 +0000
@@ -47,7 +47,7 @@
#ifdef TRE_WCHAR
/* Wide characters. */
- typedef wint_t tre_cint_t;
+ typedef uint16_t tre_cint_t;
#define TRE_CHAR_MAX WCHAR_MAX
#ifdef TRE_MULTIBYTE
@@ -78,7 +78,7 @@
#else /* !TRE_WCHAR */
/* 8 bit characters. */
- typedef short tre_cint_t;
+ typedef uint16_t tre_cint_t;
#define TRE_CHAR_MAX 255
#define TRE_MB_CUR_MAX 1
@@ -118,7 +118,7 @@
#define tre_ctype(s) wctype(s)
#else /* !TRE_USE_SYSTEM_WCTYPE */
/* Define our own versions of iswctype() and wctype(). */
- typedef int (*tre_ctype_t)(tre_cint_t);
+ typedef uint16_t (*tre_ctype_t)(tre_cint_t);
#define tre_isctype(c, type) ( (type)(c) )
tre_ctype_t tre_ctype(const char *name);
#endif /* !TRE_USE_SYSTEM_WCTYPE */
I get the same result as with my modified configure(.ac), namely:
make[4]: Entering directory
'/data/prj/cran/64/R-aix-3.2.3/src/library/grDevices'
byte-compiling package 'grDevices'
Warning in solve.default(rgb) :
unable to load shared object
'/data/prj/cran/64/R-aix-3.2.3/modules//lapack.so':
rtld: 0712-001 Symbol _gfortran_pow_i4_i4 was referenced
from module /data/prj/cran/64/R-aix-3.2.3/lib/libRlapack.so(),
but a runtime definition
of the symbol was not found.
rtld: 0712-001 Symbol
_gfortran_compare_string/data/prj/cran/64/R-aix-3.2.3/lib/libRlapack.so
was referenced
from module /data/prj/cran/64/R-aix-3.2.3/lib/libRlapack.so(),
but a runtime definition
of the symbol was not found.
rtld: 0712-001 Symbol _gfortran_concat_string was referenced
from module /data/prj/cran/64/R-aix-3.2.3/lib/libRlapack.so(),
but a runtime definition
of the symbol was not found.
rtld: 0712-002 fatal error: exiting.
Error in solve.default(rgb) : LAPACK routines cannot be loaded
Error: unable to load R code in package 'grDevices'
Execution halted
My expectation is that this has to do with some incorrect flags being
passed to the linker (not getting the 64-bit libraries of the compiler).
I have had several long discussions with an AIX gcc specialist and he
has given me several recommendations for changes to the flags used for
creating shared libraries. I shall continue looking into this.
I am happy it is not my configure.ac changes - and I have a few ideas on
how to resolve in configure.ac and/or in m4 files.
MF
> Thanks for looking - and finding!!!
>
> Michael
>>> On 01 Jan 2016, at 22:06 , Simon
>>> Urbanek<simon.urbanek at r-project.org> wrote:
>>>
>>> Ok, found the problem - on platforms that support it TRE uses wint_t
>>> (from wchar.h) as its type for characters (tre_cint_t) which on AIX
>>> is *signed* int. TRE uses liberally conversions between int and
>>> tre_cint_t apparently assuming that the latter is unsigned so
>>> conversions back to int are suitable for comparisons etc. On other
>>> platforms wint_t is unsigned so it works. Manually defining
>>> tre_cint_t to unsigned int fixes the issue.
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>> On Jan 1, 2016, at 12:20 PM, Simon
>>> Urbanek<simon.urbanek at r-project.org> wrote:
>>>
>>>> Michael,
>>>>
>>>> thanks, I'll have a look once my PDP VMs are up again (later
>>>> today). This may be a signedness issue although it's unclear why
>>>> other platforms wouldn't be affected.
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>> On Dec 31, 2015, at 10:14 AM, Michael Felt<aixtools at gmail.com> wrote:
>>>>
>>>>> On 2015-12-30 09:58, Michael Felt wrote:
>>>>>> On 2015-12-29 11:02, Michael Felt wrote:
>>>>>>> This seems to be a problem that goes back a long time - and I
>>>>>>> hope someone who understands what tre is suppossed to be doing
>>>>>>> will look at this.
>>>>>>>
>>>>>>> A short history of other people who have reported on this on
>>>>>>> different versions of AIX. I shall only add that I get the same
>>>>>>> results on AIX 5.3 TL7, AIX 6.1 TL9 and AIX 7.1 TL3.
>>>>>>>
>>>>>>> Basically, with settings that work for AIX and 32-bit - the only
>>>>>>> changes being
>>>>>>> -maix32 becomes -maix64
>>>>>>> and
>>>>>>> export OBJECT_MODE=32 becomes export OBJECT_MODE=64
>>>>>>>
>>>>>>> Then to shorten the 'make' bla bla, first run just make, then
>>>>>>>
>>>>>>> cd src/library/tools
>>>>>>> make -s sysdata
>>>>>>>
>>>>>>> http://article.gmane.org/gmane.comp.lang.r.devel/38817/match=package+tools+malformed
>>>>>>>
>>>>>>> http://article.gmane.org/gmane.comp.lang.r.devel/36886/match=package+tools+malformed
>>>>>>>
>>>>>>> http://article.gmane.org/gmane.comp.lang.r.devel/23372/match=package+tools+malformed
>>>>>>> Date: 2010-01-25 06:55:41 GMT (5 years, 48 weeks, 1 day, 20
>>>>>>> hours and 30 minutes ago)
>>>>>>>
>>>>>>> To that, to get debug data, I have
>>>>>>>
>>>>>>> * added -DTRE_DUGUG to src/extra/tre/Makefile # ALL_CFLAGS =
>>>>>>> $(ALL_CFLAGS_LO) -DTRE_DEBUG
>>>>>>> * rm src/extra/tre/tre-match-parallel.o
>>>>>>> * find . -name \*.so -exec rm {} \;
>>>>>>> * make
>>>>>>> * cd src/library/tools
>>>>>>> * make -s sysdata
>>>>>>>
>>>>>>> Attached are the two script files of the screen output. The
>>>>>>> 32-bit one is more verbose - and contains magically lines such as:
>>>>>>> found match 3037fd14 (while "found" does not occur in the 64-bit
>>>>>>> output)
>>>>>>>
>>>>>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]wc
>>>>>>> /tmp/sysdata.??.*
>>>>>>> 4730 14123 139916 /tmp/sysdata.32.text
>>>>>>> 1312 3688 40528 /tmp/sysdata.64.text
>>>>>>> 6042 17811 180444 total
>>>>>>>
>>>>>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]grep
>>>>>>> -c found /tmp/sysdata.??.*
>>>>>>> /tmp/sysdata.32.text:19
>>>>>>> /tmp/sysdata.64.text:0
>>>>>>>
>>>>>>>
>>>>>>> Hope this brings us (or me), closer to a resolution to an old
>>>>>>> concern.
>>>>>>>
>>>>>>> And, best wishes for the new year!
>>>>>>>
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>> Still hoping for someones curiosity/willingness.
>>>>>>
>>>>>> The differences show up in the first comparision that is made (of
>>>>>> the string "3.2.3" it seems) - 32-bit is on the left, 64-bit on
>>>>>> the right.
>>>>>>
>>>>>> Script command is started on Tue Dec 29 08:39:16 UTC
>>>>>> 2015. | Script command is started on Tue Dec
>>>>>> 29 08:39:56 UTC 2015.
>>>>>> root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make
>>>>>> -s sysdata |
>>>>>> root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make
>>>>>> -s sysdata
>>>>>> installing
>>>>>> 'sysdata.rda'
>>>>>> | installing 'sysdata.rda'
>>>>>> tre_tnfa_run_parallel, input type
>>>>>> 1 |
>>>>>> tre_tnfa_run_parallel, input type 1
>>>>>> length:
>>>>>> -1
>>>>>> | length: -1
>>>>>> pos:chr/code | states and
>>>>>> tags |
>>>>>> pos:chr/code | states and tags
>>>>>> -------------+------------------------------------------------
>>>>>> | -------------+------------------------------------------------
>>>>>> init> 30380200 3038014c
>>>>>> 30380098 | init>
>>>>>> 110cc3040 110cc2f28 110cc2e10
>>>>>> match end offset =
>>>>>> -1 |
>>>>>> match end offset = -1
>>>>>> tre_tnfa_run_parallel, input type
>>>>>> 1 |
>>>>>> tre_tnfa_run_parallel, input type 1
>>>>>> length:
>>>>>> -1
>>>>>> | length: -1
>>>>>> pos:chr/code | states and
>>>>>> tags |
>>>>>> pos:chr/code | states and tags
>>>>>> -------------+------------------------------------------------
>>>>>> | -------------+------------------------------------------------
>>>>>> init>
>>>>>> 3037fb88
>>>>>> | init> 110cc3310
>>>>>> 0: 3/00051 |
>>>>>> 3037fb88/0:0
>>>>>> | 0: 3/00051 | 110cc3310/0:0
>>>>>> 1: ./00046 |
>>>>>> 3037fb88/0:0
>>>>>> | 1: ./00046 | 110cc3310/0:0
>>>>>> init>
>>>>>> 3037fb88
>>>>>> | init> 110cc3310
>>>>>> 1: ./00046 |
>>>>>> 3037fb88/0:1
>>>>>> | 1: ./00046 | 110cc3310/0:1
>>>>>> 2: 2/00050 |
>>>>>> 3037fb88/0:1
>>>>>> | 2: 2/00050 | 110cc3310/0:1
>>>>>> assertion
>>>>>> failed
>>>>>> | assertion failed
>>>>>> init>
>>>>>> 3037fb88
>>>>>> | init> 110cc3310
>>>>>> 2: 2/00050 | 3037fc18/0:1
>>>>>> 3037fb88/0:2 | 2:
>>>>>> 2/00050 | 110cc33f0/0:1 110cc3310/0:2
>>>>>> 3: ./00046 | 3037fc18/0:1
>>>>>> 3037fb88/0:2 | 3:
>>>>>> ./00046 | 110cc33f0/0:1 110cc3310/0:2
>>>>>> assertion failed *** DIFFERENCE
>>>>>> *** | init> 110cc3310
>>>>>> init>
>>>>>> 3037fb88
>>>>>> | 3: ./00046 | 110cc3310/0:3
>>>>>> 3: ./00046 | 3037fc18/0:1
>>>>>> 3037fb88/0:3 | 4:
>>>>>> 3/00051 | 110cc3310/0:3
>>>>>> 4: 3/00051 | 3037fc18/0:1
>>>>>> 3037fb88/0:3 | assertion
>>>>>> failed
>>>>>> assertion
>>>>>> failed
>>>>>> | init> 110cc3310
>>>>>> init>
>>>>>> 3037fb88
>>>>>> | 4: 3/00051 | 110cc33f0/0:3 110cc3310/0:4
>>>>>> 4: 3/00051 | 3037fc18/0:3
>>>>>> 3037fb88/0:4 | 5: /00000
>>>>>> | 110cc33f0/0:3 110cc3310/0:4
>>>>>> 5: /00000 | 3037fc18/0:3 3037fb88/0:4 | init> 110cc3310
>>>>>> found match 3037fd14 *** DIFFERENCE *** |
>>>>>> match end offset = -1
>>>>>> match end offset = 5 *** DIFFERENCE *** |
>>>>>> tre_tnfa_run_parallel, input type 1
>>>>>> tre_tnfa_run_parallel, input type
>>>>>> 1 | length: -1
>>>>>> length:
>>>>>> -1
>>>>>> | pos:chr/code | states and tags
>>>>>> pos:chr/code | states and
>>>>>> tags |
>>>>>> -------------+------------------------------------------------
>>>>>> -------------+------------------------------------------------
>>>>>> | init> 110cc4780 110cc4668 110cc4550
>>>>>> init> 303811c0 3038110c
>>>>>> 30381058 | match end
>>>>>> offset = -1
>>>>>> match end offset =
>>>>>> -1 |
>>>>>> tre_tnfa_run_parallel, input type 1
>>>>>> tre_tnfa_run_parallel, input type
>>>>>> 1 | length: -1
>>>>>> length:
>>>>>> -1
>>>>>> | pos:chr/code | states and tags
>>>>>> pos:chr/code | states and
>>>>>> tags |
>>>>>> -------------+------------------------------------------------
>>>>>> -------------+------------------------------------------------
>>>>>> | init> 110cc5700 110cc55e8 110cc54d0
>>>>>>
>>>>> One day further - looks like tre_compile (or just before, after all).
>>>>>
>>>>> With TRE_DEBUG switched on in tre-compile.c and tre-ast.c I see
>>>>> (snip)
>>>>>
>>>>> --- /tmp/x.32 2015-12-31 15:09:44.000000000 +0000
>>>>> +++ /tmp/x.64 2015-12-31 15:09:30.000000000 +0000
>>>>> @@ -1,5 +1,5 @@
>>>>> - Script command is started on Thu Dec 31 15:04:39 2015.
>>>>> - root at x069:[/data/prj/cran/32/R-aix-3.2.3/src/library/tools]make
>>>>> sysdata
>>>>> + Script command is started on Thu Dec 31 15:08:43 2015.
>>>>> + root at x069:[/data/prj/cran/64/R-aix-3.2.3/src/library/tools]make
>>>>> sysdata
>>>>> installing 'sysdata.rda'
>>>>> echo
>>>>> "tools:::sysdata2LazyLoadDB(\"/data/prj/cran/R-3.2.3/src/library/tools/R/sysdata.rda\",\"../../../library/tools/R\")"
>>>>> | \
>>>>> R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../../bin/R --vanilla --slave
>>>>> @@ -167,7 +167,7 @@
>>>>> initial: 1/1,0, assert 0
>>>>> initial: 0/0, assert 0
>>>>> initial: 0/0, assert 0
>>>>> - final state 30370718
>>>>> + final state 110cba530
>>>>> tre_compile: parsing '(^|[^%])(%%)*%V'
>>>>> AST:
>>>>> catenation, sub 0, 0 tags
>>>>> @@ -177,7 +177,7 @@
>>>>> assertions: bol
>>>>> union, sub -1, 0 tags
>>>>> literal (, $) (0, 36), pos 0, sub -1, 0 tags
>>>>> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
>>>>> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>>>>> iteration {0, -1}, sub -1, 0 tags, greedy
>>>>> catenation, sub 2, 0 tags
>>>>> literal (%, %) (37, 37), pos 1, sub -1, 0 tags
>>>>> @@ -197,7 +197,7 @@
>>>>> Union
>>>>> Literal 0-36
>>>>> After union left
>>>>> - Literal 38-65535
>>>>> + Literal 38--1
>>>>> After union right
>>>>> After union right
>>>>> num_tags += 2
>>>>> @@ -231,7 +231,7 @@
>>>>> assertions: bol
>>>>> union, sub -1, 0 tags
>>>>> literal (, $) (0, 36), pos 0, sub -1, 0 tags
>>>>> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
>>>>> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>>>>> iteration {0, -1}, sub -1, 2 tags, greedy
>>>>> catenation, sub 2, 1 tags
>>>>> literal (%, %) (37, 37), pos 1, sub -1, 1 tags
>>>>> @@ -255,7 +255,7 @@
>>>>> Union
>>>>> Literal 0-36
>>>>> After union left
>>>>> - Literal 38-65535
>>>>> + Literal 38--1
>>>>> After union right
>>>>> After union right
>>>>> tre_add_tag_right: tag 3
>>>>> @@ -342,7 +342,7 @@
>>>>> catenation, sub -1, 0 tags
>>>>> union, sub -1, 0 tags
>>>>> literal (, $) (0, 36), pos 0, sub -1, 0 tags
>>>>> - literal (&, M-^?) (38, 65535), pos 0, sub -1, 0 tags
>>>>> + literal (&, M-^?) (38, -1), pos 0, sub -1, 0 tags
>>>>> tag 4
>>>>>
>>>>> It seems in 32-bit mode -1 is unsigned (65535) but -1 == -1 in
>>>>> 64-bit mode.
>>>>>
>>>>> I suspect I will "find it" - but a proposed change is appreciated.
>>>>>
>>>>> Happy New Year,
>>>>> Michael
>>>>>
>>>>> ______________________________________________
>>>>> R-devel at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>>
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
More information about the R-devel
mailing list