[R] dyn.load(now = FALSE) not actually lazy?
Michael Milton
ttm|gue|tt @end|ng |rom gm@||@com
Thu Feb 2 03:35:39 CET 2023
Hi Ivan,
Thanks for the suggestion.
I found that libpqwalreceiver.so, which is part of pgsql-15.1, made for a
simpler example of my issue. So, using your advice, I ran the following in
my shell:
LD_DEBUG=libs R -e
'dyn.load("/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so",
now=FALSE)'
I have attached the relevant part of the output from this command below.
The gist of it is that, indeed, R starts to look for dependent libraries
even if now=FALSE. Maybe it is the case that, as Duncan suggests, this is
intended behaviour? But I think it's not ideal behaviour because otherwise
it requires you to topologically sort all library dependencies and load
them in the optimal order, which is very difficult to do. Is there someone
involved in R development who can explain this behaviour to me?
Cheers.
***
>
dyn.load("/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so",
now=FALSE)
1655: find library=libpq.so.5 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libpq.so.5
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libpq.so.5
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libpq.so.5
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libpq.so.5
1655: trying file=/usr/local/lib64/libpq.so.5
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libpq.so.5
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libpq.so.5
1655: search
path=/usr/pgsql-15/lib/tls/x86_64:/usr/pgsql-15/lib/tls:/usr/pgsql-15/lib/x86_64:/usr/pgsql-15/lib
(RUNPATH from file
/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so)
1655: trying file=/usr/pgsql-15/lib/tls/x86_64/libpq.so.5
1655: trying file=/usr/pgsql-15/lib/tls/libpq.so.5
1655: trying file=/usr/pgsql-15/lib/x86_64/libpq.so.5
1655: trying file=/usr/pgsql-15/lib/libpq.so.5
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libpq.so.5
1655:
1655: find library=libssl.so.10 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libssl.so.10
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libssl.so.10
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libssl.so.10
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libssl.so.10
1655: trying file=/usr/local/lib64/libssl.so.10
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libssl.so.10
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libssl.so.10
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libssl.so.10
1655:
1655: find library=libcrypto.so.10 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libcrypto.so.10
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libcrypto.so.10
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libcrypto.so.10
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libcrypto.so.10
1655: trying file=/usr/local/lib64/libcrypto.so.10
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libcrypto.so.10
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libcrypto.so.10
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libcrypto.so.10
1655:
1655: find library=libkrb5.so.3 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libkrb5.so.3
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libkrb5.so.3
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libkrb5.so.3
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libkrb5.so.3
1655: trying file=/usr/local/lib64/libkrb5.so.3
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libkrb5.so.3
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libkrb5.so.3
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libkrb5.so.3
1655:
1655: find library=libcom_err.so.2 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libcom_err.so.2
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libcom_err.so.2
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libcom_err.so.2
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libcom_err.so.2
1655: trying file=/usr/local/lib64/libcom_err.so.2
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libcom_err.so.2
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libcom_err.so.2
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libcom_err.so.2
1655:
1655: find library=libgssapi_krb5.so.2 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libgssapi_krb5.so.2
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libgssapi_krb5.so.2
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libgssapi_krb5.so.2
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libgssapi_krb5.so.2
1655: trying file=/usr/local/lib64/libgssapi_krb5.so.2
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libgssapi_krb5.so.2
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libgssapi_krb5.so.2
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libgssapi_krb5.so.2
1655:
1655: find library=libldap_r-2.4.so.2 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libldap_r-2.4.so.2
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libldap_r-2.4.so.2
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libldap_r-2.4.so.2
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libldap_r-2.4.so.2
1655: trying file=/usr/local/lib64/libldap_r-2.4.so.2
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libldap_r-2.4.so.2
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libldap_r-2.4.so.2
D_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libplc4.so
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libplc4.so
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libplc4.so
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libplc4.so
1655: trying file=/usr/local/lib64/libplc4.so
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libplc4.so
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libplc4.so
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libplc4.so
1655:
1655: find library=libnspr4.so [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libnspr4.so
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libnspr4.so
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libnspr4.so
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libnspr4.so
1655: trying file=/usr/local/lib64/libnspr4.so
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libnspr4.so
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libnspr4.so
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libnspr4.so
1655:
1655: find library=libselinux.so.1 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libselinux.so.1
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libselinux.so.1
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libselinux.so.1
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libselinux.so.1
1655: trying file=/usr/local/lib64/libselinux.so.1
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libselinux.so.1
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libselinux.so.1
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libselinux.so.1
1655:
1655: find library=libpcre.so.1 [0]; searching
1655: search
path=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64:/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib:/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib:/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib:/usr/local/lib64:/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server:/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib
(LD_LIBRARY_PATH)
1655: trying
file=/stornext/System/data/apps/gcc/gcc-11.1.0/lib64/libpcre.so.1
1655: trying
file=/stornext/System/data/apps/pcre2/pcre2-10.36-gcc-11.1.0/lib/libpcre.so.1
1655: trying
file=/stornext/System/data/tools/openSSL/openSSL-1.1.1n/lib/libpcre.so.1
1655: trying
file=/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib/libpcre.so.1
1655: trying file=/usr/local/lib64/libpcre.so.1
1655: trying
file=/stornext/System/data/tools/openjdk/openjdk-13.0.2/lib/server/libpcre.so.1
1655: trying
file=/stornext/System/data/apps/hdf5/hdf5-1.8.20/lib/libpcre.so.1
1655: search cache=/etc/ld.so.cache
1655: trying file=/lib64/libpcre.so.1
1655:
1655:
/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so:
error: symbol lookup error: undefined symbol: work_mem (fatal)
Error in
dyn.load("/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so",
:
unable to load shared object
'/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so':
/stornext/System/data/tools/pgsql/pgsql-15.1/lib/libpqwalreceiver.so:
undefined symbol: work_mem
Execution halted
On Wed, Feb 1, 2023 at 9:05 PM Ivan Krylov <krylov.r00t using gmail.com> wrote:
> В Wed, 1 Feb 2023 14:16:54 +1100
> Michael Milton <ttmigueltt using gmail.com> пишет:
>
> > Is this a bug in the `dyn.load` implementation for R? If not, why is
> > it behaving like this? What should I do about it?
>
> On Unix-like systems, dyn.load forwards its arguments to dlopen(). It
> should be possible to confirm with a debugger that R passes RTLD_NOW to
> dlopen() when calling dyn.load(now = TRUE) and RTLD_LAZY when calling
> dyn.load(now = FALSE).
>
> I don't know for sure why the symbols are being resolved despite you
> asked the linker not to. Did something in the system set the
> LD_BIND_NOW environment variable? Do any of the libraries in the
> dependency tree have any constructors (C++ or
> __attribute__((constructor)) or otherwise mentioned in .ini* sections)
> that rely on MKL being available at initialisation time?
>
> If you launch R with the environment variable LD_DEBUG=libs set, the
> debugging output may shine some light on the problem.
>
> --
> Best regards,
> Ivan
>
[[alternative HTML version deleted]]
More information about the R-help
mailing list