[R] dyn.load(now = FALSE) not actually lazy?

[Duncan Murdoch]

According to the help page for dyn.load, I think this is within the 
allowed behaviour:

"now:
a logical controlling whether all symbols are resolved (and relocated) 
immediately the library is loaded or deferred until they are used. This 
control is useful for developers testing whether a library is complete 
and has all the necessary symbols, and for users to ignore missing 
symbols. Whether this has any effect is system-dependent."

It appears to be intended for a DLL that doesn't define all the symbols 
that your program will use, not for a DLL that has external references 
that can't be resolved.  And there's that last sentence.

I think for what you want, you'd have to write the DLL (i.e. libtorch) 
in such a way that it does delayed loading of its dependencies.

Duncan Murdoch

On 31/01/2023 10:16 p.m., Michael Milton wrote:
> On Linux, if I have a .so file that has a dependency on another .so, and I
> `dyn.load(now=FALSE)` the first one, R seems to try to resolve the symbols
> immediately, causing the load to fail.
> 
> For example, I have `libtorch` installed on my HPC. Note that it links to
> various libs such as `libcudart.so` and `libmkl_intel_lp64.so.2` which
> aren't currently in my library path:
> 
> ➜  ~ ldd
> /stornext/System/data/nvidia/libtorch-gpu/libtorch-gpu-1.12.1/lib/libtorch_cpu.so
>          linux-vdso.so.1 =>  (0x00007ffcab58c000)
>          libgomp.so.1 =>
> /stornext/System/data/apps/gcc/gcc-11.2.0/lib64/libgomp.so.1
> (0x00007f8cb22bf000)
>          libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8cb20a3000)
>          libc10.so =>
> /stornext/System/data/nvidia/libtorch-gpu/libtorch-gpu-1.12.1/lib/libc10.so
> (0x00007f8cb1e2d000)
>          libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f8cb1c21000)
>          librt.so.1 => /lib64/librt.so.1 (0x00007f8cb1a19000)
>          libgcc_s.so.1 =>
> /stornext/System/data/apps/gcc/gcc-11.2.0/lib64/libgcc_s.so.1
> (0x00007f8cb1801000)
>          libdl.so.2 => /lib64/libdl.so.2 (0x00007f8cb15fd000)
>          libmkl_intel_lp64.so.2 => not found
>          libmkl_gnu_thread.so.2 => not found
>          libmkl_core.so.2 => not found
>          libm.so.6 => /lib64/libm.so.6 (0x00007f8cb12fb000)
>          libcudart.so.11.0 => not found
> 
> Then in R, if I try to load that same file:
> 
>>
> dyn.load("/stornext/System/data/nvidia/libtorch-gpu/libtorch-gpu-1.12.1/lib/libtorch_cpu.so",
> now=FALSE)
> Error in
> dyn.load("/stornext/System/data/nvidia/libtorch-gpu/libtorch-gpu-1.12.1/lib/libtorch_cpu.so",
>   :
>    unable to load shared object
> '/stornext/System/data/nvidia/libtorch-gpu/libtorch-gpu-1.12.1/lib/libtorch_cpu.so':
>    libmkl_intel_lp64.so.2: cannot open shared object file: No such file or
> directory
> 
> Is this a bug in the `dyn.load` implementation for R? If not, why is it
> behaving like this? What should I do about it?
> 
> For reference, I'm on CentOS 7, with Linux
> kernel 3.10.0-1160.81.1.el7.x86_64.
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.