[R] inconsistency in mclapply.....

[Ivan Krylov]

On Sat, 10 Jun 2023 11:49:10 +0000
akshay kulkarni <akshay_e4 using hotmail.com> wrote:

> (gdb) backtrace
> #0  0x00007f94b6454053 in gomp_team_barrier_wait_end () from /lib64/libgomp.so.1
> #1  0x00007f94b6452a7d in gomp_team_end () from /lib64/libgomp.so.1
> #2  0x00007f94b3c280cf in exec_blas () from /lib64/libopenblaso.so.0
> #3  0x00007f94b3c263e4 in gemm_thread_n () from /lib64/libopenblaso.so.0
> #4  0x00007f94b3c39077 in dgetrs_N_parallel () from /lib64/libopenblaso.so.0
> #5  0x00007f94b39f7f65 in dgesv_ () from /lib64/libopenblaso.so.0
> #6  0x00007f948556aebe in La_solve () from /usr/lib64/R/modules//lapack.so
> #7  0x00007f948556e555 in mod_do_lapack () from /usr/lib64/R/modules//lapack.so
> #8  0x00007f94b67a9c93 in bcEval () from /usr/lib64/R/lib/libR.so

I'm afraid this is exactly OpenBLAS trying to use OpenMP and failing
because it currently breaks in programs that fork().

Related links are <https://github.com/xianyi/OpenBLAS/issues/294> and
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035>.

I was hoping that setting OPENBLAS_NUM_THREADS=1 would prevent OpenBLAS
from trying to use OpenMP, but that doesn't seem to work. If you
replace OPENBLAS_NUM_THREADS with OMP_NUM_THREADS (in order to limit
the number on OpenMP side), does it also not work?

How did you install OpenBLAS on this machine? If my environment
variable adjustments don't work, you'll need to either replace OpenBLAS
with a different build (for example, one that uses pthreads instead of
OpenMP, or a pure single-thread build, since you're already using
mclapply), or even remove OpenBLAS altogether and replace it with a
different BLAS that is fork()-safe.

-- 
Best regards,
Ivan