[R] inconsistency in mclapply.....

[Ivan Krylov]

On Sat, 10 Jun 2023 13:46:12 +0000
akshay kulkarni <akshay_e4 using hotmail.com> wrote:

>                    What if I switch to Ubuntu?
> I had a similar ( but not exactly the same) problem way back in 2018
> and switching to Ubuntu solved it.

You shouldn't have to switch distros for problems like this one.

What you need to switch is your OpenBLAS build: instead of using
OpenBLAS built with OpenMP support (which isn't even the default
when compiling OpenBLAS), use OpenBLAS built with pthreads support. It's
also likely, but not 100% guaranteed, that setting OMP_NUM_THREADS=1
will avoid the issue by never starting any additional threads for
OpenBLAS purposes. At least some documentation says that
OPENBLAS_NUM_THREADS is ignored on OpenMP builds of OpenBLAS, you
actually have to set OMP_NUM_THREADS=1 for it to take any effect.

I know very little about RHEL, but it seems that you already have a
pthreads build of OpenBLAS installed in
/usr/lib64/libopenblasp-r0.3.15.so; you just need to adjust the path to
the BLAS implementation used by R. If not, install the openblas-threads
package.

Can you use flexiblas
<https://cran.r-project.org/bin/linux/fedora/#blaslapack-switching> to
switch the BLAS implementation to OPENBLAS-THREADS? This should be
possible by using the flexiblas R package or by setting environment
variables
<https://github.com/mpimd-csc/flexiblas#selecting-the-backend-at-runtime>.

That said, I know for sure that on Debian and Ubuntu you get a choice
between libopenblas0-openmp / libopenblas0-pthread /
libopenblas0-serial, and installing any one of these packages except
libopenblas0-openmp should avoid the problem. (And the thread limit
should be set to 1 if you're running your tasks in parallel by other
means.)

-- 
Best regards,
Ivan