[R] Parallel processes collapse into one

[Nicola Lunardon]

Hi, thanks for the hints. I am quite sure that at some points R stalls as I
am used to monitor the progress of simulations (with a custom function
similar to pbmclapply) and save partial results: whenever all R instances
collapse into one, then no more updates show up.
I was able to set OPENBLAS_NUM_THREADS=1 (1=the value calculated by using
the formula you provided) as suggested here
https://stat.ethz.ch/pipermail/r-sig-debian/2016-August/002588.html, but it
seemed to have no effect on the machine I am using. However, I managed to
force R to use standard BLAS/LAPACK libraries, hope this is going to solve
the problem

sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

Il giorno gio 6 giu 2019 alle ore 13:20 Ivan Krylov <krylov.r00t using gmail.com>
ha scritto:

> On Mon, 3 Jun 2019 06:37:46 +0200
> Nicola Lunardon <nicola.lunardon.84 using gmail.com> wrote:
>
> > R seems to be running, but simulations do not progress further.
>
> Have you tried message() (or REprintf() in C code) to narrow down the
> specific part of the code where simulations stop in their progress?
> It's less convenient than a good debugger, but with parallel code it is
> sometimes the only way to reproduce the problem and get some
> information about it.
>
> > If I run simulations on 16 cores I end up having an R instance with
> > CPU usage about 1600%, never experienced such a behaviour.
>
> > BLAS:   /usr/lib/openblas-base/libblas.so.3
> > LAPACK: /usr/lib/libopenblasp-r0.2.18.so
>
> OpenBLAS can, indeed, use multiple threads on its own, inside a single
> process. Combined with mclapply, this might create a situation when
> there are far more threads competing for CPU time than CPU cores
> available. Does it help if you set an environment variable such as
> OPENBLAS_NUM_THREADS [*] to a number less than or equal to (number of
> CPU cores / mc.cores mclapply argument)?
>
> --
> Best regards,
> Ivan
>
> [*]
>
> https://github.com/xianyi/OpenBLAS#setting-the-number-of-threads-using-environment-variables
>

	[[alternative HTML version deleted]]