[R] Parallel processes collapse into one
n|co|@@|un@rdon@84 @end|ng |rom gm@||@com
Tue Jun 11 07:34:55 CEST 2019
Hi, thanks for the hints. I am quite sure that at some points R stalls as I
am used to monitor the progress of simulations (with a custom function
similar to pbmclapply) and save partial results: whenever all R instances
collapse into one, then no more updates show up.
I was able to set OPENBLAS_NUM_THREADS=1 (1=the value calculated by using
the formula you provided) as suggested here
https://stat.ethz.ch/pipermail/r-sig-debian/2016-August/002588.html, but it
seemed to have no effect on the machine I am using. However, I managed to
force R to use standard BLAS/LAPACK libraries, hope this is going to solve
R version 3.6.0 (2019-04-26)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS
Matrix products: default
Il giorno gio 6 giu 2019 alle ore 13:20 Ivan Krylov <krylov.r00t using gmail.com>
> On Mon, 3 Jun 2019 06:37:46 +0200
> Nicola Lunardon <nicola.lunardon.84 using gmail.com> wrote:
> > R seems to be running, but simulations do not progress further.
> Have you tried message() (or REprintf() in C code) to narrow down the
> specific part of the code where simulations stop in their progress?
> It's less convenient than a good debugger, but with parallel code it is
> sometimes the only way to reproduce the problem and get some
> information about it.
> > If I run simulations on 16 cores I end up having an R instance with
> > CPU usage about 1600%, never experienced such a behaviour.
> > BLAS: /usr/lib/openblas-base/libblas.so.3
> > LAPACK: /usr/lib/libopenblasp-r0.2.18.so
> OpenBLAS can, indeed, use multiple threads on its own, inside a single
> process. Combined with mclapply, this might create a situation when
> there are far more threads competing for CPU time than CPU cores
> available. Does it help if you set an environment variable such as
> OPENBLAS_NUM_THREADS [*] to a number less than or equal to (number of
> CPU cores / mc.cores mclapply argument)?
> Best regards,
[[alternative HTML version deleted]]
More information about the R-help