[R] Assigning cores

[Peter Langfelder]

The big question is whether each worker or thread uses parallel
processing itself, or whether it uses resources like cache in which
case 20 threads fighting over the cache would slow you down
substantially. If your simulations use operations implemented in BLAS
or LAPACK, be aware that some R installations use custom fast BLAS
that can use multiple cores and the processor cache. You can see some
of it in sessionInfo().

The other issue is memory usage - if you exhaust your physical RAM,
your computer will slow down not so much because of CPU load but
rather because of memory management (swapping to and from disk).

I would run some smaller experimental runs that take just a minute or
two to finish with say 4, 8 and 12 workers and see how fast these go -
you may find no or very little speed up past 8 or perhaps even 4-6
workers.

HTH,

Peter

On Thu, Sep 3, 2020 at 10:45 AM Leslie Rutkowski
<leslie.rutkowski using gmail.com> wrote:
>
> Hi all,
>
> I'm working on a large simulation and I'm using the doParallel package to
> parallelize my work. I have 20 cores on my machine and would like to
> preserve some for day-to-day activities - word processing, sending emails,
> etc.
>
> I started by saving 1 core and it was clear that *everything* was so slow
> as to be nearly unusable.
>
> Any suggestions on how many cores to hold back (e.g., not to put to work on
> the parallel process)?
>
> Thanks,
> Leslie
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.