[Rd] does parLapplyLB do load-balancing?

Ei-ji Nakama nakama at ki.rim.or.jp
Sat Dec 6 07:22:24 CET 2014


In such cases, try the Rhpc package.

The following is the result of the benchmark.

# but tuning is not finished...

2014-12-06 10:36 GMT+09:00 Chris Paciorek <paciorek at stat.berkeley.edu>:
> Looking at parLapplyLB, one sees that it takes in X and then passes
> splitList(X, length(cl)) to clusterApplyLB, which then calls
> dynamicClusterApply.  Thus while dynamicClusterApply does handle tasks
> in a load-balancing fashion, sending out individual tasks as previous
> tasks complete, parLapplyLB preempts that by splitting up the tasks in
> advance into as many groups of tasks as there are cluster processes.
> This seems to defeat the purpose of load-balancing and of the manner
> in which dynamicClusterApply is coded.
> This question basically repeats a question posed in 2013 -- see
> http://r.789695.n4.nabble.com/parLapplyLB-Load-balancing-tt4671848.html
> I'm reposting because there doesn't seem to have been any response to
> the previous posting, and it looks like the issue is still present in
> R 3.1 so it seems asking again if this is the intended behavior of
> parLapplyLB.
> I'm using R 3.1.1 and the 3.1.1 version of the parallel package under
> Ubuntu 14.04, but the code appears to be the same in R-devel.
> Chris
> ----------------------------------------------------------------------------------------------
> Chris Paciorek
> Statistical Computing Consultant
> Statistical Computing Facility, Econometrics Laboratory, Berkeley
> Research Computing
> Office: 495 Evans Hall                      Email: paciorek at stat.berkeley.edu
> Mailing Address:                            Voice: 510-842-6670
> Department of Statistics                    Fax:   510-642-7892
> 367 Evans Hall                              Skype: cjpaciorek
> University of California, Berkeley          WWW:
> www.stat.berkeley.edu/~paciorek
> Berkeley, CA 94720 USA                      Permanent forward:
> paciorek at alumni.cmu.edu
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Best Regards,
Eiji NAKAMA <nakama (a) ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb"  <nakama (a) ki.rim.or.jp>

More information about the R-devel mailing list