[R] R/parallel

[Prof Brian Ripley]

On Thu, 8 Dec 2011, Scott Raynaud wrote:

> Looks like this requires use of foreach and??lots of extra coding.??
> R/parallel only requires that the loop be bracketed with the code
> to start the parallel processing-about 4-5 lines.?? Seems??a lot??
> easier to me.?? If I were to go to the trouble of writing a lot of
> new code, it seems that recompiling with BLAS and pnmath would be
> a better option.

On Windows, that will be man-days of work unless you know the insides 
intimately (and no guarantees that it will eventually give a speed 
increase).  There is no support for parallel BLAS nor OpenMP nor 
pthreads in the current R sources/binaries for Windows.

> ??
> My main question is how to handle the random number generation
> when the child processes are spawned.?? That's a problem no matter
> what method I choose to create the threads.

A caveat: the R interpreter is not thread-safe: don't assume that you
can run R code in parallel threads.

I don't know what you mean by 'R/parallel'.  However, R has a 
'parallel' package, and its vignette discusses all this (including 
RNG).  Adding parallel support using package 'parallel' is simple (and 
well-documented), not least as it comes ready-to-roll with R.

If you meant the unfortunately named project at www.rparallel.org, a 
few comments:

1) It has not been updated in 3 years, and the pre-compiled Windows 
binaries are not going to work with recent R (like R >= 2.10.0).

2) It seems a lot less mature than the 'parallel' package and makes 
several restrictive assumptions.

3) Part of that lack of maturity is lack of documentation (including 
of the restrictive assumptions).

4) Inter-process communication seems to be by files.  That is going to 
be slow, especially on Windows.  Package 'parallel' uses sockets and 
pipes.


> ??
> From: Tal Galili <tal.galili at gmail.com>
> To: Scott Raynaud <scott.raynaud at yahoo.com>
> Cc: "r-help at r-project.org" <r-help at r-project.org>
> Sent: Thursday, December 8, 2011 12:38 PM
> Subject: Re: [R] R/parallel
>
>
> Hi Scott,
> Why not use the doSMP package from REvolution?
> http://www.r-statistics.com/2010/04/parallel-multicore-processing-with-r-on-windows/
>
>
> Tal
>
>
> ----------------Contact Details:-------------------------------------------------------
> Contact me: Tal.Galili at gmail.com |?? 972-52-7275845
> Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English)
> ----------------------------------------------------------------------------------------------
>
>
>
>
>
>
> te:
>
> I want to take advantage?? of my multicore CPU to speed up a loop in a simulation program.?? I didn???t write the code,
>> but the iterations appear independent to me, at least in the sense that the results of one loop do not depend on
>> previous ones.?? Right now I???m relegated to a Windows box that runs Windows 7.?? These appear to be the options:
>> ??
>> Pnmath-appears to parallelize non-BLAS routine but requires a special build
>> Fork-UNIX only
>> Romp-looks like this hasn???t advanced past the developmental stage
>> Multicore-use on Windows at your own risk
>> R/parallel-seems like the best option if I don???t want to recompile.
>> ??
>> Has anyone ever used R/parallel??? What kind of results did you have??? One difficulty with my simulation is that the
>> loop includes code to generate random numbers.?? If this loop is split into different threads, then I suspect the
>> randomness of the numbers is not assured.?? What can I do about that?
>> ??
>> I can provide the loop code, but it???s fairly long, say 75-100 lines.
>> ??
>> If R/parallel is not feasible then a recompile with BLAS and pnmath appears to be the next best option.
>>
> 	[[alternative HTML version deleted]]
>
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595