[R] another type of parallel programming for R

[Norm Matloff]

On 2009-07-01, David M Smith <david at revolution-computing.com> wrote:

> REvolution Computing has just released three new packages for R to
> CRAN (under the open-source Apache 2.0 license): foreach, iterators,
> and doMC. Together, they provide a simple, scalable parallel computing
> framework for R that lets you take advantage of your multicore or
> multiprocessor workstation to program loops that run faster than
> traditional loops in R.

David gave an excellent talk on this at our Bay Area useR Group a couple
of months ago.  I was quite impressed, and believe users of snow etc.
will find the new software quite useful.

I mentioned at David's talk that it would be nice to have a
shared-memory programmer's view for R, to complement the mainly
message-passing paradigms of snow, Rmpi and now REvolution's new
packages.  Since that talk, I've been working on such a package, which I
call Rdsm ("R distributed shared memory"), modeled after a similar
package, PerlDSM, I wrote for Perl some years ago.  

Note that the word "distributed" here means that the memory is not
really shared, but instead is an abstraction, to give the programmer a
shared-memory view even though the program may be running on several
separate machines.  For C/C++ this is generally accomplished by
manipulation of the virtual memory hardware.  For R, I do this by
redefining functions such as "[" and "[<-" for a new class.

Rdsm is intended only as an alternative for those who favor the
shared-memory view of things.  In the parallel processing community,
there has always been a debate between advocates of the two main
programming paradigms, shared memory and message passing.  Shared memory
advocates claim greater clarity of code, while the message passing
people point to that paradigm's greater flexibility.  I happen to be of
the shared-memory school.  Given the popularity of OpenMP for
C/C++/FORTRAN, I believe Rdsm will be of interest to many for R.
Indeed, in the next few months, I will be extending Rdsm with functions
that give it the "look and feel" of OpenMP.

I have an an alpha version of Rdsm ready, and will be releasing it in
the next day or two (on my own Web page for now, not yet on CRAN) in an
announcement here.  User feedback will be much appreciated.

Norm Matloff
UC Davis