[R] Multicore computation in Windows network: How to set up Rmpi
samu.mantyniemi at helsinki.fi
Thu Dec 20 14:54:47 CET 2007
My question is related to earlier posts about benefits of quadcore over
dualcore computers; I am trying to setup a cluster of windows xp
computers so that eventually I could make use of 10-20 cpu:s, but for
learning how to do this, I am playing around with two laptops.
I thought that the package snow would come handy in this situation, but
to use snow, I would probably need to install package Rmpi first. I
might also like to use Rmpi directly.
In order to use Rmpi in windows, I need to install MPI middle-ware, for
which I have found two options:
First I tried MPICH2 1.06 + R-2.6.0 + Rmpi 0.5-5. (I downloaded windows
binaries of Rmpi from Rmpi website
With MPICH2 I managed to connect my computers so that I was able to
remotely launch Rgui on both machines but R hanged when calling
"library(Rmpi)". If only one Rgui was launched on the localhost,
"library(Rmpi)" worked without errors, but trying to use
"mpi.spawn.Rslaves()" resulted in an error message, and so did
"mpi.universe.size()". (In my current setup I can not reproduce this
error message, but I can go back to this setup if this seems to be an
important piece of information)
After that I removed MPICH2 from the system and installed DeinoMPI
instead, thus my setup was DeinoMPI 1.1.0 + R-2.6.0 + Rmpi 0.5-6.
Using this setup on a single machine seems to work,
"mpi.universe.size()" returns the correct number of cpu cores and
"mpi.spawn.Rslaves()" creates one master and one slave on the local
unicore computer. However, trying to use two computers results in
similar behavior as with MPICH2: Rgui gets started on both, but R hangs
when trying "library(Rmpi)".
Following the advice given at the Rmpi website, this behavior could be
due to firewall settings. However, the result is the same if I take down
all my firewalls. Trying to debug this, I tried to run the example MPI
programs provided with the DeinoMPI installation. The example programs
work as expected: both machines participate to parallel computation and
the result is shown on the master node. This makes me believe that the
problem is likely related to R and Rmpi configuration, or the settings
I used when launching Rgui using mpiexec:
Settings on Mpiexec- tab:
Application: "C:\Program Files\R\R-2.6.0\bin\Rgui.exe"
Number of processes: 2
hosts: "akva26 samu" These are the computers with DeinoMPI installed.
DeinoMPI cluster-tab shows that they are ready to accept MPI jobs.)
Other options were just empty.
I am sure that someone has tried these before, and I was hoping to find
such users from this mailing list. Could you kindly share your
experiences about this issue?
For example, does anyone have a working setup with DeinoMPI? According
to Rmpi website DeinoMPI is the easiest way to set up MPI for a single
windows machine, but I am not sure whether this is also intended to mean
that one can not expect it to work with multiple computers.
Fisheries and Environmental Management Group (FEM)
Department of Biological and Environmental Sciences
Biocenter 3, room 4414
P.O. Box 65
FIN-00014 University of Helsinki
Phone: +358 9 191 58710
Fax: +358 9 191 58257
email: samu.mantyniemi helsinki.fi
personal webpage: http://www.helsinki.fi/people/samu.mantyniemi/
FEM webpage: http://www.helsinki.fi/science/fem/
More information about the R-help