[R] Problem with R on dual core under Linux - can not execute mpi.spawn.Rslaves()

Luke Tierney luke at stat.uiowa.edu
Mon Sep 29 15:05:23 CEST 2008


Look at the configure output below: configure is finding mpi.h in
/usr/lib/mpich/include, soit is using mpich, not lam.  You may be able
to tell Rmpi specifically where to look for LAM stuff or you may need
to uninstall mpich (almost surely that can be avoided but it may be
easier if you don't really need mpich).

Best,

luke

On Mon, 29 Sep 2008, Rainer M Krug wrote:

> On Mon, Sep 29, 2008 at 12:19 AM, Martin Morgan <mtmorgan at fhcrc.org> wrote:
>> "Rainer M Krug" <r.m.krug at gmail.com> writes:
>>
>>> Hi
>>>
>>> I am trying to utilize my dual core processor (and later a
>>> High-performance clusters (HPC) ) by using the Rmpi, snow, snowfall,
>>> ... packages, but I am struggling at the beginning, i.e. to initialise
>>> the "cluster" on my dual core computer. Whenever I try to initialize
>>> it (via sfInit(parallel=TRUE, cpus=2) or mpi.spawn.Rslaves(nslaves=2)
>>> ), I get an error message:
>>>
>>>> sfInit(parallel=TRUE, cpus=2)
>>> Forced parallel. Using session: XXXXXXXXR_rkrug_143706_092708
>>>
>>> Error in mpi.comm.spawn(slave = mpitask, slavearg = args, nslaves = count,  :
>>>   MPI_Comm_spawn is not supported.
>>> Error in sfInit(parallel = TRUE, cpus = 2) :
>>>   Starting of snow cluster failed! Error in mpi.comm.spawn(slave =
>>> mpitask, slavearg = args, nslaves = count,  :
>>>   MPI_Comm_spawn is not supported.
>>>
>>> and
>>>
>>>> mpi.spawn.Rslaves(nslaves=2)
>>> Error in mpi.spawn.Rslaves(nslaves = 2) :
>>>   You cannot use MPI_Comm_spawn API
>>
>> This error comes from Rmpi,
>>
>>> head(mpi.spawn.Rslaves, 6)
>>
>> 1 function (Rscript = system.file("slavedaemon.R", package = "Rmpi"),
>> 2     nslaves = mpi.universe.size(), root = 0, intercomm = 2, comm = 1,
>> 3     hosts = NULL, needlog = TRUE, mapdrive = TRUE)
>> 4 {
>> 5     if (!is.loaded("mpi_comm_spawn"))
>> 6         stop("You cannot use MPI_Comm_spawn API")
>>
>> and occurs when the compiler variable MPI2 is undefined when your
>
>> package is installed. Likely this means that your mpi installation is
>> either old (unlikely?) or that your Rmpi installation failed to
>> properly detect the installed mpi version. It's difficult to know
>> which, without more information on how the mpi and Rmpi installations
>> went, at a minimum the result of R's sessionInfo() command and mpirun
>> --version but likely including the output of Rmpi's installation.
>
> Version:
> lamboot -V
>
> LAM 7.1.2/MPI 2 C++/ROMIO - Indiana University
>
>        Arch:           i486-pc-linux-gnu
>        Prefix:         /usr/lib/lam
>        Configured by:  buildd
>        Configured on:  Sun Mar 23 08:07:16 UTC 2008
>        Configure host: rothera
>        SSI rpi:        crtcp lamd sysv tcp usysv
>
> Below find the session info and the info from the instalaltion of Rmpi.
>
> In the install log from Rmpi, it says:
> checking whether MPICH2 is declared... no
> checking whether MPICH2 is declared... (cached) no
>
> So what went wrong? Do I have to start the "cluster" on the dual-core machine?
> Can I set the compiler variable MPI2 manually?
>
>> version
>               _
> platform       i486-pc-linux-gnu
> arch           i486
> os             linux-gnu
> system         i486, linux-gnu
> status
> major          2
> minor          7.2
> year           2008
> month          08
> day            25
> svn rev        46428
> language       R
> version.string R version 2.7.2 (2008-08-25)
>>
>> sessionInfo()
> R version 2.7.2 (2008-08-25)
> i486-pc-linux-gnu
>
> locale:
> LC_CTYPE=en_ZA.UTF-8;LC_NUMERIC=C;LC_TIME=en_ZA.UTF-8;LC_COLLATE=en_ZA.UTF-8;LC_MONETARY=C;LC_MESSAGES=en_ZA.UTF-8;LC_PAPER=en_ZA.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_ZA.UTF-8;LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] snow_0.3-3    Rmpi_0.5-5    snowfall_1.53
>
> loaded via a namespace (and not attached):
> [1] tools_2.7.2
>>
>> install.packages(c("Rmpi"), dep=TRUE, repo="http://cbio.uct.ac.za/CRAN/")
> Warning in install.packages(c("Rmpi"), dep = TRUE, repo =
> "http://cbio.uct.ac.za/CRAN/") :
>  argument 'lib' is missing: using '/usr/local/lib/R/site-library'
> trying URL 'http://cbio.uct.ac.za/CRAN/src/contrib/Rmpi_0.5-5.tar.gz'
> Content type 'application/x-gzip' length 94643 bytes (92 Kb)
> opened URL
> ==================================================
> downloaded 92 Kb
>
> * Installing *source* package 'Rmpi' ...
> checking for gcc... gcc
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc accepts -g... yes
> checking for gcc option to accept ISO C89... none needed
> I am here /usr/lib/mpich and it is MPICH
> Try to find mpi.h ...
> Found in /usr/lib/mpich/include
> Try to find libmpi.so or libmpich.a
> Found libmpich in /usr/lib/mpich/lib
> ##########################################
> checking whether MPICH2 is declared... no
> checking whether MPICH2 is declared... (cached) no
> ##########################################
> checking for openpty in -lutil... yes
> checking for main in -lpthread... yes
> configure: creating ./config.status
> config.status: creating src/Makevars
> ** libs
> gcc -std=gnu99 -I/usr/share/R/include -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -DHAVE_DECL_MPICH2=0 -DHAVE_DECL_MPICH2=0
> -I/usr/lib/mpich/include  -DMPICH -fPIC     -fpic  -g -O2 -c
> conversion.c -o conversion.o
> gcc -std=gnu99 -I/usr/share/R/include -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -DHAVE_DECL_MPICH2=0 -DHAVE_DECL_MPICH2=0
> -I/usr/lib/mpich/include  -DMPICH -fPIC     -fpic  -g -O2 -c
> internal.c -o internal.o
> gcc -std=gnu99 -I/usr/share/R/include -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -DHAVE_DECL_MPICH2=0 -DHAVE_DECL_MPICH2=0
> -I/usr/lib/mpich/include  -DMPICH -fPIC     -fpic  -g -O2 -c
> RegQuery.c -o RegQuery.o
> gcc -std=gnu99 -I/usr/share/R/include -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -DHAVE_DECL_MPICH2=0 -DHAVE_DECL_MPICH2=0
> -I/usr/lib/mpich/include  -DMPICH -fPIC     -fpic  -g -O2 -c Rmpi.c -o
> Rmpi.o
> gcc -std=gnu99 -shared  -o Rmpi.so conversion.o internal.o RegQuery.o
> Rmpi.o -L/usr/lib/mpich/lib -lmpich -lutil -lpthread -fPIC
> -L/usr/lib/R/lib -lR
> ** R
> ** demo
> ** inst
> ** preparing package for lazy loading
> ** help
> >>> Building/Updating help pages for package 'Rmpi'
>     Formats: text html latex example
>  hosts                             text    html    latex
>  internal                          text    html    latex
>  mpi.abort                         text    html    latex
>  mpi.apply                         text    html    latex   example
>  mpi.barrier                       text    html    latex
>  mpi.bcast                         text    html    latex
>  mpi.bcast.Robj                    text    html    latex
>  mpi.bcast.cmd                     text    html    latex
>  mpi.cart.coords                   text    html    latex   example
>  mpi.cart.create                   text    html    latex   example
>  mpi.cart.get                      text    html    latex   example
>  mpi.cart.rank                     text    html    latex   example
>  mpi.cart.shift                    text    html    latex   example
>  mpi.cartdim.get                   text    html    latex   example
>  mpi.comm                          text    html    latex   example
>  mpi.comm.disconnect               text    html    latex
>  mpi.comm.free                     text    html    latex
>  mpi.comm.inter                    text    html    latex
>  mpi.comm.set.errhandler           text    html    latex
>  mpi.comm.spawn                    text    html    latex
>  mpi.const                         text    html    latex
>  mpi.dims.create                   text    html    latex   example
>  mpi.exit                          text    html    latex
>  mpi.finalize                      text    html    latex
>  mpi.gather                        text    html    latex   example
>  mpi.gather.Robj                   text    html    latex   example
>  mpi.get.count                     text    html    latex
>  mpi.get.processor.name            text    html    latex
>  mpi.get.sourcetag                 text    html    latex
>  mpi.info                          text    html    latex
>  mpi.init.sprng                    text    html    latex
>  mpi.intercomm.merge               text    html    latex
>  mpi.parSim                        text    html    latex
>  mpi.parapply                      text    html    latex   example
>  mpi.probe                         text    html    latex
>  mpi.realloc                       text    html    latex
>  mpi.reduce                        text    html    latex
>  mpi.remote.exec                   text    html    latex   example
>  mpi.scatter                       text    html    latex   example
>  mpi.scatter.Robj                  text    html    latex   example
>  mpi.send                          text    html    latex   example
>  mpi.send.Robj                     text    html    latex
>  mpi.sendrecv                      text    html    latex   example
>  mpi.setup.rng                     text    html    latex
>  mpi.spawn.Rslaves                 text    html    latex   example
>  mpi.universe.size                 text    html    latex
>  mpi.wait                          text    html    latex
> ** building package indices ...
> * DONE (Rmpi)
>
> The downloaded packages are in
>        /tmp/RtmpdX6zlZ/downloaded_packages
>>
>
>>
>> Dirk Eddelbuettel
>> (https://stat.ethz.ch/pipermail/r-devel/2008-September/050665.html)
>> suggested that snow's makeSOCKcluster is an easier starting point for
>> single computer 'clusters' or other configurations where significant
>> system administration is not desired -- these should work without
>> additional software on most systems, even if more limiting in the long
>> term (in my opinion). See, e.g., the examples on the help page for
>> clusterApply for basic operation. A bit oddly, apparently the
>> 'snowfall' package restricts snow's functionality to mpi clusters. So
>> you might start with snow directly.
>>
>> Martin
>>
>>>
>>> I followed the pdf "Developing parallel programs using snowfall" by
>>> Jochen Knaus,. installed the relevant libraries and programs, but it
>>> does not work.
>>>
>>> I am stuck.
>>>
>>> Any help appreciated,
>>>
>>> Rainer
>>>
>>>
>>> --
>>> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
>>> Biology, UCT), Dipl. Phys. (Germany)
>>>
>>> Centre of Excellence for Invasion Biology
>>> Faculty of Science
>>> Natural Sciences Building
>>> Private Bag X1
>>> University of Stellenbosch
>>> Matieland 7602
>>> South Africa
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>> --
>> Martin Morgan
>> Computational Biology / Fred Hutchinson Cancer Research Center
>> 1100 Fairview Ave. N.
>> PO Box 19024 Seattle, WA 98109
>>
>> Location: Arnold Building M2 B169
>> Phone: (206) 667-2793
>>
>
>
>
>

-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu



More information about the R-help mailing list