[BioC] High-performance Bioconductor experiments
rdiaz at cnio.es
Fri Dec 12 10:06:45 MET 2003
This might be irrelevant or already well known, and if so please disregard.
But I feel that several different issues are being discussed here. It is my
cursory understanding that altough paralelization (pvm) and openMosix can
coexist peacefully paralelization of R might not be a trivial issue; load
balancing, however, can be achieved using LVS (linux virtual server
---http://www.linux-vs.org/), so separate R processes could be started on
different CPUs, and then the result (the .RData?) combined, which might be
along the lines of what Greg suggested; LVS and openMosix also seem to get
In our cluster, and given the mixed results we have found with the migration
of R, we will probably use LVS (instead of relying on openMosix migrating R
processes). For Michael Benjamin's situation a possible kluge (which I have
not tried) would be to use LVS to run several R processes (i.e., as many
processes as disjunct subsets of cel files), and then combine the output.
On Friday 12 December 2003 04:13, Michael Benjamin wrote:
> This is the most exciting, and cryptic, part of the message...
> > which you can then normalize and scale. The normalizing and scaling
> > can, of course also be split up across processors.
> I was able to use snow to split up the CEL file readings--it's actually
> not that hard.
> Data<-clusterApply(cl, filenames,Readaffy)
> I find that pvm is not as easy to use as openMosix, because it doesn't
> autodiscover (or does it?!). My idea is to make a multinode cluster on
> the base computer using RPVM, then have openMosix farm out the processes
> instead of relying on pvm to do that.
> In other words, run RPVM on a single pvm node, multi-openMosix node.
> I'll try that experiment tomorrow.
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
Centro Nacional de Investigaciones Oncológicas (CNIO)
(Spanish National Cancer Center)
Melchor Fernández Almagro, 3
28029 Madrid (Spain)
PGP KeyID: 0xE89B3462
More information about the Bioconductor