[R] multi-machine parallel setup?

[Joshua Wiley]

On Sat, Apr 21, 2012 at 8:20 AM, ivo welch <ivo.welch at gmail.com> wrote:
> hi uwe---my box is different:
>
> cl <- makeCluster(c("localhost", "welch", "calc"), "SOCK")
> Error in loadNamespace(name) : there is no package called 'snow'
>
> this seems to be part of snow, not parallel.  ok, install snow first,
> then rerun.
>
> library(snow)
> Attaching package: 'snow'
>
> The following object(s) are masked from 'package:parallel':
>
>    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
> clusterExport, clusterMap, clusterSplit, makeCluster, parApply,
> parCapply, parLapply,
>    parRapply, parSapply, splitIndices, stopCluster
>
> bash: /Library/Frameworks/R.framework/Resources/bin/Rscript: No such
> file or directory
>
> but if I use two linux machines, it works.
>
> now, how do I use parallel's mclapply with it?

parLapply(cl = cl, X = data, fun = mean)

AFAIK, mclapply() does not take a cluster argument, but the par* functions do.

Cheers,

Josh

>
> best,
>
> /iaw
>
> ----
> Ivo Welch (ivo.welch at gmail.com)
> J. Fred Weston Professor of Finance
> Anderson School at UCLA, C519
> http://www.ivo-welch.info/
> Editor, Critical Finance Review, http://www.critical-finance-review.org/
>
>
>
> 2012/4/21 Uwe Ligges <ligges at statistik.tu-dortmund.de>:
>>
>>
>> On 21.04.2012 06:34, ivo welch wrote:
>>>
>>> the vignette to the library(parallel) mentions snow repeatedly (esp
>>> differences in its implementation in parallel from the original).
>>> unfortunately, it doesn't give an example or tutorial for
>>> multi-machine use with sockets.
>>>
>>> could someone please point me to a simple working example, where a
>>> master just has both itself and a 4-core slave on a second machine
>>> (IP), and wants to execute an mclapply on a list on both machines,
>>> like
>>>
>>>    master.ip<- "localhost"
>>>    slave.ip<- "Rfriend at 192.168.2.10"  ## has key ssh access
>>>    mclapply( 1:100, function(x) { cat(x); Sys.sleep(1); x } )  # on both,
>>> please
>>>
>>> iaw
>>
>>
>>
>>
>> With appropriate ssh keys that do not need passwords and with the same
>> username on all machines, it should work out of the box. At least, it does
>> for me:
>>
>> Say you want two, one on your own machine, the other one on your friends:
>>
>>  cl <- makeCluster(c("localhost", "192.168.2.10"), "SOCK")
>>
>> BEst,
>> Uwe Ligges
>>
>>
>>>
>>> On Wed, Apr 18, 2012 at 1:01 PM, ivo welch<ivo.welch at gmail.com>  wrote:
>>>>
>>>> Dear R experts:
>>>>
>>>> could someone please point me to a page that explains how to set up
>>>> more than 1 machine for library parallel (which is quickly becoming my
>>>> favorite!)
>>>
>>> ...
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/