[R] parSapply can't find function

[Duncan Murdoch]

On 29/04/2013 2:16 PM, Kaiyin Zhong (Victor Chung) wrote:
> Hi, Uwe.
>
> I still don't get how this can be done correctly. Here is what I tried.
>
> In the file funcs.R, define these functions:
>
> library('modeest')
> x = vector(length=500)
> x = sapply(x, function(i) i=sample(c(1,0), 1))
> pastK = function(n, x, k) {
>      if (n>k) { return(x[(n-k):(n-1)]) }
>      else {return(NA)}
> }
> predR = function(x, k) {
>      pastList = lapply(1:length(x), function(n) pastK(n, x, k))
>      pred = sapply(pastList, function(v) mfv(v)[1])
>      ratio = sum(pred==x, na.rm=T)/(length(pred) - sum(is.na(pred)))
> }
>
>
>
> Then do the following:
>
> library('snow')
> cl = makeCluster(rep('localhost', 12), 'SOCK')
> clusterSetupRNG(cl)
> clusterEvalQ(cl, 'source("funcs.R")')

The expression being evaluated there is a string,

'source("funcs.R")'

You want an expression, e.g.

clusterEvalQ(cl, source("funcs.R"))

Duncan Murdoch


> testK = function() {
>      k = seq(3, 25, 2)
>      r = parSapply(cl, k, function(i) predR(x, i))
>      print(r)
> }
> testK()
> stopCluster(cl)
>
>
> The error still pops up:
>
> Error in checkForRemoteErrors(val) :
>    12 nodes produced errors; first error: could not find function "predR"
>
>
>
>
>
> Best regards,
>
> Kaiyin ZHONG
> ------------------------------
> FMB, Erasmus MC
> k.zhong at erasmusmc.nl
> kindlychung at gmail.com
>
>
> On Tue, Apr 23, 2013 at 3:44 PM, Uwe Ligges <ligges at statistik.tu-dortmund.de
> > wrote:
>
> >
> >
> > On 23.04.2013 15:00, Kaiyin Zhong (Victor Chung) wrote:
> >
> >> Thanks for the reply.
> >>
> >> How can i make the functions known to all nodes?
> >>
> >
> > See ?clusterEvalQ
> >
> > you may also want to try the parallel packages.
> >
> > Best,
> > Uwe Ligges
> >
> >
> >
> >
> >> Best regards,
> >>
> >> Kaiyin ZHONG
> >> ------------------------------**------------------------------**
> >> ------------
> >> FMB, Erasmus MC
> >> k.zhong at erasmusmc.nl <mailto:k.zhong at erasmusmc.nl>
> >> kindlychung at gmail.com <mailto:kindlychung at gmail.com>
> >>
> >>
> >>
> >> On Tue, Apr 23, 2013 at 2:43 PM, Uwe Ligges
> >> <ligges at statistik.tu-dortmund.**de <ligges at statistik.tu-dortmund.de>
> >> <mailto:ligges at statistik.tu-**dortmund.de<ligges at statistik.tu-dortmund.de>>>
> >> wrote:
> >>
> >>
> >>
> >>     On 18.04.2013 11:11, Kaiyin Zhong (Victor Chung) wrote:
> >>
> >>         Here is the code, assuming 8 cores in the cpu.
> >>
> >>         library('modeest')
> >>         library('snow')
> >>
> >>         cl = makeCluster(rep('localhost', 8), 'SOCK')
> >>         x = vector(length=50)
> >>         x = sapply(x, function(i) i=sample(c(1,0), 1))
> >>
> >>         pastK = function(n, x, k) {
> >>               if (n>k) { return(x[(n-k):(n-1)]) }
> >>               else {return(NA)}
> >>         }
> >>
> >>         predR = function(x, k) {
> >>               pastList = lapply(1:length(x), function(n) pastK(n, x, k))
> >>               pred = sapply(pastList, function(v) mfv(v)[1])
> >>               ratio = sum(pred==x, na.rm=T)/(length(pred) - sum(is.na
> >>         <http://is.na>(pred)))
> >>
> >>         }
> >>
> >>         testK = function() {
> >>               k = seq(3, 25, 2)
> >>               r = parSapply(cl, k, function(i) predR(x, i))
> >>         #    r = sapply(k, function(i) predR(x, i))
> >>         }
> >>
> >>         r = testK()
> >>         stopCluster(cl)
> >>
> >>         Here is the error:
> >>         Error in checkForRemoteErrors(val) :
> >>             8 nodes produced errors; first error: could not find
> >>         function "predR"
> >>
> >>
> >>
> >>     predR is not yet known on all nodes, just on the master. You have to
> >>     tell the nodes about the definition first.
> >>
> >>     Best,
> >>     Uwe Ligges
> >>
> >>
> >>
> >>
> >>
> >>
> >>         Best regards,
> >>
> >>         Kaiyin ZHONG
> >>         ------------------------------
> >>
> >>         FMB, Erasmus MC
> >>         k.zhong at erasmusmc.nl <mailto:k.zhong at erasmusmc.nl>
> >>         kindlychung at gmail.com <mailto:kindlychung at gmail.com>
> >>
> >>                  [[alternative HTML version deleted]]
> >>
> >>         ______________________________**__________________
> >>         R-help at r-project.org <mailto:R-help at r-project.org> mailing list
> >>         https://stat.ethz.ch/mailman/_**_listinfo/r-help<https://stat.ethz.ch/mailman/__listinfo/r-help>
> >>
> >>         <https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
> >> >
> >>         PLEASE do read the posting guide
> >>         http://www.R-project.org/__**posting-guide.html<http://www.R-project.org/__posting-guide.html>
> >>
> >>         <http://www.R-project.org/**posting-guide.html<http://www.R-project.org/posting-guide.html>
> >> >
> >>         and provide commented, minimal, self-contained, reproducible code.
> >>
> >>
> >>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.