[BioC] snow library, question on clusterExport

[mattia pelizzola]

thanks Martin for the explanations and thanks Norman for pointing out
that error in the example,

unfortunately I am still stuck with the main problem:
I have to use clusterExport to export an object to the cluster nodes.
clusterExport only seems to export objects from the GlobalEnv,
unfortunately. In my case this object is created within a function and
clusterExport is called within the same function, so the object is not
available in the GlobalEnv and I get error ..

I'll try writing to the other mailing list,
thanks

mattia

On Sun, Apr 11, 2010 at 8:49 AM, Pavelka, Norman <NXP at stowers.org> wrote:
> Hi Mattia,
>
> Maybe I'm not getting what you're trying to do, but shouldn't your fun3 be using object 'data' rather than 'Mat' internally?
>
> HTH ;-)
> Norman
>
> #  Hi,
> #
> #  I have a simple function:
> #
> #  > library(snow)
> #  > fun2=function() {
> #  + cl=makeCluster(3)
> #  + Mat=matrix(2:10,3,3)
> #  + fun3=function(startInd, endInd=3, data=Mat) {Mat[startInd:endInd,]}
> #  + print(clusterApplyLB(cl, 1:3, fun3))
> #  + stopCluster(cl)
> #  + }
> #
> #  that is working fine:
> #
> #  > fun2()
> #  [[1]]
> #       [,1] [,2] [,3]
> #  [1,]    2    5    8
> #  [2,]    3    6    9
> #  [3,]    4    7   10
> #
> #  [[2]]
> #       [,1] [,2] [,3]
> #  [1,]    3    6    9
> #  [2,]    4    7   10
> #
> #  [[3]]
> #  [1]  4  7 10
> #
> #  now, if I run the same commands outside the function:
> #
> #  > cl=makeCluster(3)
> #  > Mat=matrix(2:10,3,3)
> #  > fun3=function(startInd, endInd=3, data=Mat) {Mat[startInd:endInd,]}
> #  > print(clusterApplyLB(cl, 1:3, fun3))
> #  Error in checkForRemoteErrors(val) :
> #    3 nodes produced errors; first error: object 'Mat' not found
> #
> #  so I figured out I have to export 'Mat' on the cluster nodes:
> #
> #  > clusterExport(cl, 'Mat')
> #  > print(clusterApplyLB(cl, 1:3, fun3))
> #  [[1]]
> #       [,1] [,2] [,3]
> #  [1,]    2    5    8
> #  [2,]    3    6    9
> #  [3,]    4    7   10
> #
> #  [[2]]
> #       [,1] [,2] [,3]
> #  [1,]    3    6    9
> #  [2,]    4    7   10
> #
> #  [[3]]
> #  [1]  4  7 10
> #
> #  I still do not understand why clusterExport is NOT necessary within
> #  the function 'fun2' and actually it would give an error:
> #
> #  > rm(Mat)
> #  > fun2=function() {
> #  + cl=makeCluster(3)
> #  + Mat=matrix(2:10,3,3)
> #  + clusterExport(cl, 'Mat')
> #  + fun3=function(startInd, endInd=3, data=Mat) {Mat[startInd:endInd,]}
> #  + print(clusterApplyLB(cl, 1:3, fun3))
> #  + stopCluster(cl)
> #  + }
> #  > fun2()
> #  Error in get(name, env = .GlobalEnv) : object 'Mat' not found
> #
> #
> #  I found clusterExport to be the solution for a more complex example,
> #  can I can't make it working within a function.
> #  What is it happening here with clusterExport? and how can I export an
> #  object that is not on my globalEnv but rather is created within a
> #  function?
> #
> #  many thanks!
> #
> #  mattia
> #
> #  > sessionInfo()
> #  R version 2.10.1 (2009-12-14)
> #  x86_64-unknown-linux-gnu
> #
> #  locale:
> #   [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
> #   [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
> #   [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
> #   [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
> #   [9] LC_ADDRESS=C               LC_TELEPHONE=C
> #  [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> #
> #  attached base packages:
> #  [1] stats     graphics  grDevices utils     datasets  methods   base
> #
> #  other attached packages:
> #  [1] snow_0.3-3