[Rd] Parallel computing: how to transmit multiple parameters to a function in parLapply?

[William Dunlap]

You can put the function of interest and any global
variables it needs into a private environment, which gets sent
along with the function to the child processes.  E.g.

library(parallel)
cl3 <- makeCluster(3)
y <- c(1,100,10000)
addY <- function(x) x + y
withGlobals <- function(FUN, ...){
    environment(FUN) <- list2env(list(...))
    FUN
}
parLapply(cl3, 1:4, withGlobals(addY, y=y))
# [[1]]
# [1]     2   101 10001
# 
# [[2]]
# [1]     3   102 10002
# ...

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


> -----Original Message-----
> From: r-devel-bounces at r-project.org [mailto:r-devel-bounces at r-project.org] On Behalf
> Of Yu Wan
> Sent: Monday, December 23, 2013 10:16 PM
> To: r-devel at r-project.org
> Subject: [Rd] Parallel computing: how to transmit multiple parameters to a function in
> parLapply?
> 
> Hi R-developers
> 
> In the package Parallel, the function parLapply(cl, x, f) seems to allow
> transmission of only one parameter (x) to the function f. Hence in order to
> compute f(x, y) parallelly, I had to define f(x, y) as f(x) and tried to
> access y within the function, whereas y was defined outside of f(x).
> 
> Script:
> 
> library(parallel)
> 
> f <- function(x) {
>   z <- 2 * x + .GlobalEnv$y  # Try to access y in the global scope.
>   return(z)
> }
> 
> np <- detectCores(logical = FALSE)  # Two cores of my laptop
> x <- seq(1, 10, by = 1)
> y <- 0.5  # Y may be an array in reality.
> cl <- makeCluster(np)  # initiate the cluster
>   r <- parLapply(cl, x, f)  # apply f to x for parallel computing
> stopCluster(cl)
> 
> The r was a list with 10 empty elements which means f failed to access y.
> 
> Then I tested f without parallel computing:
> z <- f(x)
> print(z)
> [1]  2.5  4.5  6.5  8.5 10.5 12.5 14.5 16.5 18.5 20.5
> 
> The results indicates that we can access y using .GlobalEnv$y in a function
> without parLapply.
> 
> The question is, is there any method for me to transmit y to f, or access y
> within f during parallel computing?
> 
> The version of my R is 3.0.1 and I am running it on a Win8-64x system.
> 
> Thanks,
> 
> Yu
> 
> 
> 
> --
> View this message in context: http://r.789695.n4.nabble.com/Parallel-computing-how-
> to-transmit-multiple-parameters-to-a-function-in-parLapply-tp4682667.html
> Sent from the R devel mailing list archive at Nabble.com.
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel