[R] error serialize (foreach)

[Doran, Harold]

I have a portion of a foreach loop that I cannot run as parallel but works fine when serialized. Below is a representation of the problem as in this instance I cannot provide reproducible data to generate the same error, the actual data I am working with are confidential.

Within each foreach loop are a series of custom functions acting on my data. When using %do% I get expected result but replacing it with %dopar% generates the error.

I have searched archives and also stackexchange and see this is an issue that arises and I have tried a couple of the recommendations, like trying to use an outfile in makeCluster. But I am not having success.

Oddly, (or perhaps not oddly), others portions of my program run in parallel and do not generate this same error

library(foreach)
library(doParallel)
registerDoParallel(cores=3)

# This portion runs and produces expected result
result <- foreach(i = 1:N) %do% {
tmp1 <- function1(...)
tmp2 <- function2(...)
tmp2
}

# This portion generates error in serialize
result <- foreach(i = 1:N) %dopar% {
tmp1 <- function1(...)
tmp2 <- function2(...)
tmp2
}

error in serialize(data, node$con) : error writing to connection


	[[alternative HTML version deleted]]