[R] Parallel runs of an external executable with snow in local

[Xavier Portell/UPC]

Hi everyone,

I'm aiming to run an external executable (say filetorun.EXE) in parallel. The external executable collect needed data from a file, say "input.txt" and, in turn,generates several output files, say "output.txt". I need to generate "input.txt", run the executable and keep "input.txt" and "output.txt". I'm using Windows 7, R version 2.15.1 (2012-06-22) on RStudio and platform: i386.pc.mingw32/i386 (32-bit).

My first attempt was a R code which, by using
  System("filetorun.EXE", intern = F, ignore.stdout = F,
          ignore.stderr = F, wait = T, input = NULL,
          show.output.on.console = T, minimized = F, invisible = T))
, ran the executable and kept required files to a conveniently named folder. After that I changed my previous R script so I could use the function lapply().This script apparently worked fine.
 
Finally, I tried to parallelize the problem by using snow and parLapply(). The resulting script looks like this:

## Not run
#
library(snow)cl <- makeCluster(3, type = "SOCK")
clusterExport(cl,list('param.esp','copy.files','for12.template','program.executor'))
parLapply(cl,a.list,a.function))stopCluster(cl)
#
##End not run

Although it runs, the parallelized version is messing up the input parameters to pass to the executable (see table below, where parameters P1 and P2 are considered. ".s" comes from the serial code and ".p" from the parallelized one):
  s r P1.s P2.s P1.p P2.p
1 1 1  1.0 3.00  2.0 3.00
2 2 1  1.5 3.00  2.0 3.75
3 3 1  2.0 3.00  2.0 3.00
4 4 1  1.0 3.75  1.5 3.00
5 5 1  1.5 3.75  1.5 3.00
6 6 1  2.0 3.75  2.0 3.75

My first thought to avoid the described behaviour was creating a temporary file, say "tmp.id" with id being an identification run number, and copying "filetorun.EXE" and "Input.txt" to "tmp.id". However, while doing so, I realised that although running the correct "filetorun.EXE" copy (i.e., the one in "tmp.id") R looks for "input.txt" in the work directory.


I've been looking thoroughly for a solution but I got nothing.  

Thanks for any help in advance,


Xavier Portell Canal

PhD candidate
Department of Agri-food engineering,
Universitat Politècnica de Catalunya