[R] R intermittently crashes across cluster
R. Michael Weylandt
michael.weylandt at gmail.com
Thu Feb 7 22:21:44 CET 2013
On Thu, Feb 7, 2013 at 2:07 PM, Ken Weiss <kgweiss at umich.edu> wrote:
> I am having an interesting problem and I wonder if anyone else has
> seen this behavior.
> I am running R 2.11.1 with SNOW 0.3-3 on a Dell cluster running CentOS 5.5.
Simplest suggestion -- can you update to current R and use the inbuilt
parallelism facilities there? It's not improbable that your problem
has already been dealt with.
> I create my cluster using:
> cluster<- makeCluster(nodes,type="SOCK",port=10191) # nodes is a
> vector of compute nodes
> I then wrap a loop around clusterApplyLB to evaluate my function
> multiple times, with different parameters, without recreating the
> cluster every time. The following code segment shows what I am trying to do:
> for (j in loopstart:loopend)
> dat=x.m[,c(1:7, j)] # x.m is data from a csv file
> read into a table
> The problem that I am having is that sometimes it will run for 50
> iterations of this loop then crash. Sometimes 15 iterations,
> sometimes 2. When the crash happens, I receive the following error
> message every time:
> Error in checkForRemoteErrors(val) :
> one node produced an error: cannot open the connection
> Calls: clusterApplyLB -> dynamicClusterApply -> checkForRemoteErrors
> Execution halted
> Any ideas as to what might be going on? I have run this code
> successfully many times when I do not use the loop. I have a lot of
> data to process and recreating the cluster every time that I want to
> run my function is a waste of time.
> "For I know the plans I have for you," declares the LORD, "plans to
> prosper you and not to harm you, plans to give you hope and a
> future." - Jeremiah 29:11
> Check out the website http://www.amazingfacts.org/ for answers to
> Bible questions that will change your life.
> [[alternative HTML version deleted]]
> R-help at r-project.org mailing list
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help