[R] combining data frames in a list - how do I add breaks?
Euan Reavie
euan.reavie at gmail.com
Mon Feb 1 01:25:04 CET 2010
With Jim's help, the solution is as follows...
-----Original Message-----
From: jim holtman [mailto:jholtman * at sign * gmail.com]
Sent: Sunday, January 31, 2010 5:41 PM
To: Euan Reavie
Subject: Re: [R] combining data frames in a list - how do I add breaks?
Your 'combined' was a 'list of a list'. I think the object returned by:
performance(fit)
was actually a list and you probably should have done:
combined[[i]] <- performance(fit)$object
Here is what I did to get your data combined:
> str(combined)
List of 2
$ :List of 1
..$ object: num [1:4, 1:4] 0.394 0.454 0.352 0.392 0.754 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:4] "WA.inv" "WA.cla" "WA.inv.tol" "WA.cla.tol"
.. .. ..$ : chr [1:4] "RMSE" "R2" "Avg.Bias" "Max.Bias"
$ :List of 1
..$ object: num [1:4, 1:4] 0.324 0.364 0.319 0.357 0.793 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:4] "WA.inv" "WA.cla" "WA.inv.tol" "WA.cla.tol"
.. .. ..$ : chr [1:4] "RMSE" "R2" "Avg.Bias" "Max.Bias"
> do.call(rbind, lapply(combined, '[[', 1))
RMSE R2 Avg.Bias Max.Bias
WA.inv 0.3944042 0.7542192 -1.421083e-15 -0.3926396
WA.cla 0.4541431 0.7542192 1.421091e-15 -0.1854335
WA.inv.tol 0.3517808 0.8044719 0.000000e+00 -0.3588896
WA.cla.tol 0.3922082 0.8044719 -6.394885e-15 -0.2735517
WA.inv 0.3242771 0.7930415 0.000000e+00 0.6145005
WA.cla 0.3641399 0.7930415 -7.105536e-16 0.6170610
WA.inv.tol 0.3194655 0.7991376 3.552714e-15 0.6267316
WA.cla.tol 0.3573659 0.7991376 4.973807e-15 0.6096722
>
Here is some code that will put the trial number as the first column
to make it easier to process:
> # add the trial number as the first column to make it easier to process
> trial <- 0
> result <- do.call(rbind, lapply(combined, function(.mat){
+ trial <<- trial + 1 # need the "<<-" because we are with an 'lapply'
+ cbind(trial=trial, .mat$object)
+ }))
>
>
> result
trial RMSE R2 Avg.Bias Max.Bias
WA.inv 1 0.3944042 0.7542192 -1.421083e-15 -0.3926396
WA.cla 1 0.4541431 0.7542192 1.421091e-15 -0.1854335
WA.inv.tol 1 0.3517808 0.8044719 0.000000e+00 -0.3588896
WA.cla.tol 1 0.3922082 0.8044719 -6.394885e-15 -0.2735517
WA.inv 2 0.3242771 0.7930415 0.000000e+00 0.6145005
WA.cla 2 0.3641399 0.7930415 -7.105536e-16 0.6170610
WA.inv.tol 2 0.3194655 0.7991376 3.552714e-15 0.6267316
WA.cla.tol 2 0.3573659 0.7991376 4.973807e-15 0.6096722
>> On Sun, Jan 31, 2010 at 3:25 PM, Euan Reavie <euan.reavie * at sign *
gmail.com>
> wrote:
>>> Thanks for your feedback Jim. The complete code is below (showing 10
>>> iterations), and if you think I should re-post this to the list please
> let
>>> me know.
>>> Best - Euan.
>>>
>>> library(rioja, pos=4)
>>> TSSstuff <- read.table("C:/Users/Owner/Desktop/Rfiles/salinity.txt",
>>> header=TRUE, sep="", na.strings="NA", dec=".", strip.white=TRUE)
>>> TSSspecies <- read.table("C:/Users/Owner/Desktop/Rfiles/sppdata.txt",
>>> header=TRUE, sep="", na.strings="NA", dec=".", strip.white=TRUE)
>>>
>>> # make a "combined" list for results
>>> combined <- list()
>>>
>>> # run iterations
>>> for(i in 1:10) {
>>> spec <- TSSspecies
>>> TSS <- TSSstuff
>>>
>>> # get 10 random numbers (20 samples total, no repeats)
>>> x1 <- sample(1:20, 10, replace=F)
>>>
>>> # remove the random samples from both datasets
>>> TSS <- TSS[-x1,]
>>> spec <- spec[-x1,]
>>>
>>> # remove all-zero species columns
>>> spec <- spec[, colSums(spec) > 0]
>>>
>>> # get performance results - jackknife
>>> fit <- WA(spec, TSS, tolDW=TRUE)
>>>
>>> # put iterative results into "combined" list
>>> combined[[i]] <- i # seems to do nothing for some reason
>>> combined[[i]] <- performance(fit)
>>>
>>> }
>>> write.csv(combined, file = "output.csv")
>>> # End
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: jim holtman [mailto:jholtman * at sign * gmail.com]
>>> Sent: Sunday, January 31, 2010 2:16 PM
>>> To: Euan Reavie
>>> Cc: r-help * at sign * r-project.org
>>> Subject: Re: [R] combining data frames in a list - how do I add breaks?
>>>
>>> How about posting your complete set of code that is manipulating the
>>> list. Normally when I am using a list, each list element is the
>>> result from a test/iteration and then I can use something like 'rbind'
>>> at the end. I would not expect the output you are getting with the
>>> results extending to the right, so it must have something to do with
>>> the code that you are using.
>>>
>>> On Sun, Jan 31, 2010 at 2:56 PM, Euan Reavie <euan.reavie * at sign *
gmail.com>
>> wrote:
>>>> I'm a week-old R user, and have become stuck trying to create usable
CSV
>>>> outputs for post-processing. I am using the package Rioja, which
> provides
>>>> small datasets of results. I am running several analyses in a loop and
>>>> iteratively adding the results to a *list* ("combined"). Within each
>>>> iteration I use the following:
>>>>
>>>>> combined[[i]] <- performance(fit)
>>>>
>>>> With two iterations I get the following list, as output to a CSV:
>>>>
>>>> object.RMSE object.R2 object.RMSE.1object.R2.1
>>>> WA.inv 0.321543059 0.86388897 0.350494954 0.882600618
>>>> WA.cla 0.345947482 0.86388897 0.373078015 0.882600618
>>>> WA.inv.tol 0.308649941 0.874585589 0.336975196 0.89148291
>>>> WA.cla.tol 0.330038831 0.874585589 0.356895789 0.89148291
>>>>
>>>> Obviously if I run thousands of iterations this continues on to the
>> right,
>>>> and the CSV files end up being too big for post-processing in most
>>>> spreadsheet programs. Instead I want it to look like this...
>>>>
>>>> 1
>>>> object.RMSE object.R2
>>>> WA.inv 0.321543059 0.86388897
>>>> WA.cla 0.345947482 0.86388897
>>>> WA.inv.tol 0.308649941 0.874585589
>>>> WA.cla.tol 0.330038831 0.874585589
>>>> 2
>>>> object.RMSE.1 object.R2.1
>>>> WA.inv 0.350494954 0.882600618
>>>> WA.cla 0.373078015 0.882600618
>>>> WA.inv.tol 0.336975196 0.89148291
>>>> WA.cla.tol 0.356895789 0.89148291
>>>>
>>>> ...but I have had no luck getting the list to respond to a break in the
>>>> data. I have tried inserting the iteration number between results using
>>>> "combined[[i]]<-i" and even trying to insert an arbitrary text string
>>>> between results. It seems to ignore any such insertions, and yet gives
> no
>>>> errors.
>>>>
>>>> I'd appreciate any thoughts on ways to create a more usable output.
>>>> Best - Euan.
>>>>
>>>
>>> --
>>> Jim Holtman
>>> Cincinnati, OH
>>> +1 513 646 9390
>>>
>>> What is the problem that you are trying to solve?
>>>
More information about the R-help
mailing list