[R] Subsetting data for split-sample validation, then repeating 1000x
David L Carlson
dcarlson at tamu.edu
Fri Aug 22 22:46:25 CEST 2014
You can use replicate() or a for (i in 1:1000){} loop to do your replications, but you have other issues first.
1. You are sampling with replacement which makes no sense at all. Your 70% sample will contain some observations multiple times and will use less than 70% of the data most of the time.
2. You compute r using cor() and r.squared using summary.lm(). Why? Once you have computed r, r*r or r^2 is equal to r.squared for the simple linear model you are using.
# To split your data, you need to sample without replacement, e.g.
train <- sample.int(nrow(A), floor(nrow(A)*.7))
test <- (1:nrow(A))[-train]
# Now run your analysis on A[train,] and test it on A[test,]
# Fit model (I'm modeling native plant richness, 'nat.r')
A.model <- glmmadmb(nat.r ~ isl.sz + nr.mead, random = ~ 1 | site, family =
"poisson", data = A[train,])
# Correlation between predicted 30% and actual 30%
cor <- cor(Atest$nat.r, predict(A.model, newdata = A[test,], type = "response"))
-------------------------------------
David L Carlson
Department of Anthropology
Texas A&M University
College Station, TX 77840-4352
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Angela Boag
Sent: Thursday, August 21, 2014 4:46 PM
To: r-help at r-project.org
Subject: [R] Subsetting data for split-sample validation, then repeating 1000x
Hi all,
I'm doing some within-dataset model validation and would like to subset a
dataset 70/30 and fit a model to 70% of the data (the training data), then
validate it by predicting the remaining 30% (the testing data), and I would
like to do this split-sample validation 1000 times and average the
correlation coefficient and r2 between the training and testing data.
I have the following working for a single iteration, and would like to know
how to use either the replicate() or for-loop functions to average the 1000
'r2' and 'cor' outputs.
--
# create 70% training sample
A.samp <- sample(1:nrow(A),floor(0.7*nrow(A)), replace = TRUE)
# Fit model (I'm modeling native plant richness, 'nat.r')
A.model <- glmmadmb(nat.r ~ isl.sz + nr.mead, random = ~ 1 | site, family =
"poisson", data = A[A.samp,])
# Use the model to predict the remaining 30% of the data
A.pred <- predict(A.model, newdata = A[-A.samp,], type = "response")
# Correlation between predicted 30% and actual 30%
cor <- cor(A[-A.samp,]$nat.r, A.pred, method = "pearson")
# r2 between predicted and observed
lm.A <- lm(A.pred ~ A[-A.samp,]$nat.r)
r2 <- summary(lm.A)$r.squared
# print values
r2
cor
--
Thanks for your time!
Cheers,
Angela
--
Angela E. Boag
Ph.D. Student, Environmental Studies
CAFOR Project Researcher
University of Colorado, Boulder
Mobile: 720-212-6505
[[alternative HTML version deleted]]
______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list