[R] {metafor} variance explaination for paired pre-test/posttest

[Viechtbauer Wolfgang (STAT)]

The standardized mean change using 'change score standardization' is described in this article:

Gibbons, R. D., Hedeker, D. R., & Davis, J. M. (1993). Estimation of effect size from a series of experiments involving paired comparisons. Journal of Educational Statistics, 18(3), 271-279.

For a comparison of the standardized mean change using change versus raw score standardization, see:

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105-125.

Viechtbauer, W. (2007). Approximate confidence intervals for standardized effect sizes in the two-independent and two-dependent samples design. Journal of Educational and Behavioral Statistics, 32(1), 39-60.

These articles also provide equations for the sampling variance of the standardized mean change. The equation 1/ni + yi^2/(2*ni) is the estimate based on the asymptotic variance of the standardized mean change using change score standardization. 

Best,
Wolfgang

--   
Wolfgang Viechtbauer, Ph.D., Statistician   
Department of Psychiatry and Psychology   
School for Mental Health and Neuroscience   
Faculty of Health, Medicine, and Life Sciences   
Maastricht University, P.O. Box 616 (VIJV1)   
6200 MD Maastricht, The Netherlands   
+31 (43) 388-4170 | http://www.wvbauer.com   

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
> On Behalf Of John Williams
> Sent: Tuesday, April 08, 2014 02:30
> To: r-help at r-project.org
> Subject: [R] {metafor} variance explaination for paired pre-test/posttest
> 
> In a previous post
> https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html
> <https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html>  , the
> following calculation was given for imputing the variance of change
> scores
> for paired studies:
> 
> // begin quote
> 
> 2) Often, the dependent variable is not the same in each study. Then you
> will have to resort to a standardized outcome measure. There are two
> options:
> 
> a) standardization based on the change score standard deviation
> 
> Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 /
> (2*ni).
> 
> // end quote
> 
> I used the sampling variance equation above in a paper that is being
> reviewed by a coauthor, who is a biostatistician.
> 
> He commented that he has never seen this equation for variance before,
> and
> it looks strange to him. To put my knowledge into perspective, I am an
> undergraduate taking my first statistics course. I imputed the t-
> statistic
> from two-sided p-values reported in the paper, and used that to get the
> sdi
> (as in the previous post).
> 
> I consulted the Cochrane Handbook and The Handbook of Research Syntheses
> and
> Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find
> that
> equation anywhere.
> 
> Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining
> the
> sample variance above? I need to be able to defend my choice of equation.
> Since it's the only method that I found that doesn't rely on a
> correlation
> coefficient (which are not included in the papers), I'd like to be able
> to
> justify it and not redo calculations for 23 studies if possible.
> 
> Thank you very much,
> 
> John
> 
> ~~~~
> John Williams
> ALB Candidate, Harvard University (Expected May 2014)
> johnwilliams at fas.harvard.edu
> jawilliamsjr at gmail.com