[R] meta analysis with repeated measure-designs?

Mike Cheung mikewlcheung at gmail.com
Wed Jun 16 10:47:29 CEST 2010


Dear Gerrit,

If the correlations of the dependent effect sizes are unknown, one
approach is to conduct the meta-analysis by assuming that the effect
sizes are independent. A robust standard error is then calculated to
adjust for the dependence. You may refer to Hedges et. al., (2010) for
more information. I have coded it here for reference.

Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance
estimation in meta-regression with dependent effect size estimates.
Research Synthesis Methods, 1(1), 39-65. doi:10.1002/jrsm.5

Regards,
Mike
-- 
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://courses.nus.edu.sg/course/psycwlm/internet/
---------------------------------------------------------------------

library(metafor)

#### Robust SE based on Hedges et al., (2010) Eq. 6
#### rma.obj: object fitted by metafor()
#### cluster: indicator for clusters of studies
robustSE <- function(rma.obj, cluster=NULL, CI=.95) {
  # m: no. of clusters; assumed independent if not specified
  if (is.null(cluster)) {
      m=nrow(rma.obj$X)
  } else {
      m=nlevels(unique(as.factor(cluster)))
  }
  res2 <- diag(residuals(rma.obj)^2)
  X <- rma.obj$X
  b <- rma.obj$b
  W <- diag(1/(rma.obj$vi+rma.obj$tau2))     # Use vi+tau2
  meat <- t(X) %*% W %*% res2 %*% W %*% X    # W is symmetric
  bread <- solve( t(X) %*% W %*% X)
  V.R <- bread %*% meat %*% bread            # Robust sampling covariance matrix
  p <- length(b)                             # no. of predictors
including intercept
  se.R <- sqrt( diag(V.R)*m/(m-p) )          # small sample adjustment (Eq.7)
  t.R <- b/se.R
  p.R <- 2*(1-pt(abs(t.R),df=(m-p)))
  crit <- qt( 1-CI/2, df=(m-p) )
  ci.lb <- b-crit*se.R
  ci.ub <- b+crit*se.R
  data.frame(estimate=b, se.R=se.R, t.R=t.R, p.R=p.R, ci.lb=ci.lb, ci.ub=ci.ub)
}


## Example: calculate log relative risks and corresponding sampling variances
data(dat.bcg)
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
dat <- cbind(dat.bcg, dat)

## random-effects model
fit1 <- rma(yi, vi, data=dat, method="DL")
summary(fit1)

estimate       se     zval     pval    ci.lb    ci.ub
 -0.7141   0.1787  -3.9952   <.0001  -1.0644  -0.3638      ***

## Robust SE
robustSE(fit1)
          estimate      se.R       t.R         p.R     ci.lb      ci.ub
intrcpt -0.7141172 0.1791445 -3.986265 0.001805797 -0.725588 -0.7026465

## mixed-effects model with two moderators (absolute latitude and
publication year)
fit2 <- rma(yi, vi, mods=cbind(ablat, year), data=dat, method="DL")
summary(fit2)

         estimate       se     zval    pval     ci.lb    ci.ub
intrcpt   -1.2798  25.7550  -0.0497  0.9604  -51.7586  49.1990
ablat     -0.0288   0.0090  -3.2035  0.0014   -0.0464  -0.0112  **
year       0.0008   0.0130   0.0594  0.9526   -0.0247   0.0262

### Robust SE
robustSE(fit2)
             estimate         se.R         t.R         p.R         ci.lb
intrcpt -1.2797914381 22.860353022 -0.05598301 0.956458098 -2.749670e+00
ablat   -0.0287644840  0.007212163 -3.98832970 0.002566210 -2.922821e-02
year     0.0007720798  0.011550188  0.06684565 0.948022174  2.942412e-05
               ci.ub
intrcpt  0.190086881
ablat   -0.028300755
year     0.001514735




On Mon, Jun 14, 2010 at 4:58 PM, Gerrit Hirschfeld
<Gerrit.Hirschfeld at uni-muenster.de> wrote:
> Hi,
>
> thanks for the references I will try the sensitivitiy-analysis in R and try out winbugs if that does not work (little afraid of switching programmes).
>
> I also had an idea for a reasonable estimate of the correlations. Some studies report both results from paired t-tests and means and SDs, and thus allow to calculate two estimates for d one based on M and SD alone the other on t. The difference between the two estimates should be systematically related to the correlations of measures.
>
> I will keep you posted, if I have a solution or hit a wall.
>
> efachristo and dank je wel!
>
> Gerrit
>
>
> On 12.06.2010, at 15:59, Viechtbauer Wolfgang (STAT) wrote:
>
>> Dear Gerrit,
>>
>> the most appropriate approach for data of this type would be a proper multivariate meta-analytic model (along the lines of Kalaian & Raudenbush, 1996). Since you do not know the correlations of the reaction time measurements across conditions for the within-subject designs, a simple solution is to "guestimate" those correlations and then conduct sensitivity analyses to make sure your conclusions do not depend on those guestimates.
>>
>> Best,
>>
>> --
>> Wolfgang Viechtbauer                        http://www.wvbauer.com/
>> Department of Methodology and Statistics    Tel: +31 (0)43 388-2277
>> School for Public Health and Primary Care   Office Location:
>> Maastricht University, P.O. Box 616         Room B2.01 (second floor)
>> 6200 MD Maastricht, The Netherlands         Debyeplein 1 (Randwyck)
>>
>>
>> ----Original Message----
>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
>> Behalf Of Gerrit Hirschfeld Sent: Saturday, June 12, 2010 12:45
>> To: r-help at r-project.org
>> Subject: [R] meta analysis with repeated measure-designs?
>>
>>> Dear all,
>>>
>>> I am trying to run a meta analysis of psycholinguistic reaction-time
>>> experiments with the meta package. The problem is that most of the
>>> studies have a within-subject designs and use repeated measures ANOVAs to
>>> analyze their data. So at present it seems that there are three
>>> non-optimal ways to run the analysis.
>>>
>>> 1. Using metacont() to estimate effect sizes and standard errors. But as
>>> the different sores are dependent this would result in biased estimators
>>> (Dunlap, 1996). Suppose I had the correlations of the measures (which I
>>> do not) would there by an option to use them in metacont() ?
>>>
>>> 2. Use metagen() with an effect size that is based on the reported F for
>>> the contrasts but has other disadvantages (Bakeman, 2005). The problem I
>>> am having with this is that I could not find a formular to compute the
>>> standard error of partial eta squared. Any Ideas?
>>>
>>> 3. Use metagen() with r computed from p-values (Rosenthal, 1994) as
>>> effect size with the problem that sample-size affects p as much as effect
>>> size.
>>>
>>> Is there a fourth way, or data showing that correlations can be neglected
>>> as long as they are assumed to be similar in the studies?
>>> Any ideas are much apprecciated.
>>>
>>> best regards
>>> Gerrit
>>>
>>> ______________________________
>>> Gerrit Hirschfeld, Dipl.-Psych.
>>>
>>> Psychologisches Institut II
>>> Westfälische Wilhelms-Universität
>>> Fliednerstr. 21
>>> 48149 Münster
>>> Germany
>>>
>>> psycholinguistics.uni-muenster.de
>>> GerritHirschfeld.de
>>> Fon.: +49 (0) 251 83-31378
>>> Fon.: +49 (0) 234 7960728
>>> Fax.: +49 (0) 251 83-34104
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________
> Gerrit Hirschfeld, Dipl.-Psych.
>
> Psychologisches Institut II
> Westfälische Wilhelms-Universität
> Fliednerstr. 21
> 48149 Münster
> Germany
>
> psycholinguistics.uni-muenster.de
> GerritHirschfeld.de
> Fon.: +49 (0) 251 83-31378
> Fon.: +49 (0) 234 7960728
> Fax.: +49 (0) 251 83-34104
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



More information about the R-help mailing list