[R] metafor - interpretion of QM in mixed-effects model with factor moderator

[Viechtbauer Wolfgang (STAT)]

Please see my comments below.

Best,
Wolfgang

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
> On Behalf Of Rachel Slatyer
> Sent: Wednesday, February 27, 2013 06:00
> To: R-help at r-project.org
> Subject: [R] metafor - interpretion of QM in mixed-effects model with
> factor moderator
> 
> Hi,
> 
> I'm using metafor to perform a mixed-effects meta-analysis. I'd like to
> test whether the effect is different for animals and plants/whether
> "group"
> (animal/plant) influences the effect size, but am having trouble
> interpreting the results I get. I've read previous posts about QM in
> metafor, but I'm still a bit confused. I've dummy-coded the factors:
> 
> MData$k.animal<-ifelse(MData$Type=="Animal",1,0)
> MData$k.plant<-ifelse(MData$Type=="Plant",1,0)
> 
> I've then run the model with the following script:
> 
> MixModel<-rma(z, v.z., mods=cbind(k.animal, k.plant), intercept=FALSE,
> data=MData, subset=(Response=="Tolerance"))
> 
> I receive the output:
> 
> Test of Moderators (coefficient(s) 1,2):
> QM(df = 2) = 8.5758, p-val = 0.0137
> 
> Model Results:
> 
>           estimate      se    zval    pval    ci.lb   ci.ub
> k.animal    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
> k.plant     0.5095  0.2209  2.3066  0.0211   0.0766  0.9424  *
> 
> Does this mean that, overall, there is a significant effect of "group" on
> the overall effect size (based on the QM test)? 

Not quite. Since you have removed the intercept, the two estimates reflect the estimated effect sizes (or to be precise, the estimated average effect sizes) for "animals" and "plants", respectively. The QM test is testing these two coefficients simultaneously, so it is testing the null hypothesis that the average effect sizes are both equal to 0. Since the test is significant, we can reject that null hypothesis. However, this does not test whether the effect is different for animals or plants.

> If I want to test whether
> the effect size for plants is different to that for animals, can I use:
> 
> MixModel2<-rma(z, v.z., mods=cbind(k.plant), data=MData,
> subset=(Response=="Tolerance")) ?
> 
> Here I get:
> 
> Test of Moderators (coefficient(s) 2):
> QM(df = 1) = 0.0039, p-val = 0.9502
> 
> Model Results:
> 
>          estimate      se    zval    pval    ci.lb   ci.ub
> intrcpt    0.4877  0.2703  1.8042  0.0712  -0.0421  1.0175  .
> k.plant    0.0218  0.3491  0.0624  0.9502  -0.6624  0.7060
> 
> Does this mean that there is no significant difference in effect size
> between plants and animals (now the intercept?) ? Or is this something
> completely different?

Correct. Now you are testing whether the difference in the average effect for "plants" is different from the average effect for "animals". The estimated difference is 0.0218 (so the estimated average effect for plants is 0.4877 + 0.0218 = 0.5095 -- the same value as obtained from your first model), which is not significantly different from zero.

This is a nice example to illustrate that the common practice of subgrouping (whether it be in a meta-analysis or in a primary trial) and testing effects within subgroups can be misleading. Note that your first example indicates a significant effect for plants, but not for animals. If you were to fit a random-effects model just within the subset of studies with animals and just within the subset of studies with plants, you will probably obtain rather similar results as in your first model (the difference only arising from the fact that in your first model, tau^2 is estimated once, while two separate models within those two subgroups will also give you two separate values for tau^2). However, the difference in the effect is far from significant and that is in fact the proper test of this interaction.

> I'd really appreciate any input,
> 
> Rachel Slatyer
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.