[R] generalized linear mixed models - how to compare?

[Prof Brian Ripley]

On Sun, 17 Apr 2005, Deepayan Sarkar wrote:

> On Sunday 17 April 2005 08:39, Nestor Fernandez wrote:

>> I want to evaluate several generalized linear mixed models, including
>> the null model, and select the best approximating one. I have tried
>> glmmPQL (MASS library) and GLMM (lme4) to fit the models. Both result
>> in similar parameter estimates but fairly different likelihood
>> estimates.
>> My questions:
>> 1- Is it correct to calculate AIC for comparing my models, given that
>> they use quasi-likelihood estimates? If not, how can I compare them?
>> 2- Why the large differences in likelihood estimates between the two
>> procedures?
>
> The likelihood reported by glmmPQL is wrong, as it's the likelihood of
> an incorrect model (namely, an lme model that approximates the correct
> glmm model).

Actually glmmPQL does not report a likelihood.  It returns an object of 
class "lme", but you need to refer to the reference for how to interpret 
that.  It *is* support software for a book.

> GLMM uses (mostly) the same procedure to get parameter estimates, but as 
> a final step calculates the likelihood for the correct model for those 
> estimates (so the likelihood reported by it should be fairly reliable).

Well, perhaps but I need more convincing.  The likelihood involves many 
high-dimensional non-analytic integrations, so I do not see how GLMM can 
do those integrals -- it might approximate them, but that would not be 
`calculates the likelihood for the correct model'.  It would be helpful to 
have a clarification of this claim.  (Our experiments show that finding an 
accurate value of the log-likelihood is difficult and many available 
pieces of software differ in their values by large amounts.)

Further, since neither procedure does ML fitting, this is not a maximized 
likelihood as required to calculate an AIC value.  And even if it were, 
you need to be careful as often one GLMM is a boundary value for another, 
in which case the theory behind AIC needs adjustment.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595