[R] [OFF] stepwise using REML???

[Douglas Bates]

"Ronaldo Reis Jr." <chrysopa at insecta.ufv.br> writes:

> Hi,
> 
> I know that is not possible make a stepwise procedure using REML in R, I can 
> use ML for this.
> 
> For nested design it may be very dangerous due the difference in
> variance structure, mainly in a splitplot design. ML make
> significative variables that REML dont make.

It would be good to quote an example that shows this.  I'm not sure
that this occurs in general.

> I read an article that is made a stepwise procedure using GENSTAT.
> 
> from article:
> "Terms were dropped from a model in a stepwise procedure by assessing the 
> change in deviance between the full model and the submodel."
> 
> All are made using REML.
> 
> It is possible?! I dont know GENSTAT.

You would need to be more specific about how the comparisons are made.
I assume that you plan to keep the random effects structure constant
and compare two nested models that differ only in the fixed effects
terms.  I can think of four ways of doing this:

1) Use the F-test obtained by fitting the full model and conditioning
on the estimates of the random effects parameters.  This is what the
anova function applied to an model fit by lme gives.

2) Fit both models and compare the values of the REML criterion in a
likelihood ratio test.

3) Fit both models by REML and compare the values of the
log-likelihood (i.e. the ML criterion) in a likelihood ratio test.
You can obtain that value with logLik(fm, REML=FALSE) if fm is your
fitted model.

4)Fit both models and evaluate the REML criterion for the full model
at the two sets of estimates.  Compare these values in a likelihood
ratio test.

I feel that 1) is appropriate, 2) is inappropriate, 3) may be
appropriate and 4) looks interesting.  4) is based on recent work by
Greg Reinsel.

In some simulations reported in chapter 3 of Pinheiro and Bates (2000)
3) fared badly compared to 1).