[R] Question regarding significance of a covariate in a coxme survival

Norm Matloff matloff at cs.ucdavis.edu
Sun Aug 29 23:53:08 CEST 2010


Using a p-value to make any kind of decision is questionable to begin
with, and especially unreliable in choosing covariates in regression.
Old studies, e.g. by Walls and Weeks and by Bendel and Afifi, have shown
that if predictive ability is the criterion of interest and one wishes
to use p-values for deciding whether to include a covariate, one should
set the p-value bar very large, at 0.25 and even 0.40.

By contrast, methods such as AIC are aimed at avoiding overfitting, by
penalizing models with large numbers of covariates.  Same for Mallows'
Cp, cross validation etc.

So, the p-value and AIC are answering quite different questions, and
thus should not be expected to give the same or even similar results.
But, worse than that, many point out that p-values tend not to be
answering ANY question of practical interest.

It's a shame that the use of p-values is so entrenched.  I can expand on
this, with references, if there is interest.

Norm Matloff
Professor of Computer Science (formerly Statistics)
University of California, Davis



More information about the R-help mailing list