[R] ridge regression

Frank E Harrell Jr fharrell at virginia.edu
Fri Jun 6 12:36:55 CEST 2003


On Thu, 05 Jun 2003 21:50:13 -0400
"Liaw, Andy" <andy_liaw at merck.com> wrote:

> Hi Frank,
> 
> > From: Frank E Harrell Jr [mailto:fharrell at virginia.edu]
> 
> [snip]
> 
> > The anova method for ols fits 'works' when you penalize the 
> > model but there is some controversy over whether we should be 
> > testing biased coefficients.  Some believe that hypothesis 
> > tests should be done using the unpenalized model.  That 
> > brings up other ways to handle collinearity: test groups of 
> > variables in combination so they don't compete with each 
> > other, or collapse them into summary scores (e.g., principal 
> > components) before putting them in the model. 
> 
> I'm not clear about the last point.  Suppose three of the variables
> are nearly collinear.  Are you suggesting to replace the variables
> with the first one or two PCs, and drop the rest?  If so, doesn't
> that also lead to biased estimators?

Yes, it's another way to get biased estimators.  But many statisticians test these components in incomplete principal components regression using ordinary tests, to get tests for the whole group of coefficients these represent (under restrictions).  -Frank

> 
> Best,
> Andy
>  
> > ---
> > Frank E Harrell Jr              Prof. of Biostatistics & Statistics
> > Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
> > U. Virginia School of Medicine  
> > http://hesweb1.med.virginia.edu/biostat
> 
> ------------------------------------------------------------------------------
> Notice: This e-mail message, together with any attachments, cont... {{dropped}}
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help


---
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat




More information about the R-help mailing list