[R] Leave one out Cross validation (LOO)

Frank E Harrell Jr f.harrell at vanderbilt.edu
Wed Feb 25 14:38:12 CET 2009

Alex Roy wrote:
> Dear Frank,
>                    Thanks for your comments. But in my situation, I do 
> not have any future data and I want to calculate Mean Square Error for 
> prediction on future data. So, is it not it a good idea to go for LOO?
> thanks
> Alex

With resampling you should be able to estimate any parameter including 
sigma.  The Design package's validate.ols function can estimate sigma 
using the bootstrap or c-v, penalizing for backward stepdown variable 
selection, although I have found some counter-intuitive estimates of 
sigma using Efron's optimism bootstrap.


> On Tue, Feb 24, 2009 at 7:15 PM, Frank E Harrell Jr 
> <f.harrell at vanderbilt.edu <mailto:f.harrell at vanderbilt.edu>> wrote:
>     Alex Roy wrote:
>         Dear R user,
>                               I am working with LOO. Can any one who is
>         working
>         with leave one out cross validation (LOO) could send me the code?
>         Thanks in advance
>         Alex
>     I don't think that LOO adequately penalizes for model uncertainty.
>      I recommend the bootstrap or 50 repeats of 10-fold
>     cross-validation.  See for example the validate and calibrate
>     functions in the R Design package.
>     Frank
>     --

More information about the R-help mailing list