[R] Hmisc improveProb() function

[Frank E Harrell Jr]

David A.G wrote:
> Dear list,
> 
> I am trying to work out how the improveProb() function works and how to interpret the results, and I have a few questions. I would be grateful if anyone could shed some light on these.
> 
> in the Net Reclassification Improvement section of the output, is the 2P column the two-sided p-value for the differences in classification? So if a limit is set at 0.05, then lower values indicate significant differences in the classification of both models?

Correct.  But why use an arbitrary level for 'significance'?

> 
> is the index value the actual percentage of improvement? Could this be negative if there was actually no improvement?

NRI is the difference in two proportions.  You may want to emphasize the 
more continuous measure IDI, and a scatterplot relating the two sets of 
predicted probabilities.  The calculations are clear from typing the 
following at the command prompt:

improveProb
print.improveProb

Frank

> 
> This one might be more related to the Pencina et al article and to my non-statistics background, but how are the # of events moving and nonevents moving up and down defined? does one have to specify a probability value cutoff for classification?
> 
> Thanks for your help,
> 
> 
> Dave
> 
> 


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University