[R] Fw: Logistic regresion - Interpreting (SENS) and (SPEC)
Frank E Harrell Jr
f.harrell at vanderbilt.edu
Wed Oct 15 14:29:16 CEST 2008
Gad Abraham wrote:
>> This approach leaves much to be desired. I hope that its
>> practitioners start gauging it by the mean squared error of predicted
> Is the logic here is that low MSE of predicted probabilities equals a
> better calibrated model? What about discrimination? Perfect calibration
Almost. I was addressed more the wish for the use of strategies that
maximize precision while keeping bias to a minimim.
> implies perfect discrimination, but I often find that you can have two
That doesn't follow. You can have perfect calibration in the large with
> competing models, the first with higher discrimination (AUC) and worse
> calibration, and the the second the other way round. Which one is the
> better model?
I judge models on the basis of both discrimination (best measured with
log likelihood measures, 2nd best AUC) and calibration. It's a
two-dimensional issue and we don't always know how to weigh the two.
For many purposes calibration is a must. In those we don't look at
discrimination until calibration-in-the-small is verified at high
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
More information about the R-help