[R] Logistic regression model + precision/recall

[Frank E Harrell Jr]

nitin jindal wrote:
> Hi,
> 
> I am using logistic regression model named lrm(Design)
> 
> Rite now I was using Area Under Curve (AUC) for testing my model. But, now I
> have to calculate precision/recall of the model on test cases.
> For lrm, precision and recal would be simply defined with the help of 2
> terms below:
> True Positive (TP) - Number of test cases where class 1 is given probability
>> = 0.5.
> False Negative (FP) - Number of test cases where class 0 is given
> probability >= 0.5.

Why 0.5?

> 
> Precision = TP / (TP + FP)
> Recall = TP / ( Number of Positive Samples in test data)

Those are improper scoring rules that can be tricked.  If the outcome is 
rare (say 0.02 incidence) you could just predict that no one will have 
the outcome and be correct 0.98 of the time.  I suggest validating the 
model for discrimination (e.g., AUC) and calibration.

Frank

> 
> Any help is appreciated.
> 
> I an write a long code with for loops and all, but is there any inbuild
> function or just few commands that would do the task.
> 
> regards,
> Nitin
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University