[R] Computing Confidence Intervals for AUC in ROCR Package

[Frank E Harrell Jr]

Even though ROC curves don't shed much light on the problem, the area 
under the ROC is useful because it is the Wilcoxon-type concordance 
probability.  Denoting it by C, 2*(C-.5) is Somers' Dxy rank correlation 
between predictions and binary Y.  You can get the standard error of Dxy 
from the Hmisc package rcorr.cens function, and backsolve for s.e. of C 
hence get a confidence interval for C.  This uses U-statistics and is 
fairly assumption-free.

Frank

Na'im R. Tyson wrote:
> Dear R-philes,
> 
> I am plotting ROC curves for several cross-validation runs of a 
> classifier (using the function below).  In addition to the average AUC, 
> I am interested in obtaining a confidence interval for the average AUC.  
> Is there a straightforward way to do this via the ROCR package?
> 
> plot_roc_curve <- function(roc.dat, plt.title) {
>     #print(str(vowel.ROC))
>     pred <- prediction(roc.dat$predictions, roc.dat$labels)
>     perf <- performance(pred, "tpr", "fpr")
>     perf.auc <- performance(pred, "auc")
>     perf.auc.areas <- slot(perf.auc, "y.values")
>     curve.area <- mean(unlist(perf.auc.areas))
>     #quartz(width=4, height=6)
>     plot(perf, col="grey82", lty=3)
>     plot(perf,lwd=3,avg="horizontal",spread.estimate="boxplot",
>         add=T)
>     title(main=plt.title)
>     mtext(sprintf("%s%1.4f", "Area under Curve = ", curve.area),
>         side=3, line=0, cex=0.8)
> }
> 
> P.S. After years of studying statistical analysis as a student, I still 
> consider myself a novice.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 


-- 
Frank E Harrell Jr   Professor and Chairman        School of Medicine
                      Department of Biostatistics   Vanderbilt University