the web's biggest credit risk modeling resource.

Credit Jobs

Home Glossary Links FAQ / About Site Guide Search


Submit Your Paper

In Rememberance: World Trade Center (WTC)

doi> search: A or B

Export citation to:
- Text (plain)
- BibTeX

Confidence Intervals for the Area under the ROC Curve

by Corinna Cortes of Google Research, and
Mehryar Mohri of Courant Institute, NYU

December 2004

Abstract: In many applications, good ranking is a highly desirable performance for a classifier. The criterion commonly used to measure the ranking quality of a classification algorithm is the area under the ROC curve (AUC). To report it properly, it is crucial to determine an interval of confidence for its value. This paper provides confidence intervals for the AUC based on a statistical and combinatorial analysis using only simple parameters such as the error rate and the number of positive and negative examples. The analysis is distribution-independent, it makes no assumption about the distribution of the scores of negative or positive examples. The results are of practical use and can be viewed as the equivalent for AUC of the standard confidence intervals given in the case of the error rate. They are compared with previous approaches in several standard classification tasks demonstrating the benefits of our analysis.

This paper is republished as Ch.38 in...

Books Referenced in this paper:  (what is this?)

Download paper (125K PDF) 10 pages