The Predictive Accuracy of Credit Ratings: Measurement and Statistical Inference

by Walter Orth of the University of Cologne

February 16, 2011

Abstract: Credit ratings are ordinal predictions for the default risk of an obligor. To evaluate the accuracy of such predictions commonly used measures are the Accuracy Ratio or, equivalently, the Area under the ROC curve. The disadvantage of these measures is that they treat default as a binary variable thereby neglecting the timing of the default events and also not using the full information from censored observations. We present an alternative measure that is related to the Accuracy Ratio but does not suffer from these drawbacks. As a second contribution, we study statistical inference for the Accuracy Ratio and the proposed measure in the case of multiple cohorts of obligors with overlapping lifetimes. We derive methods that use more sample information and lead to more powerful tests than alternatives that filter just the independent part of the dataset. All procedures are illustrated in the empirical section using a dataset of S&P Long Term Credit Ratings.

JEL Classification: C41, C52, G17, G24, G32.

Keywords: Ratings, predictive accuracy, Accuracy Ratio, Harrell's C, over-lapping lifetimes

Forthcoming in: International Journal of Forecasting.

Books Referenced in this paper:  (what is this?)

Download paper (272K PDF) 20 pages

Related reading: Multivariable Prognostic Models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors

[