Testing for Rating Consistency in Annual Default Rates

by Richard Cantor of Moody's Investors Service, and
Eric Falkenstein of Moody's|KMV

September 2001

Executive Summary: Executive Summary Investors, issuers, academics, and financial market regulators, alike, have been increasingly focusing their attention on rating consistency. To encourage and facilitate this scrutiny, Moody 's has for many years published historical default and debt recovery statistics. More recently, we have published commentary designed to provide guidance about he in ended meanings of our bond ratings. In the process, we have acknowledged historical differences in the meaning of our ratings across broad market sectors - namely, corporate finance, structured finance, and public finance. However, as discussed in prior research, due to certain steps we have taken internally, these differences are expected to diminish over time.

This Special Comment addresses the issue of measuring rating consistency, and more specifically, it evaluates he reliability of historical default rates as estimates of he rue underlying default probabilities associated with Moody 's ratings. The comment also provides some technical guidance to interpret the differences in historical annual default rates across bond market sectors. Finally, it concludes with three case studies that are designed to illustrate the usefulness of these methods for comparing the historical default rates of different bond market sectors.

Based upon his work, we make the following observations:

• Rating consistency is difficult to test because credit risk has multiple attributes - default probability, loss severity, and transition risk. When focusing more narrowly on just historical default rates, consistency still needs to be measured at multiple horizons; although consistency over longer investment horizons is clearly more central to the meaning of Moody's ratings than consistency over shorter investment horizons.
• But differences in historical default rates can indeed be subjected to rigorous tests of statistical significance, as long as such tests incorporate the volatility and persistence of macroeconomic and sectoral shocks. The presence of macroeconomic and sectoral shocks means that historical default rates may vary across bond market sectors for fairly long periods of time without necessarily implying fundamental differences in underlying default probabilities. Nevertheless, the observed historical volatility of these shocks does impose limits on expected differences in observed default rates.
• For certain sectoral comparisons, such as for Banks vs. Nonbanks and for US vs. Non-US issuers, observed differences in historical speculative-grade default rates may appear significant if the presence macroeconomic and sectoral shocks is ignored. However, when the effects of these shocks are considered, the differences are no longer statistically significant. This stands in contrast to the experience of speculative-grade issuers in he Utilities sector when compared to Other Companies. In this comparison, the differences in historical default rates are statistically significant, regardless of whether or no the effects of annual default rate shocks are incorporated into the analysis.

Published in: Journal of Fixed Income, Vol. 11, No. 2, (September 2001), pp. 36-51.

Download paper (333K PDF) 27 pages