Accuracy of Altman & other Bankruptcy prediction models LONG POST
The following are impressions I've gotten from reading some of the research on Bankruptcy Prediction Models. I offer these impressions, not as etched-in-stone conclusions but to stimulate further discussion and caution against relying too heavily on these tools for investment decisions.
Two types of errors:
Type I or alpha error rate (i.e., erroneous prediction of firms that ultimately fail as non-failing) and Type II or beta error rate (i. e., erroneous prediction of non-failing firms as failing).
Altman and others avoid Type I errors about 80% of the time. That is they accurately predict about 80% of the firms that go bankrupt within 1 year. If 100 companies are bankrupt at the end of a year, Altman will have predicted about 80% of those.
On the other hand, the models make a lot of Type II errors: According to research I read, about 40% of the firms they predict will go bankrupt are still alive and kicking within 1 year.
The base rate of bankruptcies in the general population of firms is about 3% in a given year. If the models are used as a screening tool on such a general population, they could be expected to accurately predict 80% of the 3% or 2.4% of total population that actually go bankrupt. I found nothing to specifically indicate what % of general population they would classify as bankrupt. One study estimated that error rates on screening general population of companies would be closer to 50% with the majority of those errors being the prediction of bankruptcy for firms that didn't go bankrupt within the 1 year time frame. This would seem to imply that about 4.8% of the general population would be predicted to go bankrupt and about 2.4 % of those predicted would actually do so.
Often the accuracy rates in studies are higher than they would be in the general population of firms because the studies use a sample consisting of 50% bankrupt firms [They know this in advance, an advantage we don't have] far higher than the 3% base rate of bankruptcy in the general population.
An additional error issue is that Altman and other models were mostly developed and tested on "Old-economy" industrial manufacturing firms. Many of the companies we look at are service or R & D driven technology companies whose business models as different from the traditional manufacturing firm as the space shuttle is from the Wright bros. plane. Much of the accuracy research on these models only looks at industrial manufacturing firms because of this. There's a variation of Altman that looks at service firms, but there appears to be little testing of it's accuracy. The service firms this variation was developed for were probably traditional retail, wholesale, and service type companies and may be less accurate looking at R & D driven companies.
Another potential error issue is the use of "Interesting" accounting or special case situations where something that might not be characterized as debt, is pretty much the equivalent of debt.
So what's the point?
We shouldn't count on 80% accuracy as a given
Using Altman as screening tool on general population of firms may lead to 50% erroneous results
We should and often do, preselect firms with far greater likelihood of failure than the general population, thus improving the accuracy of prediction, but we can't quite know what the overall accuracy is because we don't use pre-selected samples [Sometimes knowing what you don't know is more important than knowing what you know].
The models are still excellent filtering tools that provide very useful information.
It's a good idea to use multiple models [Altman + Springate + Enterprise Value/Book value of debt] to increase accuracy.
The models aren't going to give us as accurate a read on non-manufacturing service or high-tech firms, but it would be a good idea to use the adjusted version of Altman on these firms. |
