“The question is not ‘Did risk management fail?’ The question is ‘Why did risk management fail?”
Anonymous Board Member
A common refrain is that the financial crisis was an episode of massive risk management failure on the part of major financial institutions, not to mention ratings agencies and financial regulators. Certainly financial failure was rampant. But is it correct to blame risk management?
Highly acclaimed Professor of Finance at Ohio State University and risk management guru Rene Stulz* says “Not so fast.” He argues that the mere fact that an institution suffers a large loss does not necessarily mean that risk management failed, or even that the institution made a mistake.
The role of risk management is to identify and measure risks and convey that information to senior management. It is generally not the province of risk management to select or determine how much risk to take. That is the responsibility of senior management and the board of directors. Typically, risk management’s responsibility is to monitor and report on risk exposures and keep senior management informed. In some organizations, risk management has the additional responsibility to manage and control the risk, but that is relatively rare.
If senior management knowingly chooses to take on a strategic profile that includes a non-negligible probability of firm failure, and the failure causing event occurs, that is an outcome attributable to a poor strategy, or to bad luck, but not to failure of the risk management process.
A failure of risk management would occur if senior management was misinformed as to the riskiness of the chosen strategy. For example, if management believed the chance of failure was extremely remote, when in fact it was fairly likely, that could represent a failure of risk management. In this case, risk management either did not properly measure the risk or failed to communicate this to senior management.
Why does this matter? It is important to get the diagnosis correct in order to learn the proper lessons, and avoid making changes that are the wrong ones.
Professor Stulz suggests a number of ways that risk management could fail:
a. There could be a bad choice of risk metric
b. The metric is OK but there is faulty measurement
c. Failure to consider major risks (known and unknown)
d. Failure to communicate results to senior management
e. Failure to monitor risks
LTCM
Stulz applies this lens to the 1998 failure of the hedge fund Long Term Capital Management (LTCM). LTCM enjoyed strong returns from its startup in 1994 through the end of 1997. But in 1998 following the Russian debt default, LTCM lost 70% of its capital and the Federal Reserve Bank of New York coordinated a rescue by private financial institutions. Stulz suggests that this outcome is consistent with a very simple return generating process: 99% chance of 25% annual return, 1% chance of 70% loss. If presented with this bet would you take it? Stulz argues that it would have been compelling to do so.
Of course, we don’t really know that the probability of failure was just 1 in 100, maybe it was much higher, like 1 in 4. In that case, the expected return would have been mediocre at best (and the spectacular returns from 1994 through 1997 were pretty lucky). So, how can we tell if the LTCM failure was due to a failure of risk management instead of a very unfortunate outcome? The actual returns do not allow us to answer this question.
Application of the Stulz taxonomy does not un-muddy the waters. Stulz suggests that the principle risk metric used by LTCM was the one day Value at Risk (VaR). This metric, he opines, is useful for market risk management and has the benefit of plenty of historical data for estimation. But it is not well designed to assess the likelihood of catastrophic shocks, nor the extent of their consequences. A longer term VaR would have been better, he asserts. But would that really have helped? There is little likelihood that using a longer term VaR measure would have enabled management to anticipate the consequences for liquidity, spreads and correlations of the stressed market conditions following the Russian default.
While it is not clear, even with the benefit of hindsight and perspective, that the risk management process at LTCM was flawed, Stulz argues that we can use past crisis experiences to improve risk management in the future. For one thing, we have a growing data base of financial behavior under crisis conditions. Another lesson is the need for risk management to go beyond historical data and quantitative models. Stulz recommends the use of scenario analysis to examine the possible economic consequences of extreme shocks.
The Financial Crisis
What are the lessons for risk management from the current financial crisis? Even though we are not yet far removed from the crisis, some things are clear. There were large declines in housing prices in many parts of the country. These housing price declines prompted a surge in mortgage defaults and losses which generated catastrophic losses in mortgage securities that were held in highly levered, mark-to-market portfolios, often funded with wholesale short-term money. The mixture was combustible and it blew up.
But, what were the risk management failures? It seems to me clear that there was widespread under-estimation of the likelihood of major housing price declines. Those entities responsible for measuring the tail risk of losses on mortgages and mortgage backed securities, including ratings agencies and mortgage guarantors, appear to have dropped the ball. What do I mean by this? Well, back in 2006, a year or so before the crisis hit, the general consensus forecast for housing price appreciation (HPA) was somewhere in the range 0-4%. Few forecasters anticipated continuation of double digit growth, and few foresaw a gigantic meltdown.
Given a base case forecast of low single digits, what was a reasonable projection for the 1 in 10 or the 1 in 100 HPA scenario? Given substantial historical volatility in HPA, along with a recent extended period of housing prices rising faster than fundamentals, I’d say that it would not have been reasonable to assume that there was only a remote probability of at least modest negative HPA for the overall index, and substantial negative HPA for at least some (geographic) sub-indexes. Yet, based on FCIC testimony it appears that the ratings agencies assigned a very remote likelihood to widespread negative HPD.
Is this an episode of risk management failure on the part of the credit ratings agencies? I think it is a reasonable candidate for “faulty measurement” in the professor’s taxonomy. Of course, there is room to differ on the appropriate probability distribution for home prices. What has occurred was definitely not the most likely scenario, but negative HPA should have been chosen as one of the stress scenarios, at least, it should have been for those entities that were responsible for estimating tail risk.
Lessons
The main conclusion from the professor’s article is that there the mere occurrence of a calamity does not prove that the risk management process failed. A second conclusion is that there are inherent limits on the use of quantitative models and historical data to measure risk. Some risks are simply not quantifiable. This is probably not a significant revelation to most knowledgeable observers. The professor’s remedy is to supplement quantitative assessments with more qualitative assessments, such as the use of scenario analysis. That rings true to me, you ought to take into account the Worst Case scenarios, not just the Most Likely.
*Rene Stulz, “Risk Management Failures: What Are They and When Do They Happen,” Journal of Applied Corporate Finance, Fall 2008.