Sunday, March 22, 2009

Recipe for Disaster: The Formula That Killed Wall Street

This interesting article in Wired is the only attempt I have seen to explain some details of the recent financial upheavals in terms that a mathematically literate non-specialist should understand. It lays the blame at the inappropriate application of a elegant formula for estimating the risk of coupled investments according to the correlation of their prices. The correlation is unknown a priori (indeed it's not obvious to me that such a thing even exists given the fractal nature of market prices) so the general approach was to estimate the correlation based on historical data of how the market prices had covaried in the past. For investments that had not been around for very long, this is obviously going to be rather approximate, even if the underlying principles had been sound.

It seems quite clear from the article that everyone who looked into it (including the formula's originator) knew that this application was inappropriate, but the bankers all did it anyway. Without it, they would not have been able to bury the time-bomb of the ludicrously-named "sub-prime" mortgages to sell on to unwitting dupes.

The article mainly focusses on the risks of estimating correlation based on a short historical data set, but the problem is probably more fundamental than that. By using market data to price assets in the market, the bankers created an inherently unstable feedback loop. Even if the long-term correlation between two securities was perfectly estimated and known to be low (in the long-term limit), there would still be times that an external shock would cause their prices to move in the same direction for a few days. This would increase their apparent correlation (estimated over a finite historical record), thus reducing their value as a bundled asset. Oops! There is also going to be a regression to the mean effect whereby securities that are selected due to their historically low correlations will have higher correlations in the future, just because their original low correlation was fortuitous. It is hard to see how mathematically literate bankers could have overlooked these issues, so I suppose the obvious conclusion is that they didn't care so long as they got their bonuses in the meantime. Maybe it is all more complicated than that...

16 comments:

Alastair said...

I think they were awarded a Nobel Prize in economics for coming up with that formula, but even if the dealers knew the formula was wrong they had no choice but to use it because it was bringing others success.

The major flaw was lack of regulation in a free market. The banks had to take on these shaking loans because their competitors were doing so and making money out of it. If they did not join in then they would have been taken over by the more "successful" banks. The dealers would lose their jobs because the guy at the next desk was making more money, and the chief executive would lose his job because another bank was making more money.

You have to have an effective referee in any game, or the cheats will win!

What is not generally realised by non-economists is that banks create money. By increasing the max morgate from 3 X ave earning to 5 X ave. earnings they created 60% more spending power. Everyone felt richer, but when Peak Oil hit and the economy slowed it was obvious that these huge mortgages were not going to be repaid by the new unemployed.

The housing bubble has been burst by Peak Oil. What is so ridiculous is that Gordon Brown seems to think he can solve the problem by just blowing harder.

Cheers, Alastair.

William M. Connolley said...

I think this is b*ll*cks (the article, not your summary of it). It looks to me like the all-too-common situation o a journo seizing on one element of a very complex situation and deciding that *this* is the key factor. There is no way at all that the recent bubble was founded just on one new way of measuring risk.

crandles said...

So what is your (more complex) version of why all that sub-prime mortgage finance was available if it isn't the simple version because it could be sold on as AAA rated when the people arranging the mortgages knew it was being given to NIJAs who had little prospect of paying the terms?

William M. Connolley said...

I think you've proved my point. You can't sell on junk mortgages as AAA rated merely by applying a little formula - or if you can find a buyer stupid enough to do that, then good luck to you.

It is more complex than that, but no I can't tell you the details, because I don't know them. Nor is it clear to me that anyone does. Underpricing risk is certainly part of it

crandles said...

The simple version I was trying to make was the finance was available because everyone in the chain thought they were making money and avoiding the risk.

Sure when you get into more detail you can find more complexity. CDOs were just bundles of mortgages. The mortgage arrangers made fees and passed on the risk selling the mortgages to bankers who bundled them into CDOs and sold them to investors who were told that the quants had looked at them and assessed their risk as being equivalent to AAA. This appeared a great arrangement for bankers as they avoided the risks and there was very little regulatory costs. Banks have get towers of assets to invest and similar obligations. Regulation places a limit on the size of these 'towers' according to the banks capital. Make a little greater return, avoid the risk, avoid regulatory costs, and keep it off balance sheet so it isn't reducing the size of these towers and you can make a lot of profit.

Trouble is having created these CDOs which bankers thought were great for above reasons, their value kept appreciating. This made them look lucrative and provided the incentives to keep finding more mortgages even when the quality declined. The bankers also couldn't resist letting their traders trade them. As they were a booming market and appreciating traders bought them. This meant the bankers were not avoiding the risks they thought they were avoiding by packaging the mortgage assets into CDOs. On the packaging end they thought they had avoided the risk so they didn't pay much attention to quality. On the investing end they did what everyone was doing in using the quants formula.

More complexity in more detail but is it really much different in essence than the simple version of 'the finance was available because everyone in the chain thought they were making money and avoiding the risk'. Why did they think they were avoiding the risks? Because of the discrepancy between the known risk going into the CDOs and the way risk was evaluated on the investing side.

It looks really stupid but when people only see one side or one portion and are making good money, are they going to want to rock the boat? Even if some do, will the majority ignore them and think things must be OK as I am making good money from doing this so it must make sense.

crandles said...

>"The article mainly focusses on the risks of estimating correlation based on a short historical data set, but the problem is probably more fundamental than that."

Not quite sure what you mean by
"inherently unstable feedback loop" but any banker or chartered accountant should be able to tell anyone what happens when you bundle assets together: There may be some non-systematic risk that gets diversified away but also some systematic risk that does not. By the formula treating the correlation as a fixed quantity that is estimated this nature of risk is being ignored. When property prices are rising there is only non-systematic risk - eg planning permission for something unpleasant near a particular property. The bankers would not package lots of nearby properties into one CDO so the correlation of these events within a CDO is very low. By estimating the correlation over times when property prices are only rising you get a low correlation and a seemingly high but ridiculous AAA rating.

The systematic risk is falling property prices when all do move downwards together.

So I think the article is right to focus on "the risks of estimating correlation based on a short historical data set". Alternatively, rather than saying "indeed it's not obvious to me that such a thing even exists", it might be better to say more clearly these things are correlated differently in different market conditions.

crandles said...

>"Everyone felt richer, but when Peak Oil hit and the economy slowed it was obvious that these huge mortgages were not going to be repaid by the new unemployed."

Why do you think Peak Oil is involved?

I think it obvious to everyone that NIJAs were only going to be able to pay when property prices were rising and they could remortgage. When it became clear property prices were going to fall, NIJAs' were not going to be able to repay and large losses were going to land somewhere. If few people know where the large losses will land, you avoid lending anywhere that might be hit and you get a credit crunch.

The credit crunch came before the economic slowdown.

Peak oil is when there is more oil sloshing about than ever before so I think you have a credibility problem in maintaining peak oil caused the credit crunch.

James Annan said...

Chris - don't know what NIJA means. As for feedback, I simply mean that any external shock that caused prices to move down together would thereby reduce the value of these bundles. I agree that there is probably an irreducible level of systematic variation that cannot be eliminated by any amount of bundling - in fact this is similar to the debate over the independence of estimates of climate sensitivity (or would be, if the protagonists had got to a level of understanding that we could actually debate these things rationally).

William - it seems like a credible account to me, although I'm sure it is simplified. Yes, you have to find a buyer, but if "everyone" agrees that dodgy assets can be bundled to increase their value and sold on, then there are strong incentives to do it - even for those who can see the risks (and it can be hard to see the truth when your salary depends on not seeing it).

crandles said...

NIJA = No Income, Job or Assets

Obviously an extreme case and not all the mortgages will be that bad.

But since the losses on CDOs are estimated to be $1 Trillion (yes I do mean trillion) things did get pretty extreme (and that is just the losses to the banks).

http://www.bbc.co.uk/blogs/thereporters/robertpeston/2009/03/we_are_the_monolines_now.html

More complexity? Cannot sell junk as AAA. Yes more involved: tranching and insurance that turns out to be worthless but at the time nobody believed the monolines would have got it so wrong they would be wiped out. But does this change the nature & credibility of the story?

>"As for feedback... "
That is just a mechanism that comes into effect if everyone sticks with the system when things start to go sour. But people don't stick with a clearly a stupid system for long when it is going wrong and nor is this the issue. How frequently do you think these correlation statistics would be calculated once things are starting to go wrong? The issue is how the situation got set up not how it starts to go wrong.

It is not all down to belief in the formula. Annual bonuses for traders play a part. Suppose at least two years ago, the traders could see the formula is flawed and therefore know the system will continue while house prices rise but there will be big problems when house prices fall (or are expected to). Also suppose they expect house prices to rise in that year. The bonus system is there to incentivise their behaviour so should they get involved? If not what do you expect them to do instead? Scream and shout about the problems? Resign? Are such expectations realistic?

So yes - There are strong incentives to do it - even for those who can see the risks (and it can be hard to see the truth when your salary depends on not seeing it and perhaps even harder when you could get a big bonus depending on not seeing it). Do what you think earns you a bonus this year and hope you can sell out before others at some time in the future. If you cannot sell out in future, well you will have got some big bonuses in the meantime and hope these are safe from being clawed back and hope you can find comfort in some form of denial of responsibility like it was the bonus system that caused the mess.

So who do we blame for the sorry mess? Everyone who got involved? The traders for stupidly buying the assets? The traders for being duped? The bank execs for allowing annual bonus system to continue? The bank execs for incompetence in not understanding what the traders were trading? The bank execs for incompetence in not bringing information known to different individuals in different branches of their organisations together? The bank execs for reckless not caring if problems were building up? The bank execs for being knowingly wilful in not caring about problems building up?

Or is it something more sinister like collusion to misinform monoline insurers?

Or is it just human nature that is to blame and things like this will happen from time to time?

The government for lack of suitable regulation?

The regulators?

Anyway it seems that an extension to “Lose 20,000 and you have a problem, lose 20 million and the bank has a problem, lose 20 billion and the government has a problem” to include lose a trillion and the economy and taxpayers have a problem L

I think there should be enough anger out there to support a tax on anyone involved in trading CDOs or supervising such trade and has received a bonus in the last five years to suffer a tax on those bonus of 90% for last year, 80% for previous year, then 70% 60% and 50% for previous years. (Tax paid (40% for UK) should be deducted before applying these percentages) It is retrospective taxation but taxpayers should be angry enough that this should not matter. If it bankrupts some because they have spent it, so much the better would seem to reflect a reasonable level of anger.

(I have no delusion that this is going pay even a small fraction of the taxpayer bailout costs.)

David B. Benson said...

Well, there is obviously very little difference between

Black-Scholes

and

Black Holes.

Unknown said...

Many Japanese economists, in hindsight, think that the essential mechanism of the recent failure of American economy is the same as that of Japanese asset price bubble, boom in 1987-88 and bust in 1990-91. It was believed that the price of real estate would rise indefinitely, and the belief eventually failed. To anticipate the failure, one need not be an ecological economist like Herman Daly who think that the human society is a subsystem of the finite natural environment. Just a common sense that the real utility of an asset cannot grow exponentially for so long is enough. Sophisticated formulas entered as smokescreen to hide the otherwise obvious lesson.

Unknown said...
This comment has been removed by the author.
James Annan said...

Unfortunately there are always "exceptional circumstances" that allow people to delude themselves into believing that this time is different. In the UK, some of the rise in house prices can be reasonably considered as a structural change in response to the rise of dual income families and a period of sustained low interest rates. The problem is working out exactly how much of the rise is sustainable.

Chris, I think it is fair to primarily blame the govt, as most of the participants were merely acting legally in their own self interest, exactly as a capitalist market dictates (theoretically one can blame the shareholders, but they have little power in practice and have basically lost their money anyway). I am also horrified at the response which seems to me to be throwing (my) good money after (someone else's) bad debts. It may seem very convenient just printing money to deflate all the debts but those who have not been so profligate will end up losing as a result.

EliRabett said...

the basis for a lot of these disasters is that what people think of as a variety of risks turns out to be the same bet. e.g. Long Term Capital (?) collapsed because everything they were doing was basically the same currency gamble. Same for real estate now.

Hank Roberts said...

> most of the participants
> were merely acting legally
> in their own self
> interest, exactly as a
> capitalist market dictates

A pure Chicago School definition of capitalist market there -- but what happens to all the people whose brilliance and self-interest fit that model so well, if the definition of capitalism changes a bit and the niches for them dry up after the recent excursion?

I suspect they're likely to find new roles as predators if no longer considered suitable as top managers.

Alastair said...

The whole point is that you must have regulation (laws i.e infringment of liberty) if you don't want "those bastards to grind down."

It was Adam Smith who came up with the "invisible hand" but he also noticed that if you get a few entrepeneurs together they will soon work out how to fleece the market. Without regulation you get monopolies - Carnegie, Rockefeller (sp.). Bill Gates. No regulation leads to major scams viz. Laws in France, the South Sea Bubble in England, and Buffett in the USA. UNLESS WE HAVE GLOBAL REGULATION TO MATCH A GLOBAL ECONOMY THEN WE WILL HAVE ANOTHER DISASTER. But the "Stupid White Men" in the USA will not agree to that :-(

Cheers, Alastair.