The story of VaR

It's impossible to prepare for a hurricane if you've only experienced -- and can only imagine -- a rainstorm. That's the gist of Joe Nocera's article in the Times Magazine today. More specifically, it's about Wall Street's reliance on VaR -- Value at Risk -- statistical models that are supposed to sniff out risk, but which instead became a crutch for management to take riskier and riskier bets even though those statistical models are less than perfect, to say the least.

“Why do people measure risks against events that took place in 1987?” he asked, referring to Black Monday, the October day when the U.S. market lost more than 20 percent of its value and has been used ever since as the worst-case scenario in many risk models. “Why is that a benchmark? I call it future-blindness.

“If you have a pilot flying a plane who doesn’t understand there can be storms, what is going to happen?” he asked. “He is not going to have a magnificent flight. Any small error is going to crash a plane. This is why the crisis that happened was predictable.”

Eventually, though, you do start to get the point. Taleb says that Wall Street risk models, no matter how mathematically sophisticated, are bogus; indeed, he is the leader of the camp that believes that risk models have done far more harm than good. And the essential reason for this is that the greatest risks are never the ones you can see and measure, but the ones you can’t see and therefore can never measure. The ones that seem so far outside the boundary of normal probability that you can’t imagine they could happen in your lifetime — even though, of course, they do happen, more often than you care to realize. Devastating hurricanes happen. Earthquakes happen. And once in a great while, huge financial catastrophes happen. Catastrophes that risk models somehow always manage to miss.

In fairness, the investment banks' risk managers considered 99% of probable outcomes, and as one put it, if you only manage against the remaining 1%, you quickly go out of business.

But what I found shocking was that the VaR models had historical inputs that only went back a few years...in other words, the height of the housing bubble.

VaR uses this normal distribution curve to plot the riskiness of a portfolio. But it makes certain assumptions. VaR is often measured daily and rarely extends beyond a few weeks, and because it is a very short-term measure, it assumes that tomorrow will be more or less like today. Even what’s called “historical VaR” — a variation of standard VaR that measures potential portfolio risk a year or two out, only uses the previous few years as its benchmark. As the risk consultant Marc Groz puts it, “The years 2005-2006,” which were the culmination of the housing bubble, “aren’t a very good universe for predicting what happened in 2007-2008.”

This was one of Alan Greenspan’s primary excuses when he made his mea culpa for the financial crisis before Congress a few months ago. After pointing out that a Nobel Prize had been awarded for work that led to some of the theories behind derivative pricing and risk management, he said: “The whole intellectual edifice, however, collapsed in the summer of last year because the data input into the risk-management models generally covered only the past two decades, a period of euphoria. Had instead the models been fitted more appropriately to historic periods of stress, capital requirements would have been much higher and the financial world would be in far better shape today, in my judgment.” Well, yes. That was also the point Taleb was making in his lecture when he referred to what he called future-blindness. People tend not to be able to anticipate a future they have never personally experienced.

Yet even faulty historical data isn’t Taleb’s primary concern. What he cares about, with standard VaR, is not the number that falls within the 99 percent probability. He cares about what happens in the other 1 percent, at the extreme edge of the curve. The fact that you are not likely to lose more than a certain amount 99 percent of the time tells you absolutely nothing about what could happen the other 1 percent of the time. You could lose $51 million instead of $50 million — no big deal. That happens two or three times a year, and no one blinks an eye. You could also lose billions and go out of business. VaR has no way of measuring which it will be.

What will cause you to lose billions instead of millions? Something rare, something you’ve never considered a possibility. Taleb calls these events “fat tails” or “black swans,” and he is convinced that they take place far more frequently than most human beings are willing to contemplate. Groz has his own way of illustrating the problem: he showed me a slide he made of a curve with the letters “T.B.D.” at the extreme ends of the curve. I thought the letters stood for “To Be Determined,” but that wasn’t what Groz meant. “T.B.D. stands for ‘There Be Dragons,’ ” he told me.

And it gets murkier still as the VaR models could be gamed by traders.

VaR DIDN’T GET EVERYTHING right even in what it purported to measure. All the triple-A-rated mortgage-backed securities churned out by Wall Street firms and that turned out to be little more than junk? VaR didn’t see the risk because it generally relied on a two-year data history. Although it took into account the increased risk brought on by leverage, it failed to distinguish between leverage that came from long-term, fixed-rate debt — bonds and such that come due at a set date — and loans that can be called in at any time and can, as Brown put it “blow you up in two minutes.” That is, the kind of leverage that disappeared the minute something bad arose.

“The old adage, ‘garbage in, garbage out’ certainly applies,” Groz said. “When you realize that VaR is using tame historical data to model a wildly different environment, the total losses of Bear Stearns’ hedge funds become easier to understand. It’s like the historic data only has rainstorms and then a tornado hits.”

Guldimann, the great VaR proselytizer, sounded almost mournful when he talked about what he saw as another of VaR’s shortcomings. To him, the big problem was that it turned out that VaR could be gamed. That is what happened when banks began reporting their VaRs. To motivate managers, the banks began to compensate them not just for making big profits but also for making profits with low risks. That sounds good in principle, but managers began to manipulate the VaR by loading up on what Guldimann calls “asymmetric risk positions.” These are products or contracts that, in general, generate small gains and very rarely have losses. But when they do have losses, they are huge. These positions made a manager’s VaR look good because VaR ignored the slim likelihood of giant losses, which could only come about in the event of a true catastrophe. A good example was a credit-default swap, which is essentially insurance that a company won’t default. The gains made from selling credit-default swaps are small and steady — and the chance of ever having to pay off that insurance was assumed to be minuscule. It was outside the 99 percent probability, so it didn’t show up in the VaR number. People didn’t see the size of those hidden positions lurking in that 1 percent that VaR didn’t measure.

The whole thing is worth a read for the view it gives to the systemic failure of management to exhibit the imagination needed to see the storm coming, even as they had to know they were over-leveraged and the housing market had ballooned to zeppelin-like proportions, and they had the experience of LTCM, the hedge fund that relied considerably on VaR and whose collapse nearly melted down the financial markets in 1998.

UPDATED to get rid of that strange floating question mark.