Barry Eichengreen has a long piece in the latest National Interest arguing that economics didn't miss the financial crisis so much as the relevant folks engaged in "a partial and blinkered reading of [the] literature" in order to ignore results that would cut against them making more money. As the old saying goes, it is difficult to get a man to understand something when his income depends on his not understanding it.
But Eichengreen looks towards the future with hope: Bigger computers with more memory will mean more economics based on large data sets and less theory. "The top young economists are, increasingly, empirically oriented," enthuses Eichengreen. "They are concerned not with theoretical flights of fancy but with the facts on the ground. To the extent that their work is rooted concretely in observation of the real world, it is less likely to sway with the latest fad and fashion."
I'm less convinced by this. You can pick and choose data sets as easily as you can pick and choose theories. The ratings agencies and the investment banks, for instance, had tremendous data sets. They had housing prices over decades. They had all the risk data from the credit default swaps. They had massive models with millions of data points all describing things that had happened in the past but did not accurately predict what the future was about to bring. And these data sets, it seems, made them more confident than they should have been. It's easy to argue with theory. Hard to argue with historical trends. Housing prices, as we now know, just don't go down.
But that's largely what you get from data sets. You have to collect the data, after all. Which means you're arranging information that has already happened. And that means you're using the past to inform predictions about the future. In general, that works out pretty well. We're in a moment, however, when it didn't work out all that well. This seems like the sort of period that makes you more skeptical of strong conclusions drawn large data sets rather than very confident that human researchers will develop substantially less fallible methods of analysis. The lesson, from where I sit, is that there will always be claims of better math and more perfect data sets, but that the more outlandish conclusions that follow from those claims -- the end of unpredictable risk! the financial sector should be 50 percent of national profits! -- should be treated with substantial skepticism.