Matt is right to say that a lot of what passed for "financial innovation" was in fact innovative efforts at regulatory arbitrage. One of the pieces of the crisis that I hadn't understood until recently, for instance, was the role that Basel II banking regulations played in the growth of the structured securities market. In essence, Basel II, which went into effect a couple years ago, held that a bank only had to keep half as much capital on hand for AAA-rated securities as for other types of assets. That created a huge incentive for banks to get more things rated AAA, as it meant that they didn't need to have as much money sitting around. They could spend it buying more products that offered higher returns. They could, in other words, further leverage themselves.

Cue "financial innovation." Structured finance -- where you separate assets into different parts (tranches), and make the "senior" tranche pretty safe by making the other tranches pay out losses first -- was essentially a way to build a lot of AAA-rated securities. Churning out these AAA-rated securities became the primary job of the ratings agencies. From CKE:

According to Fitch Ratings (2007), roughly 60 percent of all global structured products were AAA-rated, in contrast to less than 1 percent of the corporate issues. By offering AAA-ratings along with attractive yields during a period of relatively low interest rates, these products were eagerly bought up by investors around the world. In turn, structured finance activities grew to represent a large fraction of Wall Street and rating agency revenues in a relatively short period of time. By 2006, structured finance issuance led Wall Street to record revenue and compensation levels. The same year, Moody’s Corporation reported that 44 percent of its revenues came from rating structured finance products, surpassing the 32 percent of revenues from their traditional business of rating corporate bonds.

Italics mine. The financial innovators acted as you would expect them to. A sudden demand for AAA securities was met with a sudden burst in innovation resulting in a lot more AAA securities. It was not matched with regulators who evinced an understanding that the process had broken down and the sudden surge of AAA-securities was a demonstration of mispriced risk rather than proof that the world had become less risky. The wisdom of the regulators did not keep pace with the innovation of Wall Street.

This, incidentally, strikes me as a tough story for liberals. The regulation in question was pretty straightforward: If you have more safe assets then you have less risk and can use more of your capital. The breakdown was first in the ratings agencies, who began mispricing assets, and second among the regulators, who didn't particularly notice.