In September top government officials confronted an excruciating choice. Lehman Brothers, one of America's most important financial institutions, tottered on the brink of bankruptcy. Rescuing it would signal to other large institutions that they, too, could count on an implicit government safety net, no matter how risky their financial excesses. However, allowing it to fail risked setting off a cascade of bankruptcies and market disruptions.

Forced to make a quick decision, officials let Lehman fail. In part, they believed that financial market participants had long been aware of Lehman's weaknesses and had limited their exposure. However, that decision turned out to be a grave misstep. Financial firms worldwide were tightly linked to Lehman's fate by a poorly understood web of derivative contracts. Within hours world credit markets seized up. The economy, already shaky, began its horrifying downward spiral.

Could officials have foreseen the catastrophic fallout? Perhaps, but clearly at the critical moment America's top officials did not have the data needed to fully understand the counterparty relationships linking Lehman to the system. Nor did they have in place any capacity for analyzing the data and thereby forming a clear picture of the consequences of the two alternatives. They were, to a frightening degree, flying blind.

The financial crisis has revealed that regulators do not have access to the data and tools they need to assess counterparty links and monitor financial system robustness. Currently available data provides regulators only a snapshot of a single institution.

Furthermore, the lack of counterparty information enabled the Madoff scandal. It should have been known that Bernard Madoff was making no trades. Clearly, given the vital information that flows from the linkages between firms and markets, this state of affairs is unacceptable.

As a solution, we propose the creation of a National Institute of Finance, which would serve as a national data archive and think tank for the financial regulatory community. The mission would be to provide regulators the data, software, computing power and analytic capacity they need to oversee and safeguard the health of the modern financial system.

A centerpiece of the NIF would be a Federal Finance Data Center. Participants in the U.S. financial and insurance markets would be required to report all positions to the center at regular intervals. Crucially, these reports would include both exchange-traded and over-the-counter contracts, complete with counterparty relationships. The center would let regulators assess systemwide contagion and concentration risks, perform stress tests, including the impact of the failure of a large institution, and hence make better-informed decisions in times of crisis.

Given the sensitivity of this data to financial institutions and, in broad terms, America, top-level data security would be essential. Financial institutions would have serious, legitimate concerns about third parties learning details about their positions. But the military and national security communities provide functioning models for handling such problems. More to the point, there is no alternative. Without detailed information on transactions, positions and counterparty relationships, any attempt to identify systemically important institutions is guesswork.

A second major role of the NIF would be to maintain a research center, including a National Risk Lab with skilled staff and facilities analogous to those used by the quants on Wall Street. In the spirit of the National Academy of Science, the NIF would act as a clearing house for ideas and problems between industry and a broader research community — identifying essential problems, sponsoring the sustained research efforts needed to solve these problems and integrating solutions into a rigorously tested, well-understood set of models.

This work would allow the NIF to work like a product safety lab, providing a widely available set of models as a basis for risk assessment and independent evaluations of models used by industry for computing capital requirements. To minimize regulatory turf battles and the risk of politicizing analysis, the NIF should not have any direct line responsibility for the oversight of financial institutions. Instead, it would act as a neutral player, supplying tools to the entire regulatory community.

Without detailed data and validated decision tools, crisis decisions will continue to be based on guesswork and intuition. With the creation of a National Institute of Finance, a new financial regulatory regime would be equipped with the data and tools to make better decisions — before the next crisis or Madoff comes.

Subscribe Now

Access to authoritative analysis and perspective and our data-driven report series.

14-Day Free Trial

No credit card required. Complete access to articles, breaking news and industry data.