Trusting Banks' Own Capital Models Can Be Dangerous

I am very pleased that Basel Committee Chairman Stevan Ingves announced recently the committee will propose new limits on banks' discretion to use internal models to assess capital strength.

As I have written extensively, large banks — typically with more than $250 billion in assets — are allowed by national regulators to create opaque credit, market and operational risk models that hide assumptions and inputs from most people at the bank, not to mention market participants. The success of these models is critical in determining the right amount of regulatory capital to sustain losses.

Risk modeling as a concept for measuring banking capital has serious problems at its core.

In an excellent paper, "Math Gone Mad," Durham University professor Kevin Dowd criticized the Federal Reserve's use of modeling in stress tests. He argues that models developed from within a group become irrational. "Most risk models, regulatory risk models, in particular, are textbook examples of the ritualistic fetishes associated with primitive tribes," he wrote. "A fetish can be described as irrational attachment to an object — in this case, a risk model — regardless of its true usefulness. This is to treat the models as if they were ritual implements with magical properties and is the very essence of superstition."

Calculating adequate capital based on credit, market or operational risk models focuses on three components. There are inputs comprised of data — if it exists — and assumptions. Then, a processing mechanism transforms the inputs into estimates. Finally, a reporting element translates the estimates into useful information for business executives and regulators.

Regulatory capital models could be very useful to market participants, the media and regulators, if the methodology used for all three components were disclosed uniformly and in a timely manner to the public. Unfortunately, we are very far from having such market transparency, and that is something agencies like the Securities and Exchange Commission should really correct. Stanford professor Anat Admati correctly argues that banks' opacity makes it difficult to be reassured by regulators' stress tests, which rely a lot on banks' data and calculations.

Having worked on Basel models for over a decade, I recommend that auditors, examiners and journalists need to question bank's ability to define, gather and process data used in models uniformly and consistently. Banks regularly lack complete data for measuring all risks, fail to store data in a centralized place where it can be validated and knowingly ignore data that shows the need for more capital.

The quality of big bank systems continues to be a significant problem. This tends to happen after multiple mergers and acquisitions, which brings together people from different corporate cultures, who exhibit playground behavior of hoarding information and not sharing it with other relevant groups inside the bank. This silo mentality is detrimental to creating a good and relevant model. Sometimes the problem with weak systems actually comes from management, such as in Deutsche Bank, which had a strategy of pitting people against each other and created an information system infrastructure that would make Kafka proud.

In recent Basel Committee and private-sector surveys, 50% of global systemically important banks say that they cannot rely on their systems to give them a relevant view of their total risks, particularly in a period of stress. The other 50% need another two to three years to comply with the Basel Committee's risk data aggregation principles, which become effective in January. If internal auditors and bank examiners cannot trust banks' data, it is not possible for anyone to trust banks' risk models to tell the market and regulators whether a bank is well capitalized.

Additionally, market participants and bank examiners need to find out what types of assumptions are being used in models and where they come from. A plethora of incorrect assumptions and methodology can occur in modeling. One big problem is that having diversity of thinking at banks is rare. This herd mentality in the industry is what leads to thinking such as "home prices do not ever fall nationally at the same time." A challenge in this area is to determine how often traders or bosses interfere with modelers when they do not see results they like.

Lastly, banks need to disclose whether the models that they create are really being used for their intended purpose. It is not unusual that once models at a bank are established, they start being used for products or markets that may not be relevant. Regulators need to be vigilant to prevent models from being misused to cover up losses.

Despite not being able to see how risk inputs are determined, numerous bank analysts, lobbyists and journalists continue to proclaim that big banks are well capitalized. It is one thing to be mesmerized by magic and superstition at a party or in an anthropology class, but it is another to let them determine how much taxpayers are on the hook if banks are insufficiently capitalized.

Mayra Rodríguez Valladares is managing principal at MRV Associates, a New York-based capital markets and financial regulatory consulting and training firm.