Agencies Beef Up Ability to Evaluate Risk Models

When the risk manager from KeyCorp met the risk management expert from the Office of the Comptroller of the Currency, the tension was palpable.

"At first it was like a couple of bulls stomping and lowering their heads and getting ready to attack each other," said Kevin M. Blakely, KeyCorp's executive vice president for risk management. "Our guy didn't know what to expect from the OCC. But over the next two or three weeks, they built up a mutual respect."

Similar encounters are taking place in banks nationwide as regulatory examinations increasingly focus on risk management practices. Bankers, proud of their risk management skills, are fast finding that regulators know more than a few tricks of the trade.

The OCC, for its part, has assembled a team of 11 PhD economists to help examiners evaluate the complex mathematical equations that banks use to size up risk.

"Examiners are not trained as quantitative economists, but they are dealing with bankers who are creating and relying on very sophisticated models," said Jeffrey A. Brown, who heads up the team. "Our role is to help our examiners understand what the bank is doing so they can make a judgment about the risks."

In this article and two to follow, the American Banker takes stock of risk management, a practice that is revolutionizing the way banks are managed and supervised. Reflecting the leading role that regulators are playing, today's report spotlights how the OCC is equipping its examiners for the new world.

Headed by Mr. Brown, the team of economists started examining risk models used by the 10 largest national banks nearly three years ago and plans to expand its scope to the top 50 this year. Credit risk models at a handful of community banks have been examined as well.

The economists spend two to three weeks participating in an exam, focusing on whether a bank's models work.

"We quiz them about where they get their data, how it is processed, how often they update it," said P.C. Venkatesh, an economist on the team.

Mr. Venkatesh, 43, typifies the level of expertise on the team, known as the risk analysis division.

After earning a PhD in finance from the University of Florida in 1983, he spent eight years teaching at the University of Houston. In 1991 he joined the Commodity Futures Trading Commission as a research economist.

"It wasn't a very hard sell to get me to come over here," he said. "This place is far more interesting-there was certainly no travel at the CFTC and not much going out and meeting people, like there is at this job."

OCC employees say it was essential to bring in people with Mr. Venkatesh's experience.

"As markets developed and got more complex, we felt we needed additional expertise to assist us," explained Alfred P. Crumlish, an OCC examiner assigned to a money-center bank. "The risk analysis division adds a great deal of credibility and integrity to the exam process."

The other banking agencies have taken steps to improve their understanding of risk models but have yet to dedicate a stable of economists to the task.

The Federal Reserve Board has capital markets examiners who are specially trained to evaluate risk models and are called in occasionally to help out on exams, said Michael Martinson, the Fed's assistant director of international supervisory policy.

"We use the personnel we find necessary to check the validations," he said, "and if that requires a specialist coming in to the exam, we have them in every reserve bank."

The institutions supervised by the Federal Deposit Insurance Corp., which generally are smaller than those the Fed and OCC oversee, are primarily exposed to interest rate risk. FDIC examiners are equipped with a basic understanding of interest rate risk models. However, scrutiny of a bank's model is only triggered when an institution faces significant exposure in this area.

In simple terms, a risk management model is a set of mathematical equations. Market variables that affect a portfolio, such as interest rates or stock prices, are plugged into the equation. The results show how a portfolio would perform under different market conditions, enabling the banker to measure risk in a portfolio.

"Models are a simplification of reality," Mr. Brown said. "They allow you to simplify a complicated world and get your arms around a problem."

The most common model predicts how fluctuations in stock and bond prices would affect a bank's securities portfolio. In addition, banks rely on models to measure interest rate risk. Increasingly, banks-especially small institutions-are using models to predict risks in loan portfolios, Mr. Brown said.

"Credit risk is the area we are seeing more and more growth in," he said. "Banks have begun using these models in making small-business loans."

All models must be proved valid.

"If you have a sophisticated model you've built yourself or had your own people modify an external vendor model, you need to have it validated by outside auditors," said Ronald H. Frake, an OCC capital markets examiner stationed at a New York bank.

The OCC economists generally do not validate the models themselves. Instead, they scrutinize the results of tests run by an outside auditor, frequently a risk specialist from an accounting firm.

"The division has been extremely useful in evaluating the quality of that external validation," Mr. Frake said.

Validation generally comprises three elements. First, the outside auditor plugs into a test model the same data the bank has used in its internal model. While the results may differ, an institution's model passes the test as long as bank officials can explain the discrepancies.

"I'm not looking for numbers that are close but whether the risk managers are able to explain where the differences comes from," Mr. Brown said. "That tells me they understand the workings of the model."

Second, auditors compare the bank model's predictions with actual portfolio performance. This is called back-testing.

Finally, auditors test the bank's model under stressed market conditions. For example, an auditor might examine how a model reacts if all the prices in a bank's securities portfolio move in the same direction at the same time.

A common problem, according to Mr. Frake, are models that haven't been tested rigorously enough.

"Often they'll need to use more vigorous stress scenarios to see at what point those models or underlying assumptions may start to break down," he said. "That's a fairly common recommendation we'll make to bankers."

OCC economists also find "intellectual risk"-or too few people at the bank who understand how a model works, Mr. Brown said. Having only one person who comprehends the ins and outs of a model is not enough, he said.

"Over time, you want to have policies so that if the modeler who built this tool departs, you're not completely exposed," he said.

In addition, banks' boards of directors do not always get as much information as they should from their risk managers, Mr. Brown said.

"A part of our task is making sure the bank is communicating its own risk internally," Mr. Brown said. "It's really difficult to communicate the results of a complicated modeling exercise to the people who are ultimately responsible for managing this risk-the board."

For reprint and licensing requests for this article, click here.
MORE FROM AMERICAN BANKER