WASHINGTON - National banks must regularly test the accuracy of computer models being used to make loans and manage risks, the Office of the Comptroller of the Currency said Tuesday.
Because an increasing number of bankers are relying on computer-based financial models for credit scoring, asset-liability management, trading-risk management, and other important decisions, the OCC wants them to adopt formal policies for keeping their models up-to-date.
The Comptroller's Office "has observed several instances in which decision-makers either relied on erroneous price or exposure estimates or on an overly broad interpretation of model results, with serious consequences for their bank's reputation and profitability," the agency said in a bulletin to bankers.
"There are many more instances in which the incorrect use of models created the potential for large losses, which were avoided only fortuitously."
Bank policies, which the OCC checks during safety and soundness exams, should include testing computer model results against actual outcomes, auditing the information entered into the model, and comparing predictions against subsequent real-world events as well as against other models.
The agency also expects bank board members to understand the meaning and limitations of a model's results. While decision-makers are not expected to understand the complex formulas upon which the models are built, they should receive plain-English explanations of what the models do, said Jeffrey A. Brown, director of the OCC's risk analysis division.
Mr. Brown emphasized that banks of all sizes should take note of the bulletin.
"This guidance applies to small banks that use vendors to supply models, just as much as to the largest banks" that build their own models, he said.
Small banks, he said, "need to know as much as possible about the products they buy and use, and need to make sure they use them well."
This means bankers are obliged to ask suppliers to prove they have validated the models they sell. The biggest banks have been using computer-based financial models since the 1970s. Now virtually all banks use them in at least one, if not all, business areas.