- Key insight: Banks' use of generative AI models escalates their need for high-quality data.
- What's at stake: Garbage in-garbage out takes on more meaning when generative AI models absorb erroneous or faulty data.
- Expert quote: "AI, absent well-organized, well-curated data, is nothing more than interesting math on a whiteboard."—Ned Carroll, head of data and automation at PNC
Like any AI model, a generative AI model is only as good as the data it's fed. But how do you know you have good data?
Some bank technologists are answering this question with the concept of a "nutrition label." A workgroup of the Financial Services Sector Coordinating Council created a
Data quality has always been an issue for financial institutions, especially in their use of AI models. In 2011, bank regulators published guidance, called SR 11-7, on managing, monitoring and validating AI models. In 2012, JPMorganChase CEO Jamie Dimon's pay was cut in half because of a $6 billion trading loss that stemmed partly from a model's use of faulty risk-management data.
Generative AI accelerates the risks of mismanaged or mis-sourced data. A famous example is Air Canada's virtual assistant, which gave a customer a wrong answer about a bereavement discount – the customer sued and won in a Canadian court.
"AI, absent well-organized, well-curated data, is nothing more than interesting math on a whiteboard, and that goes for both traditional machine learning and generative AI," Ned Carroll, head of data and automation at PNC, told American Banker.
In the past, banks have used deterministic models, where "there's a ground truth," Carroll said. "That ground truth is oftentimes the function of the structured data that's being used and the math and statistics you do around it."
Probabilistic generative AI models lack ground truth, he said. For these, PNC has come up with a separate policy.
"We got very deliberate about not conflating how we look at model risk management to how we look at gen AI risk management because they're different," he said. The heightened risk means generative AI "elevates the demand for sound data management practices," Carroll said.
Data nutrition label
Carroll worked on the Financial Services Sector Coordinating Council's data nutrition label, which is designed to help ensure that the data AI models are being fed meet minimal requirements.
"The whole notion of the nutrition label was this: How do I attribute a notion of goodness to a supply chain?" Carroll said. "That goodness could be a function of a number of different variables in terms of inputs, timeliness, quality, currency. Goodness may also be very much dependent on what am I using it for."
The level of data quality needed for a model generating marketing offers, for instance, is generally lower than what's needed to approve a loan.
"That's clearly going to be a much higher standard because I'm about to expose risk," Carroll said. "I'm going to have a higher bar on the underlying nutritional value, if you will, of that data, and I think that's where the metaphor has been really useful."
In the nutrition label scenario, the owner of the data attests to its quality. So where a bank relies on credit bureaus, market data providers and other data vendors, it would call on those vendors to attest to the accuracy and timeliness of the information.
"If you don't have clear accountability, your ability to manage quality around it is always going to be compromised," Carroll said.
Frontier models like ChatGPT and Claude have been trained on all of the internet, and therefore, they generally lack data quality controls.
But the way PNC and some other banks have engineered their use of these models, they add their own knowledge and context to them, Carroll said.
"This is about us owning our intelligence," he said. "That's not for anybody else to own. OpenAI and Anthropic would love for us to open up all of our data to them and let them train their models on it. But that's our proprietary knowledge. Our IP is embedded in our data, whether that's a policy or a procedure or unstructured data sources."
The data nutrition label provides a way to communicate data quality requirements, internally and with third parties, Carroll said. Model providers internal and external can be asked, "did you use authoritative data? What was the nutrition of the data that went into your model?" Carroll said. "That sets behavior, and how you establish expected behavior, how you motivate behavior, how you manage behavior."
Read more:
- OCC and FDIC finalize narrower bank supervision procedures
- As education costs rise, credit unions chip in for employees
- A $111 million scheme to keep bad merchants banked
- How Google and Apple's youth plays are pressuring banks
It establishes the behavior expected of data scientists and modelers, he said.
"In many respects, it's a metaphor to make a topic that, quite honestly, has sort of fallen flat historically and make it meaningful and useful," Carroll said.
Ian Schnoor, executive director of Financial Modeling Institute, supports the idea of a nutrition label.
"I do like the idea of transparency of disclosure," Schnoor told American Banker. "Food labels are great. I read them myself."
But he pointed out that food nutrition labels are sometimes open to misinterpretation. He gave the example of a cooking spray that listed its calorie count as zero. It's true that if you spray for a quarter of a second, the calories would be zero. "But if you actually sprayed enough to fill a cup, it would be thousands of calories. And yet, the world sees that cooking sprays have zero calories."
So labels are "positive but subject to interpretation and manipulation," Schnoor said. "It's not a foolproof be-all, end-all, but it's a good start to transparency and protecting investors and protecting the integrity of deals."
