Creating The Right Foundation and Architecture for AI

As financial institutions navigate complex regulatory requirements and prioritize data security, the right foundation and architecture ensures they can fully exploit AI technologies while safeguarding sensitive information. What are the enabling architectural principles of supply chain data? Join us to learn strategies for building an adaptive, intelligent banking ecosystem with generative AI at its heart. 

Transcription:

Penny Crosman (00:11):

All right. Welcome everybody. Thanks for coming. We often talk about the fact that your data needs to be really good for AI to work properly. The data has to be accurate, it has to be real time, it has to be clean, it has to be in compliance with data privacy law and cybersecurity norms. But how do you do all that? I rarely find people tell me exactly how to do it, how they do it. So today we're going to kind of get in the weeds on that, on dealing with data, getting it right, getting the foundation right for AI and other ways. And we have two great experts here. We have Ned Carroll with PNC. He oversees. How would you describe your role exactly?

Ned Carroll (01:05):

I oversee data and automation, which is sort of our broader umbrella for AI and such.

Penny Crosman (01:11):

Okay, great. And we have John Ratzan at Accenture. And how would you describe your role at Accenture?

Dr. John Ratzan (01:17):

I lead up our data and AI across the United States for FS.

Penny Crosman (01:22):

Okay. So our first question, what are some of the top challenges that banks face? They want to start using generative ai, they're using more embedded AI through all other applications. What are some of the biggest data challenges that you see starting with you, Ned?

Ned Carroll (01:42):

Well, I think one of the big challenges that we all face is just how do we ensure control of our data? How do we understand the universe of our data? I think starting out of the credit crisis, I sort of think about data quality and sort of three forcing events. There was the credit crisis, which forced us to think very much about specific attributes that had implications to capital liquidity, so forth and so on. Structure data. The second forcing event has really been the cloud, which is saying, I want to be able to run models against a much broader set of structure data. And this third forcing event is really Sam Altman, which is I now want to be able to look at unstructured data as well in ways that we've never thought of. I think the industry writ large, and not just financial services, but really across industry, we've struggled with how we manage the quality of structured data.

(02:46):

And there are lots of reasons for that struggle. It's an architecture issue, it's a governance issue, it's a technology has changed and it's been an incremental build on it versus cleaning up the past. But as an industry, we've struggled greatly with how do you scale data management around structured data? So now you start thinking about, well, how do you scale it around unstructured data? The types of statistical metrics that we look at for in terms of quality and consistency of structured data, they don't apply to unstructured data. So I think we're still learning as an industry around how do you better manage that unstructured data? What does data quality mean from an unstructured data perspective? I know I get that question from regulator. What does it mean to think about data quality for structured data versus unstructured data? Critical data versus noncritical data and the universe that we have to have care of is much broader than it was when we first as an industry started talking really aggressively about data quality, which goes back to the 2008 credit crisis.

Penny Crosman (03:56):

And when the regulars ask you about data quality in your data universe, why are they asking you that? What are they trying to accomplish?

Ned Carroll (04:06):

Well, I think number one, safety and soundness. So is the data that you're using that impacts your capital or your liquidity understanding your counterparty risk. So first and foremost, safety and soundness and then I think right now has evolved to customer harm. I think from an OCC Fed perspective, the mandate has historically been around safety and soundness. I think with the CFPB coming in under different administrations, you had a different focus on specific customer harm and that changes the game a bit, right? When I think about safety and soundness, I can think about materiality, alright, here's an issue, but does it really have material impact to a capital calculation? When I think about customer harm, now you're like, okay, Ned Carroll's address was wrong. You did something bad, or you use data about Ned Carroll that he did not consent to use. So your requirement around management, in my opinion, has become far more granular versus when you thought about it from a safety and soundness and materiality perspective.

Penny Crosman (05:26):

So a lot of it is being a good custodian of the data and making sure it's accurate and making sure it's protected and that sort of thing. How about you, John? What do you think are the biggest challenges for data challenges for banks that want to incorporate more AI into their system?

Dr. John Ratzan (05:44):

Well, it's all about big data. The data proliferation and the tool proliferation has been amazing In the past 20 years. We used to think of data as just data exhaust, and now we're able to harness data for revenue opportunities, efficiency opportunities. So the one thing that we've noticed across the industry is people are still using, banks specifically are still using mainframes and legacy data environments. And while all the new capabilities are unveiling such as the latest BI tools, the latest storage tools, all of the old ones are still there. I mean, by show of hands, who still has Teradata or mainframe? It's a hundred percent. And so I think one of the biggest challenges is for CFOs and CIOs looking at the budget, is as we introduce these new tools, what do we do with the old tools? And you commented briefly on gen AI in terms of what gen AI can bring us. So far what we've seen is really just operational efficiency and productivity. I'm looking forward to making sure that all those data underpinnings are established such that we can actually create new products and services with Gen AI and AgTech. So those are a few key points.

Penny Crosman (07:09):

So when you talk about old applications, old mainframes, old systems, what are some of the dangers of that? Are the dangers around hackers getting in? Are the dangers around data being out of date and a generative AI model pulling something that's five years old and no longer accurate? What are some of the things that you worry about?

Dr. John Ratzan (07:31):

I don't think it's necessarily danger per se as much as financial cost and moving slowly in terms of what your customers and clients need. We just ran one project the other last year. The client wanted to know how many times are we storing just address, right? And it turned out that it was nine times across different systems. So in terms of the danger might manifest itself in terms of, okay, those marketing materials are going to a mailbox that doesn't exist anymore. That's cost in terms of the MIPS that are running on the mainframe or the license fees that you're paying for legacy storage capability is too much. And then the ability to move quickly and nimbly to meet customer needs is also constrained with those environments.

Ned Carroll (08:27):

I mean, to John's point, the way I think about the perils of replication is simply put friction and all of this sort of additive architecture. We had mainframes, we had Kline server, we had big warehousing, then appliances, then on-prem, Hadoop, then cloud. To John's point, they've all been additive in some respects, but now we're sort of doing full circle and saying, Hey, I can now do model inference on the same chip that's on a mainframe. So when I think about friction, friction presents itself in this concept of, for us, one of our key measures is around velocity. So how quickly can I get to insight? How quickly can I get to discovery? Once I get to insight off of discovery, how quickly can I put it into production and how do I start measuring and monitoring that cycle time to do it? Because that friction becomes real in terms of expense.

(09:33):

That friction becomes real in terms of opportunity cost. Let's say I've got this great new idea to build a model and I leveraged a capability that Accenture helped us do, which is ML ops in the cloud. But then it takes me eight months to productionize it and it takes me eight months to productionize it because the language that a modeler developed it in is not the language that my development team wants to productionize it in. Maybe I can only do it in a near real time inference versus a real-time inference. So I could come up with the best thing since sliced bread, but if I can't get it to a customer, if I can't get it to employee, who cares? And I think in some respects we're still in that. I can come up with the greatest thing since sliced bread, but if I can't get it into the hands of our employees who are interacting with our customers who are managing risk, it's interesting, but it's not particularly economically valuable.

Penny Crosman (10:36):

Alright, so you guys have mentioned a lot of different challenges to do with data management, et cetera. What are some of the things that banks are and should be doing, like a PNC? What are some of the steps you're taking to try to get a handle on this?

Ned Carroll (10:55):

I think we've very much taken what I would characterize as a guardrail based approach. So our innovation is resident in our 60,000 employees. Our innovation is resident in our 16 million plus customers. That's where innovation exists and it's understanding their needs, it's anticipating their needs, it's learning about their needs. So for us, we want to open up that and enable that innovation. Now in order to do that, I've established guardrails. So they're really sort of what are those key considerations as I'm thinking about ai. One is what data are you using? Is it internal non-proprietary? Is it proprietary, confidential? Is it extra? What data are you using? Second, who's consuming the ai? Is this an employee doing prompts? Is it a customer doing prompts? What's third? What's the level of sophistication? Are you doing simple summarization? Are you creating content? Fourth, who uses it? Is it going directly to a customer? Is it going to an employee? And then finally, what are my requirements around observability?

(12:22):

I believe the real opportunity in unleashing AI is how do you scale its governance? And to me that's about almost applying AI to AI, right? So how do I think about observability? How do I observe the prompts that are going in? How do I detect prompts that are concerning to me or prompts that I want to exploit? How do I better observe outputs? And are those outputs within the guardrails of acceptable use and an intent that we have? So we're very much taking sort of a guardrail approach and then letting innovation flourish within those guardrails.

Penny Crosman (13:00):

So it sounds like you're taking kind of a risk-based approach to creating policies for each AI model and the data that's going to go into it and what's going to happen to it and how, et cetera. If I'm hearing,

Ned Carroll (13:12):

You right, I wouldn't say a policy for each model. I think it's more a policy. Number one, a lot of AI policy has existed for years. I mean, one of the most mature industries in model risk management is financial services. So everybody, AI is not new. I mean, I did AI in college 30 plus years ago. There's a lot of difference though. The difference is my data set isn't something I need to manually create. The difference is I don't need to know the math natively. And the difference is I don't need to get up at three o'clock in the morning to run it in the computer lab because my college experience predates laptops. So these types of capabilities have existed and the underlying academic skills to do it of math and computer science have existed, but now it's more open. So you need to think about more scalable ways to govern it.

Penny Crosman (14:18):

So John, one of the things that I think is an interesting challenge is that is the idea of data provenance. Where's this data coming from? And I think in a lot of banks you have, we were talking about replication duplication before, you have many different versions of policies lying around and you have different rate sheets and you have different customer data sets that overlap with each other that have slight differences to them. And an interesting example that I mentioned before is the Air Canada example where passenger communicated with an Air Canada chatbot. He wanted to visit his grandmother, no, didn't want to visit his grandmother died. He wanted to go to her funeral in Vancouver. So he wanted to get tickets and he wanted to know if he could apply the bereavement, the airlines bereavement policy after getting the tickets. And the chat bot said, yes, that's fine.

(15:16):

You have I think nine months after your flight. And then when he tried to apply for the bereavement discount, he was told, no, that's not in our policy. And he took them to a Canadian court and actually won. And the Canadian, I forget the word, but it's not a judge, but it's like they have a person who adjudicates these cases and they said that Air Canada was responsible for the information that the chatbot put out. Even though the chatbot delivered an answer that had a link to the correct answer, the adjudicator felt that the chatbot was still wrong. And chatbot is part of an, it's an extension of the bank. So when you, that's just one example, but how can banks be sure that if they use generative AI, either internally or customer facing, that it's going to actually get the correct piece of information that applies to that situation?

Dr. John Ratzan (16:21):

Well, you can't be sure for sure. It's a probabilistic system. We all actually are searching for how can we make AI more deterministic, which is a fallacy from its beginning, but there are techniques you can put in place to try to become as close to deterministic as you can. First is rag everybody by this time in the gen AI journey a couple of years later are familiar with the retrieval augmented generation and constraining what actually the LLMs can do. Second is fine tuning again, stating some of the obvious for those practitioners that are in this space. And then we've seen an advent, again, another buzzword of small language models and more focused. So we've recently just worked with a bank to implement something in the finance space. And so what it actually does is works with the large language model for the natural language processing, but then it works with the small language model because it understands more intricately what that particular bank calls its forecast.

(17:38):

Because all banks call their financial terms something ever so slightly different internally. And the small language model will have that context where the large language model could actually hallucinate quite badly in those scenarios. You said forecast. What does forecast actually mean to us that understand humans still understand the context? And so we can read the paragraph around it and know that that means the forecast to be submitted for the financial report where a large language model might perceive forecast to mean the weather forecast. So going back to the Air Canada example, one of the things that is paramount in rolling out the controls and governance for gen AI is understanding the permutation. So I've not seen yet very few examples what I would call in the wild where you have open data from the inside of a company and then you have prompts, which is basically a factorial example of enumeration where you have so many possibilities and permutations and so it's imperative to constrain the data that the models will go after and then also have some input filtering on the prompts. So those are a couple of techniques that you can use to try to make it more secure.

Penny Crosman (19:16):

Ned, what do you think about small language models? Are they part of the answer?

Ned Carroll (19:21):

I'm very bullish on small language models principally because if you think about it, what we're solving is language problems like interpretation problems. And as John pointed out, industries have different languages. So in financial services we'll talk a lot about the corporate reference data model. We'll talk about the buy-in model as a way to drive object oriented development. So those are two very important pieces of context that I can add to a language model and then fine tune it. PNC has its own language that is different than the language I used when I was at TIA. It's different than the language that I used when I was at Bank of America. So I think the very nature of the problem statement, which is it's a language problem, it really plays into how can you use small language models. I think like everything, it's around price, efficacy and latency. That language model, a small language model. If I want to create gen AI capabilities for how I do data management and lineage, that makes a lot of sense because there's very specific context in fine tuning because the way in which we've used ETL platforms and APIs and streaming is different than the way, let's say John deployed that at a client of his. So that language becomes really important. How I develop, how we develop at PNC is different than how our developers developed at TIA. So introducing that context and fine tuning is really important.

Penny Crosman (21:07):

Alright, in a minute we'll open up to questions, but I think the title had Creating the right foundation for AI in it. So what does that look like? What does the right foundation for AI look like and how do you build it? Let's start with you John.

Dr. John Ratzan (21:22):

Well, the Wright Foundation at current state is what we note as data swamps. How do you get the data swamp to be cleaner? I think smaller banks do have an advantage in that, that they may not have all the legacy debt of merging. All the big banks that I've worked with have that challenge. So I think coming up with a pristine environment with a source of truth and then consumption layers on top of that is what everyone's going for that at speed can then enable new products and services.

Penny Crosman (22:03):

And in a large bank, does that become a separate data layer or is it a matter of integrating what's already there?

Dr. John Ratzan (22:11):

It still remains to be seen. There is this notion of this universal semantic layer where you don't have to move the data from all these legacy environments, but still we're seeing massive migrations from the mainframe due to cost. To Ned's point earlier, let's see if that goes full circle, if there's some innovation there to allow new capabilities to access the mainframe.

Ned Carroll (22:36):

There's also a notion that I'm going to have a full head of hair at the end of this conference.

(22:43):

I think in terms of foundational ai, absent, well organized, curated data is an interesting math equation on a whiteboard. So I think data is king. It's interesting, the seemingly simple concept of how do you organize your data, which then enables you to better align accountability around how you manage and govern that data. Oftentimes it's nontechnical. I think the second key thing is how do you focus on velocity and friction? Where are those points of friction? How do you remove them? But then also how do you quickly detect that people are not operating within your guardrails? I'm a big believer in innovation is best when it's closest to the customer. So when you start creating these large centralized data science teams, you lose that connectivity to the customer. Conversely, when you distribute it all out, you run the risk of ad hoc workload that next thing you know spreadsheets are being used for capital calculations type thing. So how do you balance that?

Penny Crosman (24:09):

Yeah, that's a good question. Alright, I think we have four minutes left. Do we have any audience? Yeah, we've got a question here. Oh yeah, Bailey. Thanks Bailey.

Audience Member Bailey (24:20):

Thank you so much. Yeah, thanks Penny for actually bringing up the Air Canada case. It's a very interesting one and as John correctly mentioned, we can't really test a probabilistic model the way we used to test deterministic models. And so I have a question for Ned. I have two questions. One is really short and the other one is longer. So first one is PNC. Seriously considering investing into a conversational chatbot to handle post login conversations, meaning conversations that are associated with specific client context questions like, when is my next credit card payment?

Ned Carroll (25:05):

It's an easy answer. I'm not here to talk about PNC investments.

Audience Member Bailey (25:09):

Okay. But then the second one, in case you would do that, how would you approach testing a chatbot like that? What would be the technique maybe you have in mind?

Ned Carroll (25:22):

I think whether it's a chatbot or really any gen ai, there's a point in time testing, but then there's ongoing monitoring and how do you really think of it as a closed loop learning system? So how do you design it and architect it in such a way that you're almost always testing it? Because testing every single response, every prompt that goes in and every response that comes out. So I think of that as a closed loop learning system like I would for any gen AI solution. The important thing, and this is important that we've had to distinguish, financial services are great at model risk management. Gen AI is not the same as model risk management. I can't go validate a large language model. If I could, I'd be doing something wicked cool and making a lot of money, right? So how do you think of AI as a system? The model is a part of the system. There are other parts of that system that you need to think about.

Penny Crosman (26:25):

We have one more question.

Audience Member 2 (26:27):

Thank you both. John, I have a question for you. Among the larger financial institutions that you work with, given some of the major advancements in using AI for software engineering, are you finding that it's become incrementally easier for these institutions to build out software tools that historically they may have outsourced to software vendors? And if that's the case, which are some of the areas that you see are more viable for insourcing in the short term?

Dr. John Ratzan (27:00):

I wouldn't say it's making it the words you used a lot easier, but there's a desire for all of them to do that. I would make an argument that many large banks still struggle just with the agile methodology. And what I've seen very quickly, literally in the last three years is agile is there then you're trying to put gen AI from an SDLC to the TDLC tech development life cycle and now throwing Agen right on top of it. So I think Ned said earlier, back to foundations. So to answer the question, a lot of banks are experimenting with it. I think it's going to be a little bit of time, even just with copilot to get code productivity in terms of can you get, if you have a million hours of development, if you could even get 10% productivity, that's a lot, right? So if we can get to 30% productivity on 10 million hours of development, that's going to be the holy grail.

Penny Crosman (28:04):

Alright. Unfortunately we're out of time, but thank you for getting in the weeds with us, John Ratzan and Ned Carroll, and thank you all for coming. Appreciate it.