Want unlimited access to top ideas and insights?
While many banks have launched AI pilots, the real challenge is scaling AI across the enterprise. That requires modern infrastructure, trusted data, governance frameworks, and new operating models. This discussion will explore how banks are moving beyond experimentation to embed AI into core operations, developer workflows, risk management, and customer platforms—unlocking productivity gains and entirely new capabilities.
This panel will feature technology leaders and strategic partners who are building the AI-enabled bank at scale.
- The transition from AI experiments to enterprise AI platforms
- Data architecture for AI: lakes, models, governance, and lineage
- AI in developer productivity and software engineering
- Scaling AI responsibly: compliance, explainability, and model risk
- Building AI-ready infrastructure across cloud and core systems
- How banks measure enterprise-level AI ROI
- AI as an operating model shift, not just a technology deployment
- Partner ecosystems and where firms like Mphasis accelerate scale
LEADERS is a flagship channel that spotlights C-level executives and top experts as they discuss transformative topics for an audience of key decision-makers. We deliver thought leadership on the most pressing issues driving the future of financial services. The LEADERS series is made possible by the support from top industry collaborators including Mphasis.
Transcription:
Transcripts are generated using a combination of speech recognition software and human transcribers, and may contain errors. Please check the corresponding audio for the authoritative record.
Holly Sraeel (00:21):
Welcome to another American Banker Leaders. I'm Holly Sraeel, Senior Vice President of Strategy and Content, American Banker Life Media. While many banks have launched AI pilots, the real challenge is scaling AI across the enterprise. That requires modern infrastructure, trusted data, governance frameworks, and new operating models. We are going to explore how banks are moving beyond experimentation to embed AI into core operations, developer workflows, risk management, and customer platforms, unlocking productivity gains and entirely new capabilities. Here with me to discuss scaling AI and banking, moving from pilot to enterprise platform are Saima Shafiq, Senior Vice President and Head of AI Enablement at US Bank, Michael Storiale, Senior Vice President of AI Technology and Transformation at Synchrony, and Nitin Rakesh, CEO of Emphasis. Welcome all. This is such an important topic, so let's get started. Saima, AI has reached the point where banks must move from isolated pilots to enterprise strategy and capability.
(01:26):
Here's my first question for the panel. Almost every large bank has AI pilots underway. What separates the organizations that are successfully scaling AI from those that remain stuck in experimentation?
Saima Shafiq (01:40):
Before I answer the question, I must say this is the multimillion dollar question of the time. And I think the answer is a little loaded, but in my opinion, an experience of running AI departments for a while starts with the right operating model. And to some people's surprise, it rarely sits on technology. It is more about making sure that we are having the right set of accountability across the organization who are not just running the science experiments, but they're actually becoming a part of the initial design that we're building so they can define the success criteria. They can provide information on how to measure success when something goes in production, and really involving the end users from the outset rather than bringing them at the end of the science experiment where they might not even find things to be working their way. So operating model and engaging the right audience and accountability, clear lines of ownership, who will measure success is a really critical thing to ensure we can move beyond pilots.
(02:48):
But that's not all. You definitely need more things around being able to put the KPIs and OKRs upfront, aligning with the right business leadership to define what really matters, not simply because, oh, you have seen a new model that can do magic, but what exactly does the business care about? So aligning it with their strategic objectives, what are their pain points, where are they trying to really make a difference is really a good starting point. And then of course, technology is very important. Getting the latest and greatest is awesome, but building it with the right architectural principles such that you can scale and plug and play as needed because no model is going to last forever and you have to do the right design such that you can emphasize on reuse and have exponential value by scaling. Regardless of which model you're going to use three months down the road, your architecture and your design of your AI application should be such that going from pilot to production, you only have to do plug and play on the components and modules.
(03:46):
And that requires rigor of enabling everybody to feel proud about socializing the components that we can reuse.
Holly Sraeel (03:56):
Okay, we're done here. Michael, why don't you take a stab and answer the question?
Mike Storiale (04:00):
Yeah. I think one of the biggest things, and just to build upon what you were saying is you've got to start to move from use cases to business transformation. I think in the early days, use cases are a good thing. It gets you proof through impact. You get people who raise their hands who are excited about something that they want to change in the business, and it gets you those early adopters. But what it doesn't get you is the wing-to-wing business transformation that allows you to actually realize those OKRs, start to see the success that you need. If you make one piece of a process 25% faster, you didn't make the whole process 25% faster. And so when you start to think about the businesses that are winning here, it's the ones that are looking at entire processes from the moment an idea begins to the moment maybe the product or the process is all the way over and looking at every piece there and saying, how do I transform that process?
(04:48):
And sometimes they're not the most attractive things to work on. Sometimes these are things like risk processes that actually have a huge material impact on the business. It's not always going to be the shiny object.
Holly Sraeel (05:00):
People have a tendency to get lost with shiny objects. So Ninten, how do you keep things on course? Yeah,
Nitin Rakesh (05:07):
I think there's a lot that was said in the last few minutes, and I'll build on a couple of things. Firstly, AI is not an IT project. It has to really start with what are the most complex and sometimes the most impactful decisions that a business needs to make, and how do you embed AI into that process? I think unfortunately, and I think Mike said it, you can make a process 25% faster or a part of the process 25% faster. The first couple of years, we've really approached AI as an efficiency play. We approached it. Can we speed up the process? Can we eliminate manual processes? Can we find efficiency? Can we reduce headcount? There's so much that has been said about reduction in headcount because AI is here now. What I fear sometimes is that we don't want this to become an RPA 2.0.
(05:57):
This isn't an automation play at all. This is a transformation play. And the reason I say that is that if you look at the way current processes are run and you manage to map them and you feed them into these new powerful AI tools, you're really automating processes that were formed when none of these technology existed.
(06:15):
So a bottoms up automation of a process only goes that far. The first question you have to ask is to what end are we going to use AI? It cannot be a hammer looking for a nail, figuring out what use case goes into the next AI factory. This really has to be, can we move away from process automation, even process re-engineering, to driving an economic measurable business outcome? So if you work backwards from there, there are a few guiding principles, first principles as we call them. Context matters. I think there was a lot of conversation previously around context, but context to feed automation is only part of the story. The real impact is when you can actually start mapping decisions that can then be actually improved because it isn't just about faster processing. It's also about better processing that use faster results. I'll give one example, underwriting.
(07:04):
Heavy modeling goes into any form of underwriting, commercial, insurance, credit cards. Can you capture not just the context and the knowledge that is intrinsic to the enterprise of how the process works today, but can you also capture how decisions are made today? And not everything is a gen AI play because decisioning may or may not actually require you to have probabilistic expensive technology in there.
(07:28):
And then you feed that into the execution layer. And I think unfortunately, a lot of the focus over the last couple of years has really been mostly on governance and execution and building and launching agents. Not enough focus has been on finding a way to create an enterprise level context, linking it to an outcome, and then using that framework to embed decisions in it. Unless we embed decisions into the AI roadmap, we are really not going to be able to re-imagine the business.
Holly Sraeel (07:54):
So how important do you think it is, for example, that more CEOs and business unit heads are getting deeply involved in how to scale AI? Do you think it's significant that Jamie Diamond just came out and said what he said about how AI is not going to replace jobs and blah, blah? So tell me about how executive sponsorship is affecting how quickly AI will scale.
Nitin Rakesh (08:18):
I think the executive sponsorship's going to come a full 360. It was sponsored by the boards, the CEOs two or three years ago because everybody wanted to know what we can do with AI. I think we've come to realize that AI can only fulfill a business strategy versus bottoms-up approach of run by the CIO. Until very recently, every CIO I met was under pressure to have an AI strategy. And my question was always, why do you need an AI strategy? You should have a business strategy that you can actually leapfrog using AI. So I think Jamie's absolutely right as always. There's a reason why JP is at the number one list of banks that are AI forward. And of course, in full disclosure, we have a pretty good understanding of what's going on there. I think they've come to a point where they're not really running use cases, they're not running pilots, they're not even running AI projects.
(09:11):
They're actually embedding it into strategic bank initiatives. And that's the only way to go long-term.
Holly Sraeel (09:18):
But he has, unlike the banking industry has 4,700 plus banks. He's got the largest institution. He's got the vision, he's got the strategy, he's got the money to do it right. There are a ton of institutions though that are not in that position. So to that end, what organizational changes are necessary before AI can scale across an enterprise?
Nitin Rakesh (09:43):
I think the beauty of any new piece of technology is that democratizes many things. It's the same set of tools that are available to JP and available to bank number 4,500.
Holly Sraeel (09:54):
Yeah. But available differently. Available
Nitin Rakesh (09:57):
Differently, but the same set of tools are available. I think the difference boils down to, and I think Salma raised it a little bit, this concept of reuse. The way to think about is not how do I build five different programs and they all run on different architectures and different stacks. The way to think about it is, can I have a foundation that gives me the ability to reuse so you get operating leverage? And every time you bring on a new business unit or a business function, you're not starting from scratch again. The holy grail of this really is all about reuse.
Saima Shafiq (10:31):
Yep, exactly. And the point that you started in the beginning, really the operating model comes back in the discussion then. It doesn't matter you're the back number 4,500 or number one. The main point that you're talking about is starting from what business impact you're trying to deliver. And each company, no matter their size, they're selling their products and services, and that's where they have to start. And as long as you are staying focused on your business and then leveraging AI as a tool, you're fine. We have to stop chasing because we have AI. We have to do these use cases.
Nitin Rakesh (11:03):
It's a hammer looking for a nail concept.
Saima Shafiq (11:04):
Yep.
Mike Storiale (11:05):
Yes. Yeah. And I think you asked before about where the executives fit in there. I think a big piece of it is not only the strategy that we were talking about, it's the trust from the executives. I think if you want to democratize AI across the business, the business from the top down has to trust that the strategy is there, that they can believe that if they come up with something new with AI, that they have the backing of the business behind them. And so that's both on the employee side, but also on the customer side of what you're deploying. And so I think that really goes hand in hand with having a really good executive strategy. And
Saima Shafiq (11:37):
Building on that, to enable that trust as AI execution and strategy leaders and enablement leaders, we really have to be really forward thinking in preparing the observability landscape and making sure transparency is there. And we are not going to be able to scale without having the right controls and guardrails upfront. So observability built into your design principles is what can enable that executive trust.
Nitin Rakesh (12:01):
Observability, governance, security, traceability, ethical AI, all of that goes into the fabric that I talked
Holly Sraeel (12:07):
About. Yeah. Okay. Given where you are today, if you were beginning AI's journey right now, what would you do differently now that you've learned some things along the way?
Saima Shafiq (12:19):
If you want me to start, I will make the effort on AI lifecycle, AI delivery lifecycle a priority because traditionally companies have been having a lifecycle protocol for technology projects or business initiatives and embedding the AI delivery lifecycle such that you have a governance mechanism that is really engaging the business leaders and the risk and compliance leaders from the outset to define your controls as a design principle, and then establishing that ongoing monitoring eventually as well rather than just trying to chase that and negotiate that through the finish line. So the first and foremost thing that is really taking the most time for most companies today is being able to get that risk and security and compliance approval at the end and spending weeks and months. And if we were to rethink the governance and the risk procedures around it, I think that would be the first thing.
(13:13):
There are many other things I can talk, but that's one of the things that comes to mind. Okay.
Holly Sraeel (13:17):
Let me get Mike's input too.
Mike Storiale (13:18):
Yeah, I think if I were starting over today, I would've given a little more clarity to the business as where I wanted people to focus. So I think early on, we talked a second ago about use cases. Everyone had the ideas of where they wanted to make change. And the exciting part about that is that we can't possibly know every part of the business where change is possible. The flip side of that is that we probably missed out on some opportunities where we could have been more targeted at the start because we tried an area that maybe we learned really early on wasn't great, but we let some other use cases continue to go to see if those would pan out. If you're starting over again, you look back and say, well, if I had been really targeted in A, B, and C and told people, Hey, when you're thinking about your area, I want you to target in one of these areas, one of these benefits you can get back from AI.
(14:03):
I think you'd be able to accelerate in a different way. And so it's a little way to look back on it. Can
Nitin Rakesh (14:08):
I comment on that? I think the - Please. They've got a good question. While we ask ourselves what couldn't we do differently based on what we know over the last two and a hal three years, we also have to remember that the pace of change in this tech stack is really rapid. You really didn't know what was coming down the road.
(14:28):
So a year ago, everybody was focused on embedding Copilot. You want to go from pilot to copilot. Then they said, okay, Copilot isn't good enough. We got to go from Copilot to autopilot. And then of course, because we were so focused on how many agents have I built, how many use cases have I handled? We really didn't focus a lot on this foundational element. So I think the one thing that, and again, it's funny, this morning I was on a call with a really senior group of people from a large insurance company. And to their benefit, they haven't really done much in the last couple of years. So now they're in this concept of, I'm going to leapfrog and I'm going to learn from everybody else. Did
Holly Sraeel (15:04):
They pass it off as strategic?
Nitin Rakesh (15:06):
Well, they were in an M&A environment, so there was a lot going on and they had to stand themselves up, and now they're actually going into a merger environment again. So I think they really have a once in a lifetime opportunity to set the foundation before they go into the merger. So this is all about foundation, foundation, foundation. And foundation isn't just about security, guardrails, governance. Foundation is also about context and decisions. I mean, think about this. They're going to merge with a very large peer.
(15:33):
And if you have the foundation set, the whole integration process actually becomes really, really interesting because you're folding versus starting and integrating. You're folding it into an integrated run model, you're folding into integrated build model, you're folding into integrated cyber operations, you're folding into integrated business solutions. So I think the art of the possible is immense. I don't think we should beat ourselves up and saying we could have done things differently because we didn't know what we did know. But now we know better. We've done a lot of experimentation, but we got to get out of this pilot, copilot mindset. One
Saima Shafiq (16:07):
Of very critical things that you mentioned, I just want to add one that I would say is worth emphasizing, is identifying accountable leadership across the business units. I think that's one of the very instrumental thing. We can have all of these discussions on foundation. They're absolutely necessary, but who's the decision maker? So having the right authority and the right decision and accountable executives being present within their business lines to represent their goals is also very critical in my mind. Every
Nitin Rakesh (16:30):
Time you want to emphasize something, I like it.
Holly Sraeel (16:32):
All right, Michael, successful AI depends on modern infrastructure, trusted data, and engineering discipline, as well as better models. Simon, reference this. People often say your AI strategy is only as good as your data strategy, but what does that mean in practice?
Mike Storiale (16:50):
I think first of all, in practice, it means that garbage in, garbage out continues to apply. I think I heard this interesting thing from people for the first few years where they said, well, I heard GenAI is really good at unstructured data. My response would be, you don't have unstructured data, you have messy data. And those are two different things. And so I think first of all, this isn't the thing that solves every problem for everything wrong with your data. You still need good data, clean data, data that you can trust. But then I think at the other side, it's actually about making sure that people throughout the business realize how much all of them are data stewards here. As they are creating new things, as they're working with their data engineering partners, understanding that the data that you are producing can make really remarkable changes with AI, but at the same time have impacts if you're putting the wrong data in there or you're not treating it with the right respect or the right rigor that it deserves.
(17:45):
People need to be able to understand what it means to be that kind of a data steward. And so that's a huge area we've been focused on is making sure people understand who their data partners are, but also what their responsibility is with the data that's going into it. One
Saima Shafiq (17:57):
Of the things AI has done, I mean the subjectivity of some functions in some areas where groups of people are serving things, there has always been a disconnect or a subjectivity in how a different person would decision a certain thing. And it was not tracked as well. The problem now is that with AI trying to augment those decisions, we are getting more transparency and visibility around that difference of the results. The challenge that we have to deal with now is if we start applying agentic AI on top of that, we are only going to amplify the problem. So it's really the time of discipline and standardizing a lot of things within the companies.
Nitin Rakesh (18:39):
This topic is a bit of a catch-22 topic because while you've heard on one end that we need clean data or AI, the programs won't work, if you spend the next 12 months, 18 months, assuming you can clean the data and bring it together, you really don't have that luxury of time. So we actually coined this interesting concept. If you know what outcome you need to drive, press tech, underwriting. I need to be able to take better underwriting decisions faster. That's an outcome. What does better mean? Well, you don't need to know with time, but loss ratio, things like that. Again, metrics matter, who cares matter? Then you work backwards and say, okay, to drive this particular business transformation, I really don't need to go find every piece of data and clean it. We coined this concept of minimum viable data. What do you need to drive that outcome?
(19:30):
Minimum
Holly Sraeel (19:30):
Viable data.
Nitin Rakesh (19:31):
Minimum viable data. Very
Holly Sraeel (19:32):
Good.
Nitin Rakesh (19:33):
What do you need to drive that outcome? And of course to extract data, structure or unstructured, you can actually. There's so much tooling available that you don't necessarily have to spend even weeks standing up the context and the ontology around it. You should be able to do it in a matter of days because that's where Agentic AI comes into play. Because for us, Agentic AI is a tool stack that should actually help you drive agency versus Agentic AI taking over the world. So those are two different things. So if you start with that approach, within a matter of days, you should be able to find all the right data elements that you require to drive that particular outcome. And
Saima Shafiq (20:12):
Once again, for that agency point, that's where the human power and accountability is so critical. You want to give this agency to the decision makers and the owners of these systems and the processes that you're looking to reimagine. That is the only way agent AI would work.
Nitin Rakesh (20:28):
We want to give agency to the agent themselves, with the human in the loop. I think that's the holy grail, right? Yes. What is agent? Agency is the ability for AI agents to make decisions - To make decisions. Autonomously with of course the human in the loop.
Saima Shafiq (20:38):
But informed and empowered by the experience coming
Nitin Rakesh (20:41):
From
Saima Shafiq (20:41):
The human and the accountables. That's
Nitin Rakesh (20:43):
Always the human in the loop, right? Yep. 100%.
Holly Sraeel (20:45):
Yep. So let me ask this question. How do you balance building proprietary capabilities with leveraging external foundation models and technology partners?
Saima Shafiq (20:56):
How do you balance.
(20:58):
Well, my answer's going to be very simple in this one. You always buy the commodity. You don't want to spend your scarce talent in building things that others have done better and they're going to get cheaper and economical every day passing. And then you build something that's proprietary, that's complex, that is really sitting on your own institutional knowledge that really goes around, is the system around your data and really catering to your clients, to your customs, the workflows and the processes and the products and services that you're building for your customers. That's where you have the power to do things that others cannot copy. So buy the commodity, build the complex things, make sure that you're giving enough room for domain-specific modifications as required, and then buying things such that the enterprise can really become more self-sustained, self-service in terms of enabling low-code and no-code kind of common use cases.
Nitin Rakesh (21:55):
I agree with that. A very simple way to think about it is you can buy agentic AI tool stacks, but you cannot buy agency. If you want to get to agency, you have to build your own foundation. And that may be a combination of third-party tools and the work that you will need to do to capture context and decisions. With
Saima Shafiq (22:14):
The mind that you control the logic and the outcome, being able to adjust partners as needed because they will evolve in the rapid accelerating.
Nitin Rakesh (22:21):
And also remember, every software vendor has an agentic AI tool stack. There is temptation to say, well, if I actually really, if I'm running infrastructure or service desk or ITSM, very easy for me to say, let me just build them on Snow. Or let me just build my entire customer engagement on Agent force. But then the question is, how do they work? How do they fit into the fabric? How do they fit into the -
Saima Shafiq (22:45):
Ecosystem for your workforce.
Nitin Rakesh (22:46):
Ecosystem for your workforce. I think, again, first principles wise, what you need to do is you need to build the stack that gets you to agency, and that build will have really strong elements of your own proprietary way of working.
Mike Storiale (22:59):
I think a lot of what you're saying though is about we need to treat our employees like they are customers of our data. I think too often when we're building things internally, we forget that the tens of thousands of people using it are our customers, our internal employees. And it's funny because part of what you were saying a second ago, there are these moments that you look and say, well, you hear a company saying, "We're all in on this tool set or this product." And you're like, "How many people at your company didn't ever use that product before? Can we meet the customer where they are, the employee, the customer, where they are, and make sure that in whatever flow they use today, this feels so organic to them that of course it's better than what they're doing." Because as they start to adopt it and feel excited about it, you get a snowball effect because they're excited.
(23:44):
So they're investing more in it if you continue to meet them where they're doing their work every day.
Holly Sraeel (23:52):
All right, let's shift gears a little bit. Let's talk about scaling AI requiring scaling confidence and embedded governance, risk and trust. How has governance model evolved as AI moved from experimentation to production? Nitin?
Nitin Rakesh (24:10):
I think again, to me, there are certain must-haves that you have to embed into the way you think about scaling these. Unfortunately, fortunately, banking is a trust business. You have to be able to embed trust into the fabric of everything you do. You have to be able to document why you made a certain credit decision. There are laws and regulations that govern that. You have to be able to demonstrate the traceability of every decision. So it has to be embedded, and that's kind of the workflow and the proprietary way of working that brings that about. And then there are things that are many CIOs have three plus nights about. And the oncoming wave with Mithos is one of those.
Mike Storiale (24:56):
You
Nitin Rakesh (24:56):
Have to be able to embed that into the fabric and the way you're going to launch it. And the only way to do it is to work with the principle of symmetry. You have to use the same tools, but those tools have to be available to you as you start building these AI applications. You cannot wait for building an application and then focusing on remediation. So there are multiple different facets to trust and governance. And then of course, as I mentioned, you have to have certain common principles with which you build every agent. So you don't have building up 5,000 agents and having them work through an NCP server is not the answer to that. So I think there are five or six different dimensions and nuances to this area of governance. And the only way to deal with those is to embed them in the foundation.
Holly Sraeel (25:38):
Do you agree?
Saima Shafiq (25:40):
I do agree totally. I think one of the other things that I will share that we have seen evolve is that historically, as I mentioned previously also, the governance was something that was considered a last step thing. Before we go to production, we have to make sure that our risk and compliance partners would review and approve everything. Now with the way AI is embedded in everything that we are doing, it has to start from the beginning is important. But what we have done and have seen it work really, really well to an extent that we were able to scale exponentially in terms of the true business value was having this governance and risk-based processes aligned based on the risk tiering. So you want to have built-in inherent go-to guardrails available based on the risk tier of an AI application or an AI use case. So the application teams that are still trying to figure out and navigate how to go to prod fast, how to go from pilot to production, they are given this guideline that based on the risk tiering, these are the sets of basic guardrails that you must enable.
(26:42):
And as much as we can, and we have enabled it, we have seen success in deploying more to production faster.
Holly Sraeel (26:50):
Actually
Nitin Rakesh (26:50):
Even in broad, you have to be able to do tiering and - Absolutely. And monitoring
Saima Shafiq (26:54):
Based on the tier. Absolutely.
Nitin Rakesh (26:55):
So think about not only - Design and runtime
Saima Shafiq (26:57):
Both.
Nitin Rakesh (26:58):
But blocking the impact of the thread, but actually even remediation could be automated based on tiering. Yes.
Saima Shafiq (27:04):
Fully controlled.
Mike Storiale (27:05):
I think to your point, responsible AI has to be at the very start. So you've got to ask yourself a question. In our case, we have a responsible AI working group that asked the question early on, is this a responsible use of AI? As the way that we define it. And that is going to change over time. Responsibility doesn't just mean, is it a good way to use AI? It could mean, are we mature enough? Is the technology stack mature enough to use it in the way that we want to? Do we feel we have the observability or the guardrails or the subject matter expertise in-house? And so I think so much of that is about asking those questions upfront to your point, also creating that matrix so it's really clear and transparent for people as to why it is or is not responsible and making sure that they're getting that input and that feedback throughout that entire process.
Saima Shafiq (27:48):
Absolutely agreed. But I think the real major thing that I failed to mention earlier, I will do that now, is the risk teams were super focused on the drift, the potential drift of data and the model output. And now with generative AI, it's a much harder thing because you are so dependent on the prompt. So having, to Nathan's point, having the runtime controls that are not only observing and monitoring and tracking, but they're also able to control and intercept. That is the real switch that has enabled the successful launch of gen AI. And that's
Nitin Rakesh (28:22):
Part of the reason I was saying, this isn't all about gen AI. I think we use the term AI and gen AI interchangeably, and that's a bit of a injustice to all the other tools available to us, even with machine learning models. And gentech
Saima Shafiq (28:35):
AI.
Nitin Rakesh (28:37):
The ability to make deterministic decisions will catch the drift. I mean, let's take the example of something that has a rules-based decision matrix. You don't need gen AI for that. So I think it's a combination of what kind of tool stack goes into what layer of governance and risk.
Holly Sraeel (28:55):
Okay. What's up the ante here? So the greatest impact of AI will be in changing how banks operate and compete and how they measure enterprise value, notably productivity and ROI. What KPIs best demonstrate enterprise AI success beyond proof of concept metrics? Michael.
Mike Storiale (29:14):
I mean, I think when we look at what we're seeing, you really have the ability either measuring capacity, productivity, or revenue. And so much of what we've been looking at is how do we get either the productivity so I can do more with the same or I can do the same with less? Or how do I start to increase revenue? Can I get a better NPS score out of whatever it is that I'm building? Is it possible for me to lower my cost per account? And then we talk to our employees about the capacity is the stuff that is the onesy twosy things for each individual employee that can be really tricky to measure. I bought you 10 minutes back in your day. There's some replacement work that happens there, but that's not going to be the big nut to crack there. And so we're measuring that in those areas.
(29:53):
So when you look at something like productivity, if I'm able to perhaps keep my headcount the same or I'm able to perhaps Keep my expenses the same, but I can continue to grow the business. We can trace that back in a really meaningful way. And then we're also being really candid about the things that we don't think we can measure. I don't like to put numbers out there that I don't feel confident in. And so when we get those questions, we're very happy to point our teams to industry data. We're saying, we don't feel confident in being able to measure that metric in that way. And so we're not going to try to track to something that we don't think is realistic.
Holly Sraeel (30:25):
Do you think other institutions will follow suit and push back on not advocating for metrics that just at this point in time are not achievable?
Mike Storiale (30:35):
I would hope so. I think part of the thing that we all kind of get lost in, and this is, I've spent my career in banking, we're all data and finance people at heart, I think. And we do get stuck sometimes on trying to measure everything and forgetting that what you're measuring might change the outcome because that's going to motivate people to track towards something specific. I mean, it's where you've seen the token maxing from some of the tech companies and all of these different kind of third order effects that start to occur. If you actually start to measure the things that matter and then focus on those, you're going to encourage your employees to focus there as well. And so I would hope that that shift starts to happen in a way that people can start to motivate the right behaviors.
Saima Shafiq (31:15):
Couldn't agree more with that. I will say that I'm responsible for the enterprise enablement of AI, which means I am accountable for making sure that we are, when we are investing in AI, we are actually truly getting the value out of every single AI investment and we are able to make a good decision on stopping investment if there are areas that are not worth pursuing because we can't measure, we don't have clear outcomes that we can really do that. I think two things that are important, and they're prevalent in the organizations that are in the early stages of maturity. Companies have been focused on the output measuring oh, number of models in production or a number of prompts and the number of tokens, which is not the output that we expect. What we really need to focus on is the outcomes, the business value. So as my role of enablement, looking at the value that we are able to produce, we really have to look at the KPIs like, okay, starting from, for me, from an enterprise standpoint, how many AI applications we were able to truly deploy in production that are adopted and in use by the end users and how are the end users feeling about them?
(32:22):
Are they happy? Are they saving time? Are they producing the hardcore numbers that we said once defined as a success criteria? We will save this much time, we'll generate this much revenue, the cost or the revenue or the risk reduction per transaction or whatever you're looking to calculate. Those are the precise points that we have to measure against everything in production and get the feedback loop with an ability to be able to identify how quickly the organization can then take an action from the early pilot feedback and incorporate that back and then measure the AI lifecycle duration as well. How long did it take for you to come up with an idea and actually put it in production? And that is your true success as well because the faster you can do it, that is your magic trick on safer, faster path to production.
Mike Storiale (33:06):
One of the metrics that's really getting a lot of traction lately is cost per outcome. You can measure that by model. What does it cost me to get the outcome on this model versus that model? Because asking for something really basic of a really expensive model is a very high cost for that outcome. But you can also look at it across what you're trying to transform in the business. What is my cost to achieve the outcome?
Saima Shafiq (33:29):
Yeah. Tokenomics.
Nitin Rakesh (33:30):
I think if anybody says that they've figured out how to measure the ROI, they're not being truthful. This is the honest truth. It is. I'll give you two anecdotal examples of, and I mean I have an opportunity to spend a lot of time with folks that make a lot of these decisions. And one CIO of a very large wealth management firm told me, "I think I know how much productivity you're getting, but I don't know whether people are going home three hours early on a Friday because it's not showing up in business metrics." Another one told me very recently, very, very large bank, that two things that they've measured that are working well, the backlogs from software development have disappeared, which means
(34:12):
Same number of people are writing more code. The risk there is semantics, syntax and code quality. You can find ways to control it because of course nobody's really coding right now. They're orchestrating the whole coding development process. And then also the fact that because people have all these tools, they're also starting to fill white spaces with these expensive capabilities that they really don't need to. So there's a little bit of that drift in work happening. Oh, because I have these tools, maybe I should just automate this. I really don't like doing this. I'm going to just take these expensive tokens and fill up the white space because I have these tools. And I think the three or four areas that have been early successes like SDLC transformation and higher velocity throughput tier, but has that driven. Yes, backlogs are done, but has that driven true business value?
(35:09):
I think it's a question mark right now.
Mike Storiale (35:10):
Well, and I think the question there too is, is the stuff at the bottom of the backlog the stuff I should have been working on anyway? So if I was never getting to the stuff on the bottom of the backlog, yeah, maybe the backlog is clear now, but maybe you're not seeing the business value because actually you started working on stuff that wasn't providing business value. That's why it was at the bottom of the backlog. We
Saima Shafiq (35:26):
Are re-imagining and really bringing in the key decision makers who care about the business outcome need to be brought in to make those decisions on reallocating the resources that you're freeing up rather than just mindlessly letting them go play with the most expensive tools.
Holly Sraeel (35:39):
Or taking them away.
Saima Shafiq (35:41):
Yeah.
Holly Sraeel (35:41):
Or taking the resources too.
Nitin Rakesh (35:42):
Unfortunately, this can only happen if you go top-down, not bottoms up. If you start with high impact business decisions that you need to drive using these kind of tools, that's the only way you can truly measure a business outcome. Now, whether it's business outcome per token, whatever is the metrics. Another interesting case, there is a large investment org that is trying to use this capability towards driving 50 basis points of additional net investment income. That's a real measurable business outcome. I mean, if you're running a trillion dollars in assets, you can imagine what kind of impact that can have. Now of course, it won't be 50 basis points across all asset classes. But even if you just take some of the more complex ones, private credit, that's a big deal. So again, I think it's top down versus bottoms up. I have all of these tools, democratize the access.
(36:33):
Everyone's got access to these everyday AI tools. It's very hard to measure that.
Holly Sraeel (36:38):
And all of this cannot happen internally. So let's talk about technology partnerships and how they're becoming increasingly important. Where do external partners accelerate enterprise AI and where do they not? And where do banks need to
Nitin Rakesh (36:53):
Maintain - Now invested interest in this. I literally answer
Holly Sraeel (36:56):
For now. Well, I'm asking this subjectively as a journalist. Setting aside the top 10 or 25 US banks, it's just not possible to have all of the internal talent and resources and whatnot necessary to build. So technology partnerships are becoming more important. So two questions. Where do external partners accelerate enterprise AI and where should banks retain strategic control when working with partners?
Mike Storiale (37:26):
I mean, I think you have to look at what your competitive advantage is. We were talking about this earlier. I don't know that our competitive advantage is creating our own models. But we do have a competitive advantage in the products that we build and the underwriting that we perform. And so focusing more on that and instead coming to technology partners. Now, when I look for a tech partner, I'm not just looking for a vendor. I'm looking for a partner who's going to bring expertise and challenge my thinking.Because I think that's where you're looking to add something to your team or fill those gaps with that partner is to say, "Hey, maybe what I don't have is expertise in this domain. You don't only build a technology or provide a service, but you're also going to challenge our business to maybe think differently or transform it because you have a unique thing that you bring to the table." And so for me, it's been a lot of, I know what my team's core competencies are or what my organization's core competencies are.
(38:17):
There are areas where people spend all day on a totally different problem. I want them to come in and make us smarter.
Saima Shafiq (38:23):
Yeah. And what I would say is, so in our company, we run an AI center of excellence, and then we do have accountability across the technology as well as the business lines to make sure that we are enabling federated democratized implementation of AI. The AI Center of Excellence's goal is not to show them, oh, look what this new technology partner can do for you. That's not the goal. Our goal is that we work very closely with our business partners to see what exactly is their focus. Where is it that they're looking to head and what are they trying to achieve in the next five to 10 years? And based on that, we find with their vision in mind, the right technology partners. And that's where we can do strategic investments and we can have the tech partnerships that are, to your point, Mike, are able to not just come give us advice or tools, but they are able to challenge us because we are building a roadmap for multiple years.
(39:15):
So technology partners do play a critical role. We do not want to do the things that they're good at because we can use it and it's getting cheaper and cheaper every day. From a model standpoint, at least from the frontier model standpoint, it will. And it has to because we want to scale that. Our competitive advantage is in us paying attention to the right goals, the right objectives that will really count for the bottom line. And that's where we find the best technology partners, but with the design principles upfront on architecture that can plug and play and can actually pivot when it is time.
Holly Sraeel (39:49):
Okay. If every bank has. You can answer this, you start. If every bank has access to similar AI models, where will sustainable competitive advantage actually come from?
Nitin Rakesh (40:01):
I think that's been the case for banks pretty much always. What you do is independent on just the tools at your disposal, but how you run the business.
(40:12):
Mike mentioned, how do you underwrite? They have a very interesting business because they specialize in a segment of the market that is very, very hard to make money in. That's their competitive advantage. Can I really deal with this class of borrowers? And can I really find enough specialization and sophistication to sustainably make money in this book of business? And hence, I have the same tools available to me. And it goes back to what I said earlier. How do I capture my decision process into these tools and make those decisions even better? It's not about taking a cloud or a Gemini because those are generic tools. It could be about using open weights because that's how they're able to capture the best decisions. Or it could be just about capturing the way they run their business today in a manner that you're not only looking at a digital twin of your business, but you actually are going to create a decision twin of your business at scales.
(41:06):
So I would say it's a new way of working, but the old sustainable competitor advantage can actually get enhanced if you do it right.
Holly Sraeel (41:14):
Okay. Final question. If your board gave you one additional billion dollars to accelerate AI transformation, but you could only invest it in one area, where would you put it and why?
Saima Shafiq (41:30):
I can start if you like.
Nitin Rakesh (41:32):
Billion dollars made her a smile for sure. She was
Mike Storiale (41:35):
Ready.
Saima Shafiq (41:36):
Yeah, I'm about to make many people happy with my answer too, hopefully. We always think of the organizations running successfully because of the people, the process, the technology, and the data, all of those foundational pillars. If I had a billion dollars, I would actually invest those in really upskilling and adding talent to the people because they can be the bridge that can build the real foundation of success between the business leaders and the technology. And yes, data is essential. So if you were to pick just one, I would say people who can then do the right thing with the data and make that happen. But if it was just one, it would be people. Absolutely data is important. And I will say the reason it's so critical to invest more in people is because not only they can make the right choices to rationalize and build the data that you can trust or bring in the right technology and help us govern and scale and write the clean processes.
(42:35):
It's really where the competitive edge sits. I mentioned earlier in the first question, aligning with the principles and then enabling teams to socialize the aspect of reuse, that's how you scale the true return on investment. I have examples to quote for those. I mean, US Bank has some flagship AI success stories that are publicly known. Think about AI augmented decision for statement analyzer, or think about the design assistant really improving the product life cycle across multiple stages of product development and SDLC. We have client-facing AI that is enabling the client-to-client and bank-to-bank integration and our ERP integrations through developer portal that is fully AI augmented where you can complete from start to end of the entire integration you can do with AI assistant. So removing friction and adding value. So investing in your people who can make the right choices I think is really enabling them to add value for the company.
Mike Storiale (43:32):
I have a bit of a twist on that one. I was thinking about this answer a little bit, and I'd want to invest in areas where we can really focus on human transformation. What I mean by that is the technologies that are going to transform, if you're able to look at the things that people do every day, every single one of us has something that we do that is unfulfilling in our job up to the CEO. And the same is true for things that we do with our customers, that there are areas that they experience an interaction with us that they probably didn't wake up and want to call us today. They probably weren't hoping that there was a disputed charge on their credit card. And so if you have that kind of money, being able to say, "Well, what can I invest in to transform the human experience?" There's been so much focus away from humans lately with AI.
(44:22):
And I actually look at AI and say, "That actually is what's going to help all of us be better and focus in the right area." So that's where I would make the investment.
Holly Sraeel (44:33):
Want to weigh in on where they should make the investment? Well,
Saima Shafiq (44:36):
You can
Holly Sraeel (44:36):
Join the
Saima Shafiq (44:36):
Billion
Nitin Rakesh (44:36):
Dollars. I will enjoy the billion they spend, but as long as it's on people that we also have some skin in the game with. But jokes aside, I think going back to first principles, right? Spend the money where the foundation matters. Almost every bank still has a lot of tech debt and legacy. You have to be able to remove that. That's the only thing that I can hear. Spending some of that money in unleashing the true ability will probably go a long, long way.
Holly Sraeel (45:04):
Interesting that I didn't hear from any of you really. The huge impact on the intersection of AI with on-chain finance is going to be enormous, and it'll reset this game again, and what sort of timeframe that's going to happen in. So something for our next discussion.
Nitin Rakesh (45:26):
People don't really want to talk about the blockchain or on the chain because it's been a promise that's been long time coming, but starting to gather a lot of stream today. Oh, for sure. For sure. Question though is, will it get accelerated or conflated when these two tracks collide? I think it's more like acceleration. What's another topic for another day?
Holly Sraeel (45:45):
It's another topic for another day. That's it for today. So I'd like to thank my panelists, Nitin, Saima, and Michael for American Bankers Leaders. I'm Holly Sraeel, and I'll see you at the next Leaders. Thank you guys.