- Aligning AI use cases with business value
- Data readiness as a foundation
- Risk management and responsible AI
- Scaling AI beyond pilots
Transcripts are generated using a combination of speech recognition software and human transcribers, and may contain errors. Please check the corresponding audio for the authoritative record.
Penny Crosman (00:09):
Thank you. We are going to have a session all about the AI topic of the moment. We have a panel with very high credentials. We have Teresa Heitsenrether, who is the Chief Data and Analytics Officer at JPMorgan Chase. She and her team won our innovation of the year award this year for their LLM suite. Thank you. This is a homegrown technology that provides a wrapper or a portal for generative AI models that they're providing throughout the entire company. It's currently available to 250,000 people and heading toward 340,000, I believe. We're going to talk a little bit about that. She and her team have also led the Evident AI index four times in a row for the number one bank in AI maturity. We have Kim Bozzella, who is managing director and global head of technology consulting at Protiviti.
(01:20):
Before this, she spent 15 years at UBS in chief information officer roles. She spent 15 years at Accenture where she was a partner in the financial services practice. We have Prashant Mehrotra, who is chief AI officer at US Bank. He previously was director of data engineering at Capital One and led the AI Center of Excellence at Allstate. And we have Sumeet Chabria. When I first met Sumeet, he was Chief Operating Officer of Global Tech and Ops at Bank of America. He also was at HSBC for 18 years, and he was chief information officer of two different divisions there. He's also a faculty member at Carnegie Mellon University and he now leads ThoughtLinks, which is a strategic advisory firm that works with a lot of the top banks. Let us start with you, Teresa. Someone on your team recently said that the entire bank is being rewired for AI.
(02:32):
What does that mean? What does that look like now and what might that look like in the future?
Teresa Heitsenrether (02:38):
Well, first of all, thank you. It's great to be here with all of you. I think you alluded to the LLM Suite that we rolled out last year. The real point of that was to be able to safely give almost all of our employees access to large language models. The premise is that you have to put this technology in people's hands for them to really experience it, to understand what is possible and how to incorporate it into what they do every day. But I think the big change for us over the last year has been not so much just giving people access to a large language model, but you have to almost connect it—hence the rewiring. If you're a banker, you're going to be looking at doing different things than if you're working in our finance team. This is the idea of not just giving people access to the large language models, but connecting it to the data and the capabilities they need.
(03:33):
Tapping into client information, being able to create PowerPoint decks, being able to work with numbers—all of these things that actually now get integrated into what you do day-to-day to make you more efficient. I think where we see the world going in the next iteration is as the models continue to get better and they can do much more complicated things and reason. It's that continuous, deeper integration into everything we do and just really rethinking how you work and where this technology can be useful to you. It is a pretty comprehensive rethinking and rewiring of the way we work.
Penny Crosman (04:12):
Thank you. Prashant, what is US Bank doing with generative AI today? Where are some of the places where it's useful?
Prashant Mehrotra (04:21):
Thanks for having me. For folks who are not that familiar with US Bank, we are a Minneapolis-based firm. We are a 160-plus year old, second oldest chartered bank with a physical presence in 26 states. Where we see—and this is an interesting point—is I know lately we talk a lot about just GenAI, but we look at all facets of AI from old-style machine learning and deep learning to generative AI and agentic AI. What we are doing is looking at the application of these, as Teresa mentioned, in every part of the business in the entire value chain. No matter where you are, you can't just use one thing in every single scenario. You need to have task-specific, job-specific, or role-specific AI capability.
(05:35):
What we are doing is creating solutions customized to your need. Honestly, there isn't an area of the bank where we don't believe—whether it's machine learning, deep learning, or generative AI—that we cannot adopt it. What we are doing more is prioritizing which areas we want to focus on. We are starting with the strategy that the bank has set. We talk about our three focus priorities in terms of organic revenue growth, expense management, and our payments transformation. We are focusing on those areas and enabling those objectives with all facets of AI capabilities across the bank.
Penny Crosman (06:38):
So not just generative AI, but machine learning, neural networks, and Agentic AI, which we'll talk about later.
Prashant Mehrotra (06:45):
Absolutely. It's really important that we leverage the most emergent of technologies, but we want to use it in the context of what a frontline rep, a branch person, or a fraud analyst is doing. We want to make sure that we are more focused on those roles and every time they are delivering an experience either to our internal customers or to our clients, the technology and capability is present behind the pane of the glass helping them with that mission.
Penny Crosman (07:32):
All right. Thank you. Sumeet, an MIT study came out in July that found that 95% of organizations are not seeing a return on investment on their generative AI projects. What did you think of that result and what do you think are some of the keys to actually getting a return on your GenAI projects?
Sumeet Chabria (07:58):
Frankly, Penny, I was surprised to see that number. The headline number of 95% failure rate is very high, and I don't see that in this industry. I wasn't clear how they calculated return on investment; I read the study twice. I don't think with technology like generative AI or agentic AI, you can calculate ROI that easily because there's a one-time infrastructure investment needed at the enterprise level—a financial commitment that has to be made—and that overhead cannot be passed to the first few use cases. A lot of times, the first few use cases are all about experimentation anyway. I think our industry was more mature than that. That doesn't mean we have a 100% success rate, but I will say you should measure value for every use case with metrics before and after implementation.
(08:53):
The study also mentioned the reasons why projects failed. I agree with that; it was less about the underlying tech and more about integration, business process changing, getting your data right, and setting up those core foundational elements to be successful. I agree with the reasons, but I didn't agree with the headline of that study.
Penny Crosman (09:17):
Did anyone else agree or disagree with that finding?
Teresa Heitsenrether (09:20):
I think you have to break it out a little bit. We absolutely are seeing value from AI at JPMorgan that's very quantifiable, but it's mostly from traditional machine learning right now in areas like fraud, credit card marketing, and various things like that. When you look at GenAI, I agree with what my panelists are saying. We've rolled it out to a lot of people, and I know for a fact they're gaining hours of productivity a week, but that doesn't show up anywhere in an income statement. It's very hard to measure. On the flip side, if you think about other technologies we've experienced—PCs, the internet—it's just a ubiquitous way everyone is going to work. If your employees don't have it, by definition, you're that much less efficient.
(10:09):
We look at it that way. I also agree that you have to pick your focus areas. It is not about the technology; the technology is there. Agentic technology is there. Where the value gap comes from is that it only works if your data is ready. It only works if you have cross-functional teams looking at the whole process end-to-end and thinking about how to completely redo the way you're working. That is a much bigger, deeper transformation. I think this phase of slight disillusionment is coming because of the magnitude of the effort to integrate it, not the technology capabilities.
Kim Bozzella (10:52):
The other thing we're seeing is when firms can really identify the productivity individuals are gaining and what they're doing with that time—not just measuring that a process is running faster, but seeing where that capital is being reinvested and trying to articulate the value there.
Prashant Mehrotra (11:14):
There's an old saying: "Computers are everywhere except in the productivity matrix." Same with AI and GenAI. I've been in technology for a very long time and I cannot remember a single piece of technology that became mainstay and widely used so quickly. We need to build that foundation and infrastructure, but more importantly, we have to rewire our workforce and rethink how we deliver what we intend to deliver. On that study, I think we need to differentiate between pilots and experiments. We are seeing significant value and widespread adoption. From a sample size of one, I can tell you this is the real deal and it's here to stay.
Penny Crosman (12:38):
Just to follow up, what is your most popular use case for generative AI right now?
Prashant Mehrotra (12:46):
The most popular use case for us is any time someone from the bank is interacting with an external client—how we can do that faster. It's providing the right answer and right knowledge to get that task done faster and deliver that overall solution.
Penny Crosman (13:31):
That makes sense. Kim, we've been seeing an arms race with the biggest banks putting billions of dollars into this technology. Where does this leave smaller banks with much smaller tech budgets? Will it be hard for them to compete in this new order?
Kim Bozzella (13:57):
For sure, the bigger banks have more resource to put behind it and more in-house capability. It is a bit of an arms race, but I think the advantage smaller banks have is threefold. First, the need to be very focused on use cases that create ROI. Second is the ability to leverage ecosystem partners. Many large product platforms used by smaller banks have embedded AI; if they take advantage of that, it doesn't have to be self-created.
(14:46):
Third, hopefully the smaller banks' data is in a better state—less complex, cleaner, and with less bureaucracy to get it where it needs to be. It's really about finding the use cases that embed the end-to-end business model where you can create value. I don't think the difference in firm size will make that difference.
Penny Crosman (15:21):
Theresa, back to you. I've read that JPMorgan Chase is in its next phase of its AI blueprint, which is deploying agentic AI to handle complex multi-step tasks. You mentioned that foundation work has to be laid regarding data integration and governance. What are some of those pieces and how are you building them?
Teresa Heitsenrether (15:52):
Just two seconds on what an agent is: as these models have evolved, they are now able to reason. Much like when a person gets a problem and thinks about how to solve it—researching, accessing tools—the models can now reason and take action by calling on tools within applications. That is going to be a big boost from a productivity standpoint. The important difference is that unlike traditional software programs that are predetermined with "if-then" rules, these systems are non-deterministic. If you give two people the same problem, they will solve it differently; so too will the agents.
(16:58):
You have to understand what information they are accessing and what actions they are taking. If they're working in combinations, you have to look at those outcomes, give feedback, and ensure they continue to improve. You now have machines speaking to machines. While people can handle ambiguity, machines don't do so well with that. Those foundational elements need to be in place before you can leverage agents at scale. There's a lot of work in the industry more broadly on new protocols for how agents should communicate. It's a very new paradigm and there is work to do before we see this really scaling up.
Penny Crosman (18:11):
That makes sense. There is more risk because errors can be compounded. Prashant, does this resonate with the work you are doing at US Bank?
Prashant Mehrotra (18:27):
Absolutely. To build on that, there are many things we do subconsciously—ingesting, processing, and actioning information. These agents have to reason, plan, and validate their plans. In buildings with hundreds of SaaS applications and thousands of products, you're going to have extremely diverse agents. We have to make sure they are interacting correctly. We are finding issues as these agents scale, so we need to plan for that.
(19:42):
As we adopt agents, we are working with external partners to ensure the capabilities are governable and meet regulatory standards and our risk posture. We are starting with what I call an "empowered human in the loop"—someone who understands what we are delivering rather than just validating in three seconds. Once we have validated the infrastructure, we will scale it out to more external-facing or less controllable environments.
Penny Crosman (20:59):
It's a very complicated environment. Kim, when companies start deploying agents, do they have to rethink access management, identity verification, and security for this new world?
Kim Bozzella (21:41):
The simple answer is yes. Firms are thinking through "zero trust" and understanding every single endpoint. You have humans, internal agents, external agents, and third-party products. You have to look at this as a blended workforce. You have to think about an agent's identity, permissions, and whether they are still operating as per design. You may even need to decommission or "terminate" them at some point. It's about adding healthy challenge to the output and building that QA into the ecosystem.
Penny Crosman (22:49):
Sumeet, how does this change the job of a manager who used to manage people but is now managing a mix of humans and independent agents? How does it change job reviews, salaries, and the dynamic of power?
Sumeet Chabria (23:22):
These are great questions. This gives rise to the concept of a "digital worker." It's highly sensitive to use the word "worker" for something digital, but if these agents start to collaborate and exhibit character, it will feel to some that they have taken their job. We've spent hundreds of years learning how to manage people, and some of those disciplines can be used for agents. Some companies just want to know what added technical controls are needed, while others are saying, "Let's get ready for a digital workforce now."
(24:17):
How does a blended human team work? How do we observe their performance? Eventually, you'll have metrics showing agents are getting 45% extra productivity. If 1,800 people did a task today, why are 18 agents doing it faster and cheaper with the same quality? It's early days, but ideas are being floated with HR teams. Some companies, like Moderna, have even merged the CHRO role with the CIO role to have a "workforce officer."
Penny Crosman (25:34):
I've heard people say AI agents will eventually draw salaries. Do you think that's true?
Sumeet Chabria (25:41):
There has to be a cost for an agent. I was never a big believer in the robotic process automation (RPA) wave because the underlying fundamental process never changed. In the end, you have to calculate the total cost of ownership. Whether you call it a salary or not, you need shadow accounting for this.
Penny Crosman (26:32):
That's fair. Kim, what new jobs might be created in this new world?
Kim Bozzella (26:38):
We're seeing it already: AI technologists, software developers leveraging AI, and many more governance and data roles. There will be more roles on the compliance and HR side to manage the workforce. It's more about existing roles being augmented with AI rather than just being replaced.
Penny Crosman (27:22):
Sumeet mentioned you can't just lay RPA or AI agents over existing processes; you have to rethink the way you handle tasks. Do you agree, and is that work you're doing today?
Prashant Mehrotra (27:41):
Absolutely. If all you do is something slightly faster or cheaper without fundamentally changing how you deliver value, you won't harness the true power of this technology. We need to think about bringing the value chain together very differently than we do today.
Teresa Heitsenrether (28:26):
At the risk of being cliché, I like analogies. Someone gave me a great one: when steam engines were replaced by electric engines, factories were originally built vertically around the steam engine in the center. When the electric engine arrived, people just dropped it into the same place. It wasn't until 20 or 30 years later that someone realized they could make smaller engines attached to specific machines on an assembly line.
(29:14):
It wasn't until that rethinking occurred that productivity truly came. We're in the same spot with AI. Rethinking the way you work is a people aspect, not a technology aspect. We've started working with senior leaders to help them understand what's possible. If the cost of the next transaction was zero, or if you could cover 20 times the number of clients, how would you change what you do? That's a hard thing to wrap your mind around, but it's the most important unlock.
Kim Bozzella (30:48):
It's also a perfect diversity opportunity. Bringing in junior people who are nimble with technology but don't have the business understanding yet can force you to think differently. They bring up ideas that might seem outrageous but are actually very thoughtful.
Penny Crosman (31:23):
Anything to add to the idea of re-engineering processes?
Sumeet Chabria (31:31):
Bolting things on gives some benefit, but over time you lose that. Anything bolted on adds risk because it has to be maintained; nothing works automatically in the background. You might have resilience or control issues. You should look at the process holistically and not just bolt it on.
Prashant Mehrotra (32:36):
We also have to think about how to educate our workforce so they understand the "art of the possible" and can use these capabilities well. Educating the workforce is critical.
Penny Crosman (33:08):
Do we have any questions from the audience?
Audience Member 1 (33:26):
Thank you for being here. My question is about upskilling and reskilling the workforce so they are prepared to move into new positions while keeping institutional knowledge in the bank. Where do you think those jobs will go so we can start preparing now?
Prashant Mehrotra (34:08):
We are significantly underestimating the creative power of generative AI. To educate the workforce, we have to look at personas—what people are doing today and what is applicable to them. We need to make sure associates have the tools, know how to use them, and are encouraged to be creative in making every process better.
Penny Crosman (35:54):
Let's have a lightning round. What is one thing you hope or fear in this AI-driven future?
Sumeet Chabria (36:12):
I hope it improves financial wellness and health. Improving access and support for people managing their money would be a very good thing.
Prashant Mehrotra (36:44):
Don't drive your technology initiatives by FOMO; drive them by business value and your vision for your clients.
Teresa Heitsenrether (37:08):
I'm on the optimistic side. I hope we see this really taking off in the next few years. For anyone juggling life and work, I hope this removes "no joy" work and makes you more efficient, giving you time back and making the work experience more fulfilling.
Penny Crosman (38:07):
All right. Awesome. Thank you all so much for coming. Great panel.