Artificial intelligence is rapidly reshaping how banks serve customers—from intelligent mobile apps to smarter branches and contact centers. But beyond the hype, most institutions are still in the early stages of operationalizing AI. This session will explore how banks are deploying practical AI use cases today—from customer service automation to developer productivity—and how leaders are building the data, governance, and culture needed to make AI work inside a regulated financial institution.
- Where AI is actually delivering value in retail banking today
- AI-driven customer service: copilots, chatbots, and agent assist
- Using AI to modernize mobile banking and digital channels
- The evolving role of the physical branch in an AI-enabled experience
- Building internal AI capabilities: data infrastructure, governance, and talent
- Practical lessons from early deployments and pilot programs
- How banks measure ROI and productivity gains from AI
- Avoiding common pitfalls: data quality, trust, and explainability
Large banks have already begun embedding AI into development and operations workflows, including internal large language models and AI-assisted software engineering supporting digital banking platforms.
LEADERS is a flagship channel that spotlights C-level executives and top experts as they discuss transformative topics for an audience of key decision-makers. We deliver thought leadership on the most pressing issues driving the future of financial services. The LEADERS series is made possible by the support from top industry collaborators including Mphasis.
Transcription:
Transcripts are generated using a combination of speech recognition software and human transcribers, and may contain errors. Please check the corresponding audio for the authoritative record.
Penny Crosman (00:20):
Hi, welcome to Leaders. I'm Penny Crossman, tech editor at American Banker, and I am here with Chris Higgins at Flagstar Bank, Ned Carroll at PNC, and Krish Swamy at Citizens Bank. And we're here to talk about the AI-enabled bank and how AI is going to change, how retail banking looks in the future, and what all of these banks are doing today to AI ready their organizations. So to start with, is there any way that someone walking into one of your banks today would notice that you started using AI or is it all kind of behind the scenes right now? Starting with you, Chris.
Chris Higgins (01:02):
It's mostly behind the scenes right now. There's a balance between protecting data, providing capabilities and services. Flagstar has chosen to build our own proprietary StarIQ platform, which is just one component of 12 in the overall S2 platform. So we're using it for high-end analysis. It's self-contained, and that's important because we have to preserve the protection of PII and confidential information. It is patent pending, and we'll continue to expand it. But we've chosen at this point for AI to really stand it up in that modality in a protective pattern. I do chair an AI strategy committee, which is also attended by our CEO and other of my peers on the executive leadership team. And so we're prioritizing a backlog of work, balancing investments in revenue and investments in expense reduction. So I would say we're probably six to nine months away from it being evident.
Penny Crosman (02:28):
Okay. Well, you've put out a lot of threads that we're going to want to tug at later. But this platform that you mentioned that's patent pending, is this all homegrown or is this in conjunction with vendors or how are you thinking it? No,
Chris Higgins (02:41):
Our distinguisher engineers built it.
Penny Crosman (02:44):
Okay.
Chris Higgins (02:45):
We imagine it on the whiteboard and we've hired the engineering talent to build it internally.
Penny Crosman (02:53):
Okay. Well, let's go back to that because we're going to want to get into the weeds a little bit on what models you guys are using. But how about you? What would you say to that? Do consumers notice AI today?
Ned Carroll (03:05):
Not in a direct obvious way, but if a customer comes into one of our branches and has a question, and that question could be explain this overdraft fee, that question could be explained the nuance of this product, the way in which one of our teammates is able to answer that question in a quick, concise, accurate way is enabled with AI. So does the customer or our client interact with it directly? No. Do they interact in the way we've established our responsible approach towards AI with one of our teammates in the middle of it? Yes, because we're helping that teammate answer that question with accuracy and speed.
Penny Crosman (03:51):
So Krish, I know Citizens is also equipping call center people and branch people with some AI to help them answer questions better as well. Can you say a little bit about that?
Krish Swamy (04:02):
Yeah, so similar to what Ned said, it's really about enabling our contact center colleagues to be able to answer questions quickly and accurately. Without AI tools, they would be going through hundreds of documents for the very specific question that somebody has. I'm sure a really good call center agent would know a lot of those answers, but when you ask something which is narrow, easoteric, you would need to consult documents, you would need to search for it. That's wait time. Sometimes you think you know the answer and you say something which may not be accurate. So I think AI serves it just as a handy tool to be able, in all of those situations, to be able to come back with good, accurate answers. So I think it makes employees' lives easier, which I think is big because firm believer that when you've got happy employees, they'll treat customers well.
Ned Carroll (05:01):
This also helps us tap into the power of our employees because every time one of our teammates is engaging with our AI, we're learning and we're improving the system itself. So in many respects, not only is AI enabling our teammates, but our teammates are enabling that because we're learning from them.
Chris Higgins (05:23):
And I think that human assist dimension is so important. I know that's a buzzword in the industry, but I actually truly believe it to be powerful. So within StarIQ, we've trained it on all things, compliance, regulation, et cetera. We've downloaded all of our branch procedures, policies, doing the same thing with call centers. So if it's a branch banker, a teller, a call center agent, they have more rapid access to the correct information, and that makes them better able to serve the customers and the clients. It's not Abracadabra. It's just faster response time. And that puts our teammates in a much better position to serve. And it's actually, I tour our branches. And what's really interesting is it's not threatening. They're looking for the help, that assist aspect of it, because it makes their accountability, their job, their role easier to perform.
Penny Crosman (06:44):
And so where all of you are using AI in the retail bank, is it strictly today to help employees do their jobs or do you also have some customer-facing applications that people might start using like Chris?
Krish Swamy (07:01):
Yeah. See, I think where we use it or where we have started using it in customer-facing applications is really to understand sometimes what customers are trying to say. So take, I'm sure we all have been on an IVR where we are yelling away, "I want to talk to an agent. I want to talk to an agent." We've all done that. And oftentimes it's because the IVR asked you, describe your problem. You used maybe some non-standard words, which is not programmed into the IVR. And the IVR just keeps repeating the same question. What we've done using the power of large language models is to be able to use it as a translator pretty much to be able to say, take what the customer's saying and map it to a small set of things that we can then give an answer to. Because the language models have infinitely more language parsing capability than any person would be able to program and write all of the 50 ways in which somebody could say, "Tell me my balance." So more from an understanding standpoint, which I think is effective because again, you get to those intents that the customers are asking for with much greater accuracy.
Penny Crosman (08:32):
And where do you see this going? Where do you see three years, five years from now? What might bank apps look like that are AI enabled?
Krish Swamy (08:42):
See, I think definitely starting to give back answers. So the kind of thing that Ned talked about, you could do that conceivably even on a mobile app. You ask a question saying, "Oh, I need instructions on how to send a wire." And the same technology that allowed a contact center agent to give an answer back, you could also have it on the chatbot. I don't think we are there quite as yet in terms of being able to execute transactions on customer's behalf because I think that requires a certain level of determinism. And I think banks, when we move money, for instance, we don't want to be accurate 99% of the day. So that's how we're cut it.
Penny Crosman (09:38):
So
Krish Swamy (09:39):
I don't think we are quite there in terms of executing transactions, but certainly to be able to answer questions as an assist for customers.
Chris Higgins (09:49):
And I think what's central to really enabling AI in a much bigger picture than maybe what we're doing today in the industry is it's not about the technology. I think we get too focused on the technology. It's all about the data. So companies have to invest yes in AI, yes in capabilities, engineering, but companies need to invest in modernizing their data ecosystem, making it of reasonable high quality. And then AI, it's a force multiplier at that point, can really differentiate companies in the marketplace by serving customers quite elegantly. But at the end of the day, it's not the technology of AI in my opinion. It's the data. Well,
Penny Crosman (10:51):
And that's something I've heard a lot. I went to a conference recently and all the bankers I spoke to said that data is the biggest challenge for them.
Krish Swamy (10:59):
Are they all GitHub people?
Penny Crosman (11:00):
Yeah. No, but - Data software. Getting data right, getting it accurate, having it in the right place so the models can access it. There are all these multiple. And you were talking about compliance documents, making sure it's the right versions that's not going back to something that's been updated five times. So Ned, you're head of AI and automation. I mean of data and automation at PNC. So you have to oversee all this. How do you look at getting the data right for data models?
Ned Carroll (11:36):
I think financial services have done, generally speaking, a pretty good job when we talk about structured data, rows and columns in the database. And with very traditional machine learning, we've gotten really good at starting to understand networks associated with that data and are able to do some really cool things. I think what AI, GenAI in particular, is really starting to shine a light on is unstructured data. So that could be a trust document, that could be a commercial servicing document, that could be an internal policy or procedure. It could be email. Believe it or not, although the industry had challenges with it, structured data is pretty easy because it's sort of binary. It's either good or bad. And you
Krish Swamy (12:29):
Can tell.
Ned Carroll (12:30):
And you can tell. Yeah. Unstructured, it's a little bit more nuanced because it could be good, it could be bad. Well, it sort of depends on what you're using for. Is the content in that email good data or bad data?
(12:45):
So I think there are nuances around unstructured data that become a bit more challenging. We participate in a public-private partnership with the financial services sector coordinating committee. And one of the work streams that we recently worked on there was around explainability. And we recently introduced a concept called the data nutrition label. So how do I know if I'm a modeler, if I'm a quant, if I'm a data scientist, or maybe even if I'm a customer, how do I know that what went into that system is good? When you think about AI now, it's not just the model, it's what went into it. How was the prompt constructed? What came out of it? So you need to have more of a systems view to it. And understanding what went in is really important. I mean, think about how much time you spend at the grocery store looking at what goes into your food and whether or not you're willing to eat that based on what you see went into it.
(13:47):
So I think you'll see more examples of that where organizations need to focus on building that transparency into the ingredients, if you will, that went into a particular system that ultimately delivered some sort of product or service or experience.
Penny Crosman (14:08):
I like that analogy. How do you create the nutrition labeled and how do you make sure it's accurate?
Ned Carroll (14:15):
So part of it is deterministic in nature, because remember when I'm looking at data, I'm looking at structured data as well, and I can be pretty deterministic. And as I said earlier, sort of binary about whether or not that's good or bad. On the unstructured side, a lot of it's around the currency of that data. What was the context that that data was built for? The context around how it needs to be used. So it begins to become a little bit of a rubric, if you will, around how to take a fairly subjective set of questions around is that data good and turn it into a bit of a quantitative answer.
Penny Crosman (14:58):
Yeah. I mean, there was that famous Air Canada example where the customer asked the chatbot, "Can I go to my grandma's funeral and apply for a discount when I come back?" And it said, "Sure, you have nine months." And then he applied and he was told, "No, the policies you have to apply beforehand." And the bank was liable. I mean, Air Canada was liable. It wasn't a huge amount, but I could see other cases going that way as
Ned Carroll (15:29):
Well. There was also the example of the ability for a Python developer to use a fast food chatbot to decode their Java, their Python. So their Python script saved all my token expenses. So I think there's the model, but then there's the harnesses and the guardrails you put around it. And I think Chris hit on this with his point around his distinguished engineers. There's a lot of engineering that needs to go into how and when you deploy AI and how and when you use it and how you go validate that what you got is good enough for what you're using it for. It's not a trivial task to engineer the guardrails around how you use these systems.
Chris Higgins (16:21):
No, and I think the guardrails are actually what's going to distinguish companies who responsibly use AI and maybe companies who don't responsibly utilize AI because it is an engineering challenge. And it's actually exposing the complexity of banking in a way that's never been done before.
Penny Crosman (16:48):
Because
Chris Higgins (16:48):
You've got people, you have process, you have data. Those are always the first three things I think about. It's actually technology's the last. And I know that sounds maybe strange from a CIO perspective, but you got to get the people, process, and data right. That takes engineering. And then you can architect solutions that can then be sustainable and predictable. But if you don't do the homework upfront and it's hard, it's a lot of hard homework, you're not going to get the backend right.
Penny Crosman (17:25):
Do you have something to add to the nutrition label discussion?
Krish Swamy (17:28):
Yeah. See, Ned, even when it comes to structured data, right? Henny, I think you said something around context. Context means a lot. Take a simple question like, give me my balance. If a customer says, give me my balance, they want balance at that moment in time. If a reporting person says, give me balance, and they don't specify anything, they're probably looking at balance at the end of the last day, maybe at the end of the last month. So balance, just the word balance or an account balance could have so many different meanings. And I think it's important to attach those labels saying this is point in time balance or as of now balance. This is balance at the end of the last working day. This is the balance at the end of the quarter. And it's important to have these things because the people asking the question may not always be precise enough in terms of saying, "This is what I'm looking for." And when you've got agents trying to understand human intent and reach the data and try and figure things out, you better be very clear about definitions of data.
(18:51):
So I think nutrition label is important for all of those balances, but balance is not just one number. It could have different meanings.
Penny Crosman (19:00):
That's a good point. Clarify.
Ned Carroll (19:02):
Crisp raises a great point, which is for years and years and years with a focus on structured data, we've organized our data for human consumption.
Penny Crosman (19:11):
Yes.
Ned Carroll (19:13):
And you've got data marts around that, you've got BI tools around that, you've got integration tools around that.
Krish Swamy (19:20):
Humans know where to go.
Ned Carroll (19:21):
Exactly.
Krish Swamy (19:22):
Yeah.
Ned Carroll (19:24):
It's a fascinating engineering problem around how do you organize your data for machine consumption, for large language model consumption. It's a fascinating engineering opportunity.
Krish Swamy (19:36):
Yeah. So
Chris Higgins (19:38):
Let's talk about - And it's just that I was going to say it's engineering at its most detailed level.
Krish Swamy (19:46):
It's engineering. It's also, I think, training humans to ask the right questions sometimes. We had a situation recently where somebody was talking about, okay, what is the credit that we have put out in Q1 against a bunch of lending products? For something like a mortgage, credit is your balance. However, for a line product like a credit card or a HELOC, is a credit the exposure that you've approved for somebody? Or is it how much of the exposure they're using? And this was humans doing the analysis. And so we were able to send emails to each other and figure out, okay, this is exactly what the person wants. But if you're not clear saying, I want either credit exposure or balance, the agent is not going to have an idea. So I think there is engineering, but there's also, I think, training humans to be, I think, a lot more precise about language.
Chris Higgins (21:02):
I think that aspect is, it's actually the most important. Yeah. You need data, you need engineering. We need to teach humans how to leverage all these new tools and capabilities. None of us grew up with this, but certainly not my 40-year career. So there has to be a balance between investment in data, investment in technology, but I think we need to all double down and invest in human education.
Penny Crosman (21:39):
But you can't necessarily train your customers to ask the right questions. You have to accept whatever they give you.
Chris Higgins (21:46):
But I think it's more about our employees, our teammates inside the company, because if we can better educate them on how to use these way cool tools that are only going to get more sophisticated and not be scared of them, but to embrace them to help them do their job better, then they'll serve our customers and our clients in a very differentiated way.
Krish Swamy (22:12):
Yeah. See, but even with customers, Benny, when things are untier, you could ask a clarifying question, which again, a contact center agent or a bank telling me not to, because they have history of having interacted with customers and they know when the customer says this, this is exactly what they want. An agent may not know, or at least may not know initially. And that's where building, I think, very explicit human interaction elements into the agent becomes important. Ask the clarifying question, which a human might never even dream of asking, but those are the types of things you've got to explicitly program into the technology into the agent to make that work.
Penny Crosman (23:00):
So Chris brought up training people in AI fluency and helping them have the AI skills that they need. What are your banks doing to get there?
Ned Carroll (23:12):
I think most effective to take a combination of a top-down and bottoms-up approach. So from a top-down perspective, there are skills that are important. There are some that are going to be more important. How do we coach, provide from a training and education perspective, people to get at those skills, those new techniques, what have you. From a bottoms-up perspective, the tools. Get them to the tools. Do it in a safe way. Yeah.
Penny Crosman (23:42):
Let
Ned Carroll (23:42):
Them play with
Penny Crosman (23:43):
It.
Ned Carroll (23:44):
But let them get their fingers on it and play with it. Some of the best innovation we see is at the edges. And it's at the edges of where our teammates are interacting with our customers and our clients. Our teammates are even sometimes interacting with our own internal processes. We had a recent sort of company-wide experience called agent hack, like hack your process with agents. One of our finalists were two young tellers who had an idea around how to deal with a problem that they experience on a daily basis. We enabled, empowered them with the
(24:29):
Tools. Now, did they build something that was ready for production? Absolutely not. But they built something that you could touch and feel and see the value in it. Far more value than if they'd put together a PowerPoint that went up the food chain, so to speak, to determine whether or not it got approved for funding, et cetera, et cetera, et cetera. They were actually able to do something and they were able to do something without having to start a project, call an engineer, do this, do that. They're actually able to do something, put their fingers on a keyboard and build something.
Krish Swamy (25:04):
Yeah. I think you're exactly right. I think giving people that opportunity I think is a big unlock that all leaders can do. Anyone who's listening to this podcast can do. We brought a data and analytics training that we've made. It's a big suite of training which we made accessible and free for all of our employees. Every month we've got a data analytics spotlight where I get to read out the top 10 people who've spent the most amount of time. Invariably, eight out of those 10 people have nothing to do with the data analytics job family. They are QA engineers, they are security people, they are loan processors, they are tellers. People have an interest in learning this stuff. And the more you can put it in front of people, they'll find good use to it. And then I think you got to take some of those bright lights and then get them to be advocates.
(26:13):
In our commercial bank, underwriters who prepare credit memos or who do credit analysis, some of them got pretty skilled at using Copilot to be able to go through documents, identify, extract industry trends, business trends, and so on. They created a prompt library, which then they shared with their entire group. So now everybody was able to start doing that. So there'll be those early adopters, but then how do you get those early adopters to share their knowledge with the rest of the organization so that you start to build scale? Because everyone is not going to have either the interest or the inclination or the time to do that kind of learning. But when you see somebody else doing it, they're like, oh, you know what? I can make my life easier by using it.
Ned Carroll (27:09):
We've turned those into communities of practice actually. And what's fascinating is they don't form because somebody from on high said, "Build this community." They finish organically. They organically emerge and you start seeing these cohorts start to form and they really do some fascinating and great work. And then you tap into that and you shine a spotlight on, like Chris was saying. You say, "This is the example of how we need to be working."
Penny Crosman (27:42):
Well, that's a good segue into what I wanted to ask you about token maxing and where you're encouraging people to use AI a lot, but then your AI costs go up. And some companies are rethinking that and they're thinking about, do they want to use open source models more? Do they want to use proprietary models more than the frontier models that are getting more expensive? How do you guys look at all of that?
Chris Higgins (28:07):
Well, that's actually why we built our proprietary solutions, StarIQ, not relying on tokenization. Safe, secure, self-contained, quite elegant, part of the S2 platform. And what's really interesting around StarIQ, I'm not a betting man, but if I had bet, I would've thought my technology organization, because I run tech and ops, but I would've thought my technology organization would be the number one user of Star IQ. Turns out it's risk management and marketing. And they're the ones that are doing more than dabbling. They're actually create some pretty unique things.
Penny Crosman (28:57):
Like what? What are they creating?
Chris Higgins (29:01):
They're taking, even though we haven't centralized all of our data, they're figuring out ways to go find the data that is relevant to marketing campaigns. Our credit underwriting is looking to find the data that's relevant, most relevant to making more rapid underwriting decisions. Because the faster you could make a really good decision based on data and analytics, you win in the marketplace more times than not. But it's this creativity and this imagination that is being unleashed. And it's not for everybody. And I would've thought my technology team. Now my distinguished engineers, very nice cadre, they're the ones who built it. I though others in the technology community would latch Onto it. It actually didn't turn out that way.
Krish Swamy (30:03):
Yep. See, but in a sense, tokens are resources and I think technology and data teams are used to managing resources like SCARS commodities. So in a sense, I'm sure all of us at various points in time have had the situation where somebody writes a poorly constructed SQL query and it runs for hours and hours, consuming resources. What do you do? You go and shut it down. You reach out to the person, you coach them, tell them, "Hey, you know what? You shouldn't be running the query like this. What are you trying to do? You should maybe structure a query this way and then you'll get more efficient results. You'll get results way quicker." I remember this was 20 years ago, 25 years ago when I started as an analyst. My data engineer friends would tell me, okay, when you create a table, create indices or set indices when you do that, because that makes it so much easier when you're on the query on it.
(31:09):
And I didn't know what set indices was, but I did it. I believed them. And yes, it optimized resources. So I think there is a lot of that type of education again. I think we've got to do with users to be able to say, "This is how you get more efficient at running stuff." And people will be willing to learn when you do that. And there'll always be the person who's like, "Okay, you know what? I'm not going to listen." And then you reach out to them in person. And once people realize that, hey, there are resources available for me to learn and do these things more effectively, people by and large will be willing to do that. So I think we'll go through all of that. I remember database technologies 20 years ago were super expensive. So you had to figure out a way to conserve CPU cycles.
Ned Carroll (32:16):
I think, I mean, everybody reads the press about the out of control token expense, the token leaderboard, so forth and so on. I mentioned, Chris and I both mentioned about putting tools in the hands of people and putting guardrails so you don't cause harm. Well, one of those potential harm is unnecessary expense.
(32:37):
So how do you put guardrails around that? I think what's obviously emerging is you need to be thoughtful and have optionality around the right model for the right problem. I wouldn't want to introduce and leverage a foundational model that I access via an API into a highly scaled, scalable operational process because business school 101 says don't introduce that kind of variable expense in a process that scales. So I want to be more thoughtful and deliberate about being able to control that expense, that unit when I talk about a process like that. The fact of the matter is I don't need a model that knows advanced calculus to address some of the questions and problems that I want a large language model to do. Now, there are some scenarios where I do want that or do need that, but not all. So I think architecting your solution such that you have optionality and that optionality is governed, meaning it's not by choice because if by choice, I'm always going to use the best model.
Krish Swamy (33:56):
I think it's a classic, it's not gold plating your solution. Again, this came many, many years before the first transformer model was built. You don't need to gold plate your solutions. So you can get back to your calculus versus reading a policy procedure kind of example. There are models that will give you a response within a fraction of a second or within a second or within two seconds. Does every use case require answers within two seconds? No. If you're reviewing contract documents that gets submitted and you want to run a nightly batch process, you don't need the answer back in two seconds. And the token costs for submitting something in batch and getting something back are way lower than what it would be for a real-time response. So I think you've got options, but to Ned's point, you've got to engineer those options and set those hard boundaries in advance.
(35:04):
And
Chris Higgins (35:05):
That's what we've done within StarIQ because we don't want to over-compute the solution. So there's three tiers, the equivalent of a bachelor's degree, a master's degree, and a PhD. But you don't get to default to the PhD because that's time, costs, money. And so we created engineered criteria. So it's more fit for purpose. And I'll use the horsepower of a PhD to solve a highly complex problem, but I don't need to do that for a bachelor's degree. And that's literally how we've configured it to try to optimize that risk reward, cost benefit type of scenario. You've got to manage it. You just can't let it run.
Penny Crosman (36:04):
It's interesting that you're all using some of your own proprietary AI models. Are you putting these in your own data center? Are you running these in your own data centers or are you using cloud or is it a combination?
Krish Swamy (36:19):
So we are migrated 100% to the cloud. So when we run it on the cloud, we will do it within our VPC. So nothing goes out of the VPC, out of the firewall. So it is hosted within our VPC and we run it within our VPC. But we do it for some use cases. We don't do it for all of them. Because Andres, would be interesting to hear how you manage it. Because setting up a model instance and running it is also its own kind of engineering task. There are some situations where you're like, "I'm okay with the frontier model, people running it. If I get a choice of the bachelor's model versus the PhD model," which most of these people provide. If you've got to run it in your data centers or in your VPC, then you got to worry about uptime and maintainability and residency and things of that sort.
(37:20):
Which may be important in some use cases, may not be important in other use cases.
Chris Higgins (37:27):
Now we just modernize our entire data center footprint. So we're three banks coming together to equal Flexstar. So three banks, you have a primary and secondary. So we had six data centers. We partnered with Emphasis and in 12 months we built two new data centers, Ashburn and Chicago. They're co-losed. So Equinix, we don't own the data center footprint, but they're perfectly mirrored and we engineer all the specifications for that. StarIQ runs in an AWS tenant. Highly secured, over-engineered, some may say, but we're just relentless on protecting the safety of our customer and client data. And that's where we've also engineered into StarIQ that balance in terms of metering out the usage based on the problem you need to solve. So it's not perfect, but it is relatively fit for purpose.
Penny Crosman (38:48):
Well, you mentioned safety and you've mentioned compliance a couple times. I've heard that regulators are asking banks more specific questions about how they're using AI. And they've done this before, but it seems like it's something that they're scrutinizing these days. Do you have any comments on how to make sure that what you're doing is keeping your regulators happy or not unhappy at least?
Krish Swamy (39:13):
See, my answer to any question about how do you keep regulators engaged and satisfied is just be very transparent with them. I think transparency matters. They're smart people. They will point out things which may not be obvious to you. And so the more you're transparent with them and the more you just embody the spirit of, you know what? We are here to do good, safe work. Let's show you how we are doing good, safe work. Give us your feedback on things that we could do better. I think they'll engage and they engage and that's exactly the posture we take with them.
Ned Carroll (39:59):
I mean, I would second the point on transparency. I think the key thing is we got to recognize that it's different. Banks and regulators have great knowledge and comfort on how to look at model risk management when you think about it from a deterministic point of view. You've been very good at it for many years. From a regulatory perspective, from banking perspective, safety and soundness, et cetera, et cetera. GenAI is different. It's different in that it's by definition non-deterministic. It's different in that the unit of observation is different. It's not just the model, it's the system, it's the prompts that go into the model, it's the output. And it's
Krish Swamy (40:51):
Not one prompt. It could be a whole - Then it could be a whole collection.
Ned Carroll (40:56):
So I think first and foremost, recognizing it's different is a really good start coupled with being very transparent about how you're looking at it. I mean, obviously I'm sure as Chris and Chris both do as well, ensure that you're aligned with what's there, which is mostly NIST-based and take very much of a principle-based approach in how you're addressing that. At the end of the day, it's about your customers. It's about the data and ensuring accuracy and consistency.
Chris Higgins (41:30):
Yeah. Yeah. And that's when we engineered StarIQ on the whiteboard and then built it, tested it, et cetera. One of our leading principles was high audit ability. So we can push buttons and produce an inordinate amount of detail on step-by-step, transformation by transformation, everything.
Penny Crosman (42:02):
Even with generative AI?
Ned Carroll (42:04):
Yes. We
Penny Crosman (42:04):
Could do it.
Ned Carroll (42:05):
So I worked for Chris when he ran technology audit, so trust it. It's allginable.
Penny Crosman (42:14):
It's not being generated.
Chris Higgins (42:16):
No, no, because that's so important to be able to prove that you're safe and sound.
Krish Swamy (42:24):
And when something goes wrong, to be able to diagnose what went wrong.
Chris Higgins (42:27):
Right. And then make a correction or adaptation. But that's the key.
Ned Carroll (42:34):
It's an interesting point because it sort of goes full circle to something Chris said earlier. Historically, we've thought of governance as a risk management type of problem. So you start thinking about process and control and all the things that you typically think about with risk management. Governance in this space is an engineering problem.
Chris Higgins (42:56):
Yes.
Ned Carroll (42:57):
Because you have to be able to scale it. And the traditional sort of process and risk and control frameworks don't easily scale to the volume of signals and telemetry that you now need to observe when you're looking at AI these days.
Penny Crosman (43:18):
That raises a good point because I wanted to ask you guys about getting from pilot purgatory to scaling up a model across a large organization, which you all have. Do you have any thoughts on how you've been able to do that? And can you give an example of something where you started small and broaden it out through the whole organization?
Krish Swamy (43:39):
Yeah. So maybe I can go first with, again, going back to the example of contact center and putting knowledge articles in front of them. Started out with a pilot, contained pilot, very specific area. We added more documents, expanded both business level coverage, product level coverage, so that very quickly we were able to get to that point where we started consuming all the documents that the contact center would use. And I think the important thing here is when somebody else comes up with the exact same idea, point them back to the solution. And people will come up with ideas. They're bound to come up with ideas. They should be coming up with those ideas. But what you do there is bring coherence and consistency and say, "You don't really have to build your own solution here. Why don't you point it back?" I think a lot of the problem that you see out there with pilots is because organizations either lack the means to know that things are going on or sometimes lack the discipline to be able to stop efforts and say, "You don't need to do that because somebody else is doing it.
(45:10):
And so partner with them." We do it all the time. We've had situations where somebody wants to build a new document intelligence platform and we'll point them right back to somebody else is building the same thing and they came four months ago. Why don't you try and partner with them and try and see how both of you can leverage your own solution, the same solution versus trying and doing something on your
Penny Crosman (45:37):
Own? That makes sense. I
Ned Carroll (45:38):
Mean, exactly what Chris said. The only thing I would add is focus.
Krish Swamy (45:46):
Yep.
Chris Higgins (45:46):
Focus.
Ned Carroll (45:47):
Yes. Focus. You read about the hundreds of use cases, and I'm not sure when use cases became a metric of success, but for a while when you read most earnings reports, people would talk about number of use cases,
Krish Swamy (46:07):
A number of agents
Ned Carroll (46:08):
Or number of agents, number of agents on use cases. So I don't know when that became a measure of success, but for a while it sort of was. And what you ended up creating was this risk of just spreading the peanut butter very, very thin across bread. And we know what happens. The bread starts to break apart. So I think picking a few things that you just wanted to get really, really good at. It's not to say you shouldn't work on other things, but you're focused on the things you want to get really, really good at. And it's fascinating that you think that that's the lesson. But I think when you get caught up in the exuberance of, well, I could do this and I could do this and I could do this and I could do this, it's easy to lose sight of what is it I really need to focus on?
(46:55):
And what is it I really need to learn? Can
Penny Crosman (46:57):
You think of one example where you've chosen one thing and you've scaled it across the board?
Ned Carroll (47:03):
I mean, for us, it was knowledge search. One of those examples was knowledge search and it started small similar to. I mean, it's the exact same pattern that I would just be repeating what Chris said, which is how do you start small? How do you make it a closed loop learning system? Because the interesting thing is, unlike other instances where if you told me, well, go do what they're doing, I would've said no, because I want to control it myself. I want to do it myself and all that. Well, the interesting thing around GenAI is when you do it together, it gets better because it starts to learn. You can learn better and you can accelerate how you learn, and then just continue to sort of momentum. It's the snowball.
Chris Higgins (47:49):
So we've taken a two-pronged approach. So StarIQ, because it's safe and sound is where experimentation happens. But when we choose to pick a business case, we build for scale. No experimentation. I mean, it may start over in StarIQ, but then we go build for scale. And we're very deliberate. We can do four, five, six meaningful implementations of the capability from a people process data technology perspective. But to Ned's point, we don't want to spread the peanut butter so thin that it rips apart the bread.
Penny Crosman (48:41):
You don't have 120 use cases.
Chris Higgins (48:43):
That makes a bad sandwich at the end of the day, and it's not a sound business case. So we play it both ways. We pick and choose where we want to scale.
Penny Crosman (48:56):
Gary, final question. If you could only invest in one AI technology over the next two years, what would it be? No one wants to go first. I
Ned Carroll (49:08):
Mean, observability. I think when you think about the potential of inherent complexity that could get created with agent deployment and so forth and so on, especially if you begin to use low-no-code platforms that provide that capability to the broader teammates, I think being able to have observability around that system, understanding what are those patterns, and how do you begin to be able to detect and then prevent patterns that you don't want? Observability.
Chris Higgins (49:54):
I would pick, and we are within the S2 platform, the most sophisticated workflow case management system who's augmented by AI. It's not about AI. You engineer your processes to create an experience that differentiated in the marketplace. AI is a hammer of screwdriver. It's a tool in the toolbox, but you have to use it deliberately.
Penny Crosman (50:26):
And why workflow? Because of all the cost savings you could get from that?
Chris Higgins (50:31):
Cost savings, decision time to market, answer customer's questions faster. I'm a black belt in Six Sigma, earned it the hard way. But to me, everything is a process and a workflow. And you have to define your customer experience upfront, build the process, and then identify the data. Now, thank goodness we have AI, Gentiq AI that makes those workflows and that data more valuable to serve customers and clients, but it's not Abracadabra AI. It's part of an integrated workflow management system.
Penny Crosman (51:17):
How about you? What would you say?
Krish Swamy (51:19):
I'll pick context. Providing context within an AI application goes back to some of the examples we were talking about, like the balance example. Can we build context to say when Ned is asking for balance, he's a customer and he wants a real-time balance. And Chris is asking for balance, he's a financial analyst, and he's looking for end of month balance. When Penny's asking for balance, she is trying to compute maybe ODA fees or something else. And so she's looking for end-of-day balance. Yes, you can train humans to be more precise, but you can't always be successful doing that. And that's where context ends. Again, and there's this whole fascinating space around context engineering that's, I think, starting to take shape and becoming a science in its own right to say, how do you provide the right amount of context so that you get good answers?
(52:29):
You give the example of the Air Canada chatbot. If you trace through what happened, the chatbot kind of contradicted itself in the middle of the conversation with the customer. It gave one set of answers and then seven or eight points down, it gave a different set of answers. Because a lot of these models, they've got this thing called a sycophancy bias. They tend to agree with what somebody says. How do you manage context so that you're able to overcome that bias and allow the chatbot to remember what it said and not just agree with whoever's asking a leading question or something. So I think context is going to be big and I think it's starting to emerge as its own area.
Penny Crosman (53:22):
Yeah, it's such a fascinating dynamic where the frontier models want to please you by telling you what they think you want to hear. Yeah. It's kind of wild. Well, Chris, Ned, Chris, thanks so much for joining us today. And thank you all for listening.
Chris Higgins (53:35):
Thank you. Much appreciated.
Penny Crosman (53:37):
Thank you.