Enhancing Business Outcomes with Generative AI: Driving Innovation in Data Analytics

Generative artificial intelligence is driving significant innovation in data analytics, which, in turn, is accelerating business outcomes and mitigating risk in banking and financial services. It's also changing how customers interact with and benefit from their banking relationships. This expert panel will delve into the evolving use cases of GenAI within banking and provide a roadmap for success.


Transcription:


Penny Crosman (00:13):

All right. Welcome everybody. Can you hear me? Welcome to this session on generative ai. I am very fortunate to have two people with a lot of expertise in the area. We have Derek Waldron, who's Chief Analytics Officer at JP Morgan Chase. Last night, Derek and his team received our innovation of the year award for their LLM suite. And thank you. And you're going to hear more about that today. It was a huge project. A lot of work went into that. And we have AlexJimenez, who's a Consultant at Backbase. He's been a consultant at several other firms, including EPA in the past. Worked with a lot of banks over the years on a lot of different kinds of technology projects. So to kick off, there seem to be different ways of looking at generative ai. Can each of you give your definition of what Gen AI is? Go ahead.

Derek Waldron (01:15):

Go ahead. So we throw around the word generative AI a lot. So it seems interesting to be talking about how we define it. We define it very simply at JP Morgan. It's just the use of AI to create any form of new content, be it code or a text or videos or documents. And colloquially, we tend to use it quite synonymously with large language model usage. But I think what's really important to note is that it's not a strict one-to-one. And that nuance is very important, particularly when we talk about governance, because sometimes the use of generative AI comes along with different risks and controls. A couple of deviations of the one-to-one are that there are other non large language model based generative AI solutions, for example, in marketing to be aware of. And then there are usages of large language models, which are non generative incapacity. And that's important to think about. So large language models can be used for classification purposes. It can used for verbatim textual extraction use cases. And in those cases, the use of large language models, we would not consider it to be generative. And that would then come along with certain simplifications in governance and controls.

Alex Jimenez (02:41):

I don't think any improve on that. But one of the things why we wanted to start this discussion this way is that out there in the world, we hear people saying ai, AI and ai, and what they mean is a generative ai, and it's not the same. So recently I was asked, how do you use generative AI to do better underwriting? And I corrected them on the question because to me, that is not the best application of generative AI or the best way to use AI for underwriting, right? There's other kinds of AI that you use for underwriting. Now, there is a piece that you could use generative AI for underwriting, but it's not the main thing that would be driving that sort of use case. So when we're talking about generative, I think of the word generative. And so it's generating language, generating video, generating images, generating code, which is a kind of language. It is not how you crunch numbers, it's not how you use AI for decisioning and so on. That's different types of ai. So I wanted to clear that up. I think all of us probably know this, but when we are out there talking to other people, they throw out the term inappropriately and we got to be careful when we talk about it because you're not going to use CHE GBT to drive your under underwriting decisions.

Penny Crosman (04:19):

Alright, good distinctions there. So where do both of you see generative AI having the highest value in organizations like banks?

Derek Waldron (04:31):

So I think, I love the distinction that not all AI is generative ai. And that's something that I also spend a lot of time when people ask, how are you using ai? We need to distinguish, and the way we talk about it internally is traditional machine learning versus generative ai. And the reason that distinction is very important is because where we get value from today is still predominantly traditional machine learning. So JP Morgan at past investor days has communicated value attributed to the use of AI between one and a half and 2 billion annually are the gross benefit numbers that we've communicated in the past. And the vast, vast majority of that still today is traditional machine learning, primarily in the domains of credit underwriting, instrument pricing, deposit pricing, marketing and fraud. And I think that's probably the same for most banks because those are the use cases that really address the economic leverage points of a bank.

(05:35):

Generative AI is newer, it's the fastest growing part of the value of the portfolio. And the nice thing about the technology is that it tends to open up a whole new set of complimentary domains above and beyond the ones we're using. Traditional machine learning for generative AI is primarily being used as a productivity play, but the types of domains where we're capturing the most value from it are code generation and creation in technology, front office support. So think investment bankers, private bankers, wealth managers who would use generative AI to very quickly assemble all the relevant information curated to a particular client's situation and automate the creation of a briefing note that pivots much more time and attention of front office employees to spending time with clients having quality discussions instead of preparing for them. The third domain is all of the support functions, legal, wholesale, credit risk, procurement areas that are very heavy in paper documents and unstructured documents that before were very out of scope for traditional machine learning, but now gain tremendous benefit from the use of large language models.

(06:59):

Operations is an area where we are revisiting all of new automation potential. In the past we've had RPA and trying to do straight through processing, but the bottleneck was always vision, document, understanding, human reasoning, these types of things, which can now be very aptly. Many of them tackled with generative AI technology. So it's rekindling a whole new look at how we can increase straight through processing rates. And then finally, general purpose, which we call out as a category because generative AI is the first form of ai, which is really quite democratized. It's generally useful, and I think we'll talk a little bit about what LM Suite was trying to do, but it was really trying to equip the preponderance of our employee base with general purpose useful tools. And that turns out to be a huge driver category in its own right as well.

Penny Crosman (07:55):

Thank you. How about you, Alex?

Alex Jimenez (07:59):

Two years ago when Chad GPT was presented to the public, and a lot of people said, great, now we're going to close call centers. We're going to use that as a customer service tool. That's yet, it's yet to happen, right? For all the reasons we probably all know, and we'll get to some of those in a minute. But an interesting use is actually getting the feedback back from the client or the questions from the client using generative AI to put it in a format that the rest of whatever the solution you're using to respond to a client understands. And so it's actually not responding to the client using large language models, it's actually getting the information from the client. You're using a large language model. Glia is an example of a tool that does that. It's really interesting to think it's not what's being generated is what a customer says, translate it in a way that the system understands it. So we can give a already defined answer. I think that's a really interesting application that I just learned about today.

Penny Crosman (09:14):

So understanding the prompts, the thing people are asking

Alex Jimenez (09:16):

Exactly,

Penny Crosman (09:17):

And then maybe you change your products accordingly or you change your message about them. And we did actually, one of our other award winners last night was truist for its client pulse product, which is trying to take all feedback from customers all over the organization, whether it's through the app store, through the call centers, through any other mechanism where customers give feedback and using generative AI to analyze all of that and start to, like you're saying, craft better answers and address those things. So Derek, can you tell us a little bit about what is LLM Suite, how did you build it and what does it do?

Derek Waldron (10:04):

Yeah, so LLM Suite was a project that we started two years ago, and it was really a full on bet to provide self-service generative AI capabilities to our employee base and our business teams at large. That was a very different sort of framing In the past. In order to do any type of use case in the bank, you would take the business problem, you would get the data, get a data science or AI team, get an application engineering team, build out that solution. That was the traditional cycle in the past. And once you take all into considerations, all of the controls and reviews that banks have, that process can easily take six to 12 months plus. And generative ai, it is a very democratizing technology. You can prototype very quickly, you can solve general purpose problems, you can assemble problems. And what we wanted to do was really empower the employee base and our business teams at large in a safe, secure way.

(11:08):

So LM Suite was the platform to do that. It was solving a number of different things simultaneously. First of all, it was designed to enable safe access to specific onboarded large language models. So we curated the very specific models that would be enabled. We ensured that the data lineage on those protections were in place so that we could feel good about using that with the various forms of data internally that we had. The second thing is we wanted to build it as an abstraction layer from the models we didn't know. And we still don't know how the model landscape is going to involve, who's going to win, what models are going to be good for what. So we wanted to be able to have a layer that we could flexibly pivot or switch depending on the application, and that is to position it differently than some of the commercial solutions, which are hardwired intrinsically into one specific model.

(12:04):

And then the third thing that we wanted to do was really democratize self-service access. And this started out as just being what would look like a chat GPT two years ago. It was basically just a chat bot terminal. But over the last couple of years, we've significantly built out the capabilities now where we've added in value added modules, data connectivity, agentic workflow. So now if I'm sitting as a banker, I can do things. I'm meeting client X, Y, Z, help me prep for it, and LM Suite will come and pull in information from many different types of systems, news, earnings releases, corporate databases, research, and then assemble client ready PowerPoint presentations or briefing packs. And that's done in 30 seconds. So it's powerful as an ecosystem. And then we've also created the capabilities then for teams to be able to configure and assemble their own solutions by giving their particular tenant or instance specific instructions, specific tools, specific constraints to tap into the at scale development that's across the organization. And that's been really, really successful. So what we've seen is that today about a quarter million employees at JP Morgan have access to the tool one in every two JP Morgan employees globally use it every day just for general purpose use. But then we've also had hundreds of business teams that have taken the platform and designed very specific assistance or workflows or other tools that they can to help automate and bring value in their day-to-day operations.

Penny Crosman (13:46):

And have you done this all yourself or can you give any vendors out here hope that you might work with one of them?

Derek Waldron (13:52):

It was a big team effort. One of my colleagues business and G up here, he's led AI for that. But JP Morgan was fortunate enough to have a lot of AI and ML engineering talent. That's our CEO has been a real proponent of investing in AI for many, many years. So we had the talent. So two years ago when it became clear generative AI was a very exciting, transformational technology. We had the talent internally that could be redeployed on this, and it was the contributions of dozens of people over the last couple of years.

Penny Crosman (14:27):

Alex, are you seeing other banks wanting to do something similar or

Alex Jimenez (14:32):

Yeah, I would say all the banks have AI somewhere in the roadmap. But does that mean it varies? So for us at backbase, we are using ai. About two years ago we were in the same place. We were thinking about should we do and we opened an AI center of excellence in Vietnam, and we started playing really to find what we wanted to do. And so now we've implemented a data layer within our system that includes agents, AI agents, to do all kinds of things and all kinds of use cases that use the capabilities. So we are doing things like using the generative AI to present a daily summary of a wealth management client's position.

(15:40):

If you're a wealth management client, generally you get your summary of your portfolio once a month written by your advisor, and of course it might be old by the time you read it. So AI can produce language that drives, that tells you what has happened in the market in the past day, or it's happening right now and how it's affecting your portfolio. That's a use case. That was one of our first use cases. And so we're presenting that to our clients. Our clients are getting very excited about the use, but everybody's very nervous about it. Everybody's thinking about what guardrails do we have? And one of the challenges particularly for a partner like us is that we are building AI into our platform, as are many other organizations. And then we're going to a bank, and the bank doesn't own that ai and their concern is we're going to have 10 vendors, all of them doing ai, and we don't have control over it.

(16:55):

And so there's a lot of concern about governance, a lot of concern about the impact, how is it going to comply with rules that eventually will show up, all of that. So banks are being really, really careful. And then of course we see things out in the market that makes people even more careful. So a year and a half ago, air Canada rolled out a chat, GPT call center, sorry, chat bot to other clients, and he gave some wrong information to a customer. It's a long story, you can look it up. And they had to pull it out. They had to pull out the chatbot because of the information that they gave wrong to a client. So those questions, those sort of things, we don't see it in the press and bankers are very nervous about it. On the other hand, that doesn't stop people out of banks to start using the platforms that are out there.

(18:05):

And there's been instances where OpenAI has actually gone to a bank and said, Hey, do you guys know that there's some of your customer's information in our database? You got to be careful. So number one thing you should be doing right now is figuring out what tools you're going to use, what governance you're going to have, what directions you have for our clients. And it could be just as basic as, Hey, you can't use these tools whether personally or otherwise, or this is how you use them and this is the guardrails that you're going to have. You have to be out there and putting that out because right now it's the wild west, unless you're doing something like what JP Morgan is doing. But Google and Microsoft are already putting copilots in their platforms, and we at backbase are using Gemini and using Microsoft's copilots as well just for the general use as you were saying. Right. So it's early days at this point,

Penny Crosman (19:15):

But

Alex Jimenez (19:15):

Everyone wants to do it.

Penny Crosman (19:16):

You raised some interesting points about all the different things that banks have to be careful about, and I think it's interesting that banks are so worried about getting their proprietary data or customer data into these models, but as you say, employees are going off on their phones or on their personal laptops, putting things into data, GT asking questions, getting the answers, and it's like there's a lot of stealth mode AI going on.

Alex Jimenez (19:41):

Yes.

Penny Crosman (19:42):

So obviously these were concerns for you, Derek. How have you protected the bank's proprietary data? The bank's customer data prevented bank data from being sucked into large language models, and also making sure that models that have been trained elsewhere are going to give answers that are truly accurate and appropriate for your employees.

Derek Waldron (20:07):

So the risk around employees personal using bank information and in their own personal devices is really quite real, more real. I think that it's ever been. It's so easy now you can actually just take, a employees could in theory take a screenshot or a photo of a screen and use a visual model. There's just so many ways now that data can get in. That was one of the motivations that we had two years ago. We said, well, we're going to have to enable the technology in a safe way because if we don't, there is the equally opposite real risk that employees will just go and use it in other ways. And so we took the steps that Alex described. We were very deliberate, said, okay, what are the ways in which you're going to permit large language model technology to enter the firm? We wanted to have an inventory of those needed to conform to the right lineage, the right standards, the right contractual protections, et cetera, and we shut down everything else. So we have a very wide category block in effect where if employees want to go to some website, which is not an approved channel, it's blocked by the network itself. Of course, there's always the risk that employees could go around, but the idea is that by giving them really, really good tools that work with JP Morgan's data in a safe and secure way, there's actually very little incentive to have to go then around that. And I think that's been our primary protection.

Penny Crosman (21:39):

And just to follow up on the, you talked about data lineage, which I think is very interesting because chat P two is basically trained on the entire internet is my understanding. So how can you be sure when you're using a model that's someone else trained, how can you be sure of the data lineage?

Derek Waldron (21:55):

Yeah, yeah. So it's a difference between the training process that it went through, which absorbed all of the data and then how it's hosted. And the models are generally hosted. They're frozen in time, so they're not continually learning. And most of the major providers and the cloud providers now do offer enterprise grade hosting in the right architectures with the right contractual provisions that can meet very, very high grade security requirements to be comfortable of how data is being used. It's not so different than how banks have gotten comfortable with just in general storing quite sensitive data in the cloud already. They've gotten comfortable, regulators have gotten comfortable with those constructs. And this is just the analogy to that in the large language model world, but you have to be very, very, very, very careful because reputable enterprise offerings will have the right protections, but other companies may not. And you need to really kick the tires to understand what's the contractual provisions that are in place around the data, what's the tech stack that're working on what's even fourth party risk? So if you're using a SaaS that in turn uses large language model providers, do they by extension, are they using it the right way? There's a lot to think about, but when done the right way, I think banks can get very comfortable with solutions out there.

Penny Crosman (23:31):

And are you seeing the outcomes that you hoped for so far?

Derek Waldron (23:34):

Yeah, very much so. As I mentioned at the beginning, most of the value that we've still see is traditional machine learning, but generative AI is the fastest growing component of that and will be, I think into the future. Just as an interesting anecdote, LLM suite is a general purpose productivity tool. We track through surveys, the type of productivity people report that they gain, and it's been steadily increasing. So last fall, it was just under an hour, then we saw it go to two hours. Then most recently, as of last month, it's four hours per employee proactive users. If you multiply that out by the size of JP Morgan's employee base, the numbers are really significant in terms of capacity that's being created, plus that people love it. So it's created quite a viral cultural shift in JP Morgan, which I think is turbocharging innovation into the next wave.

Penny Crosman (24:42):

And one thing that I don't see too many banks doing yet is customer facing generative ai. What do you guys think? Do you think that that's coming and how long might it take for that to come and what might that look like? Is it a generative AI chat bot that just has some guardrails around it?

Alex Jimenez (25:02):

It's definitely coming timeline. It's really the question. The challenge of course is that the capabilities are there, but whether it's giving the right answer or not is the big question. So all of us have seen many examples of hallucinations coming from large language models. And so models like the Che GPT Open AI version three for example, was kind of using its own generated text to come up with for further learning and therefore just getting dumber and dumber. So some of those things need to be addressed. Some of those things need to be improved before we actually allow it to answer questions for customers.

(26:04):

Earlier, we were talking about this before, but there's a FinTech that rolled out that closed down their call center and turned chat GPT as their cult center. And that did not go well. And a year later they had to rehire or hire a new people for the brand new cult center because that was an issue. And so it's just too early for it. It's too early for those of us in banking that have been there for a long time to really feel that it's appropriate and it's ready to do that. So I think a lot of us are testing those capabilities, but it's way early. If you talk to some of the vendors out there who do chatbots and so on, they're all going to tell you that they're using it, they're testing it, but they themselves don't feel as comfortable. It's not appropriate for a regulated industry. You do see it out there in the world. Retailers have bots doing some interesting things, but the stakes are much lower than financial services or healthcare or insurance.

Derek Waldron (27:33):

I think well said today, JP Morgan Chase doesn't use generative AI in client facing channels. That's been a risk appetite consideration, and we've ensured that it's only for internal use, primarily with human in the loop. But it is coming. That much is clear, and I think that the way we'll see it begin to emerge in client facing channels is by sort of stepping into it. So the first modality of usage, I think when JP Morgan goes live with its first generative AI client facing channels, it will be mostly used for semantic understanding of the client's issues, which it's very, very good at. So it can understand exactly what the client is trying to get at, and then map that in much more sophisticated ways than was previously possible to help resolve it. But there's not going to be a generative component where the large language model is generating the language that will go to the client that will likely still be canned or heavily templated, et cetera.

(28:38):

And then over time, these things will be opened up. It'll be allowed to be a little more creative, a little more freeform and templated. But the risk that I think that everyone needs to grapple with is the fact that a large language model in each generative mode, no matter how compelling it seems, no matter how much you test it, it's an intrinsically statistical system. So you can put into the prompt, here's the policies to conform to, here's the templates to conform to, here's the things not to say at the end of the day, how do you know that it won't do it? What's the probability? Even if it's 0.0001%, which would be very hard to test when you're a large bank, the size of JPMorgan Chase with a hundred million customers using a Chase digital app, the likelihood to manifest itself is there. And I think that's what the whole industry is going to have to grapple with, is how do they ultimately get comfortable with an intrinsically statistical system in a very highly regulated. So the good news is there's many, many, many steps you can step into it. But then I think to advance that, there's still some foundational technology questions that need to be solved.

Alex Jimenez (29:46):

One of the challenges though, it's the tools are out there for people to do in other industries, and there's going to be an expectation that for my friends at B of A, that Erica is going to be driven by chat GPT at some point or some other language model. And it's frustrating as a customer of using Erica and asking it a question, and it doesn't give you what answer that makes any sense. And you're thinking, what OA spends a ton of money on technology, why can't get this right? So the expectations from the customers are going to be such that we're going to have to get to a point where we have something that can compete with an Amazon or something else that's using these sort of tools.

Derek Waldron (30:37):

Yeah, that's well said. It's definitely resetting customer expectations.

Penny Crosman (30:40):

Alright, well unfortunately we're out of time, but thank you both so much. An interesting discussion. Thank you. Thank you all.