Data: Pragmatic Uses of Artificial Intelligence for the Banking Industry

Michael Hoffman, Director, Public Sector Financial Services, Guidehouse; Praful Mainker, Chief Risk Officer, Moonstone Bank

Transcription:

Carter Pape: (00:08)
I'm Carter Pape. I'm a cybersecurity and technology reporter for American Banker, and I'm gonna let my co-panelists, Michael and Praful, introduce themselves,

Praful Mainker: (00:18)
Go for it, Michael.

Carter Pape: (00:22)
Praful.

Praful Mainker: (00:22)
Right, I'll get started. I'm Praful Mainker. I recently joined a new bank called Moonstone Bank, which is striving to serve medium and small industries in underserved sectors, such as cannabis banking, crypto exchanges, money services, business, etcetera. we are gonna be a hundred percent digital bank online, before I joined this firm as Chief Risk Officer, I was with JPMorgan Chase. I was there in compliance, area applying artificial intelligence and advanced data science techniques. And before that I was in various other financial services firms, suggest Capital One, MoneyGram Citibank to name a few. Thank you.

Carter Pape: (01:02)
And Michael,

Michael Hoffman: (01:05)
Is that the slide? We're good. Yeah, hi, I'm Michael Hoffman. I'm a director at Guidehouse. I've been with, Guidehouse for about a year, I wrote a book, customer worthy, which if any of you are, looking at AI and, from a customer management customer, experience management CRM really was, based on customer profitability and that little framework there, has been used pretty extensively to design a lot of systems. It's about 11 years old, there's some new pieces coming out on that too, to make it more user friendly. And then I'm working on another book customer sensor. I worked initially in AI with Experian when Experian became Experian. I had about directly sold and worked with about 500 banks nationally and internationally. And then also a number of the, like leading data companies, consumers like publishers, clearinghouse, readers, digest, E-Trade, fidelity where they had, where you worked with like a group of, anywhere from 30 to about a hundred PhD statisticians. So, and then other places where you had half a statistician who kind of work part-time. So it's probably a good place to like, to like level set, that this is a pretty broad spectrum that we're speaking to used to be very expensive to do AI and modeling, now it's, wildly inexpensive, which isn't necessarily a good thing. We'll talk about that. Yeah.

Carter Pape: (02:28)
So I personally love AI, Something that I studied in university and was really like a, just a super exciting thing to get to work with. So what I wanted open with was just talking about applications, just kind of briefly just to, set the stage for this applications of AI, outside of personal finance that we people might see in their everyday lives. So what are some good examples that you can share of

Praful Mainker: (02:58)
What looks like? Yeah, thank you for that question, as Andrew said from, you know, previously from Stanford and now he has his own firm, AI is the new electricity, so everybody should be able to use it. Everybody should be able to understand not everybody has to be a programmer necessarily, I feel that we are already surrounded by it, you know, Alexa, Siri, right? The autonomous car, in fact, the first ever and most successful application in my opinion, has been around for decades, the autopilot. So the AI in it was rather elementary from today's standards. I understand, but it was there, it was making decisions and, letting the planes fly with almost no man intervention, so those are a few I would like to name, which, like I said, we are surrounded by AI applications now.

Michael Hoffman: (03:45)
Yeah. I think the, kind of back to the book, but I, my, big interest in AI was being able to predict who was gonna become a customer and also who was going to then buy a second product, which is kind of the foundation of most business. One interesting thing to pay attention to, that a lot of people don't consider is measuring things that didn't happen. And that's actually, it's probably the may have the largest lift in your organization, so if a customer looks like they're going to buy the next product and they don't, you should be able to ask why, why didn't people complete the transaction? If you can examine, how far they went through that transaction, especially, across multiple channels. Because the other thing is somebody who, I mean, very few people go into a branch or, go face to face anymore.

Michael Hoffman: (04:34)
But if they're on the phone and people are waiting, then they'll also go online. And that multi-channel piece, if you can add up all those different contacts as they're going on, whereas they happen historically, you can actually see that the other side of the equation, which I also work for for side cell. So you talk about answer answering calls and, helping people online for like TiVo, and AT&T and Verizon, et cetera, identifying where a customer wants to go and then respecting the amount of time that they've invested is a huge, huge issue. So if somebody's been on hold for, for 11 minutes, they've probably gone and done other things. If they've been online for 45 minutes, they probably try to solve the problem themselves, and if you can respect that, and if you look at like call scripting software and chat software, you see how they try to model that.

Speaker 5: (05:21)
If you haven't worked on that, it's a good place to look, but being able to get in sync with where the customers at that time, at that point of contact and whoever they're contacting, whether it be a bot on a, online self-service, or if it's a service, representative, however, you can get that person, the, your representative and the customer in sync as quickly as possible. And then the idea is to be able to curate that interaction, right? So that's the other piece of AI is how do I put together all the content and all the knowledge so that I can make a rewarding experience for that person. So they have a positive outcome. Now that positive outcome kind of going back upstream should be tied back to your corporate objective, which means I also should know my customer's lifetime value, right?

Michael Hoffman: (06:07)
So I'm investing in customers, that I can afford to, you in financial services, it's pretty dispare. I always used to point out that when you do lifetime value, in retail, financial or small business, you'll identify customers that it's worth driving out to their location to answer their problem. Okay. That investment to retain that customer and how outstanding that looks. You can totally justify that, if you do that to the wrong customers, you'll be out of business very quickly, cuz it's not sustainable. So those types of factors shouldn't go away that, so lifetime value and the idea of being able to monetize each interaction and you should really monetize each formula or each AI piece that you're looking at and then visualize with that outcome is gonna be so that you're in sync with all the right up, people and, channels that you have and then be able to go and prioritize, what you're gonna do and where, and then optimize and optimizes the next generation of AI. So that it becomes a repeatable process.

Carter Pape: (07:05)
Something that I think is interesting to think about with, AI, as I was looking a little bit into the history of it, just kind of surprised how long it's actually been being used, just quick tidbit between 93 and 95, 1993 and 1995, the financial crimes enforcement network used an AI system that it built to flag, $1 billion, potentially laundered funds, which I found interesting, cuz pitches I get all the time right now are about using AI to identify fraud, turns out it's been doing that for a long time. So I'm interested in thinking about other applications that have that longevity, the way that applying AI to fraud has longevity, cuz I think it's useful to kind of, you know, consider where AI has been useful for a while versus, where it's kind of, you're getting newer applications. So what are some of those older applications, more proven applications,

Praful Mainker: (08:01)
Right? I mean you're right in pointing out, financial crimes, right? So I am in risk in compliance, through some, series of good fortune and generosity of others. My background is electrical engineering. My original research was in artificial intelligence. Those were the days when you had a small set of data and you had a algorithm and you submitted that data overnight to a Unix machine and came back next day, perhaps next week, if the machine wasn't that great, but things have changed since then. And to answer your question Carter, there are two aspects of financial crimes, two major aspects. One is money laundering and the other is fraud, right? The challenge with money laundering is we rarely ever get confirmation that it was truly money laundering. We suspect it was. And we may file a report with the government, but actual money laundering convictions are few and far between they happen after many years and when they happen, they're settled out of court.

Praful Mainker: (08:56)
So it's not in public domain. So it's rare unless there is a celebrity involved or DA decides to publish it. We rarely find out who was convicted. All we find out is that looked unusual. And that's the reason I said, there are two patterns, the same unusual spotting works for fraud as well. In fact, it works much better. So some of the things I learned while I was in MoneyGram, where financial crimes was, the main concern to the money services business was the fact that you start, with the transaction where the customer called in and say, Hey, I was a victim of fraud. That's the advantage of fraud in fraud. People call in to say, I was victimized. There is a fraud. I did not have this charge. You start from there.

Praful Mainker: (09:39)
And you start seeing pattern as to who are they sending? And as a micro document, where is it going? Where is coming from et cetera. And if you go to six degrees, of separation, you can actually see the fraud ring itself, as long as you have a big data set and the data is accurate, right? And you know, we are talking right after the data governance session, literally data is everything. The actual incremental gain by using an artificial intelligence and training algorithm. Stuff is not that big. The biggest gain that I have gotten in my life is just by having data cleaned up data, well, wrangle data, well presented and being accessible. And then there's an incremental gain by using the pattern recognition where we are not able to spot it because we can only think from cause and effect.

Praful Mainker: (10:22)
Humans can only understand cause and effect the machine does not constrained by it. How do we unconstrain the machine it's by not telling it what to train it on by going unsupervised learning. So those are some of the things that, that work best. But, coming back to your other question about, what else has existed? Something that I had to, learn very fast was during the COVID era, which has been around now COVID is not over yet, but you know, we can say that's it's in the past, from this perspective, when the cares outcome came, which was the law that was that was passed by the government, right after COVID pandemic came into being that, that meant that compliance and risk teams had barely a couple of weeks to put together their analytics to starting patterns as to what customers were saying.

Praful Mainker: (11:10)
Imagine that is the new law. We don't even know what customers might be saying. There is no data historically, so there is nothing to train on, but I'm personally believe that that is the best example of AI. So I used, data, analytics, visual analytics, structured data, connected to unstructured data and on unstructured data, I actually let the data itself organize itself. So I didn't actually constrain it by any known pattern. So those are some of the examples, you know, those have existed. So none of this is.

Speaker 5: (11:40)
Brand new, but those are some of the specific examples.

Michael Hoffman: (11:43)
Yeah. Now I would add to that. So, kind of the old way of working on fraud was, was more of a reaction right. Somebody would identify something or, you'd see something that was suspicious in data, love granular data, you know, that's where you can see everything is, can start to see patterns, when you design systems though, you're, especially with automation going as quickly as it is. And that's part of what happened in the PPP program was, treasury assumed that, when they moved that into production very quickly, one that they thought they were bringing all the pieces over. I won't go to that in too much detail, they didn't pull over the check backs to the data to make sure that those were actually businesses actually had all the capability for whatever reason don't know.

Michael Hoffman: (12:30)
but anyway, there'll be a movie on that eventually, but this idea of patterns, you should think of algo versus algo. So an algorithm, somebody who's that good at math, who's going to spoof his system either is a digital twin. And there's been a lot of, you can see payment systems where somebody has gone and look like they were a company processing transactions, or they in, they were living inside somebody else's system. They were able to capture all those patterns and everything looks like it makes sense. It's actually like an over modeled kind of result until it isn't and all that data disappears, now there are literally children that can imitate,, an entire country, an entire business just by going and tracking or monitoring some of those transactions. So I think it's a responsibility from an AI standpoint to build that in, to identify, not, not over model or let something be gamed really is a responsibility of whoever's designing things, to make sure that you don't over model things.

Michael Hoffman: (13:33)
Once in a while, I did a presentation recently where I said, you're gonna have to like do an nth, transaction, like every one out of every hundred transactions that requires a manual step to make sure that you're not auto, you're not completely being spoofed. Otherwise, is much, much too easy for somebody to game your system when you go very heavy digital. And there's a lot of that, you know, there's pretty much every system is vulnerable to that, to anybody that can listen. So you, I think that's part of the responsibility. One of the newer things is you don't have that latency of data. You can identify an anomaly very, very quickly, and those anomalies may be internal, there's a number of examples where somebody set up an internal incentive program, I had, a client a bunch of years ago, where for the first time we actually dropped all of the GRA daily transactions, business and, consumer put it in the system.

Michael Hoffman: (14:30)
You could do sub second, ad hoc queries and identified some patterns of some branches and, identified like $20 million going across like individual DDA accounts. Nobody has 20 million every other week going into their checking account. That's kind of a big red flag, but what it turned out was they had an incentive program. So the branch managers were moving around. So they would be able to gain and actually win that program and they would be rewarded for it. So when I'm talking about looking at patterns and where there could be fraud, everything is really up for a game you're responsible when you design those systems to make sure that you're not being gamed.

Carter Pape: (15:08)
Something about, artificial intelligence, I think is interesting is, there, there's so many different ways of doing it. So the two primary ways that I can think of are supervised and unsupervised learning, and they kind of have different functions, you know, in applications as well. And what I wanna ask about next was basically, you know, assessing those applications, you know, how do you know if you're a bank and you're looking at a problem and you might be being told by people on your team that the answers are official intelligence, how do you assess that's the answer, or if it's maybe like just a red herring that it's only gonna be a marginal gain or, you know, maybe not a gain at all, as you're alluding to earlier

Praful Mainker: (15:48)
That's right. Artificial intelligence is not the answer in many cases. In fact, that would be the last resort, in my opinion, so the gains I have made, I was fortunate to make, in other words, in my previous roles, majority of those came from data cleansing data, governs the people who were here before us, they were talking about it in other so let's take mortgage, right? So there was a specific thing I was doing related to mortgage underwriting decisions and trying to see if we were doing that consistently across, you know, various factors, if you look at the procedures, we talk to business and say, absolutely we are right, because people are looking at the data in a certain way. They have all the causal thinking and they're not able to look at 200 or 500,000 loans in one picture that's just not possible.

Praful Mainker: (16:35)
So, the most of the gains I made was simply by looking at the data to say, Hey, there are, fields which are form filled, so they're manual. So the people were easily able to omit it. So let's say if you are a home lending advisor and you know that there is something about title vendor, which people are watching because there is a crime involved in it. What will you do? You will not put the title, vendor name simple. And if it's if it's not automatically required, that's it, that's all it's required. So I could show my counterparts how the behaviors changed. As I started putting together analytics, the word got to business and they started changing their behavior. So in other words, the biggest gain I made was cleaning up data organizing it better, understanding it, better, automating it.

Praful Mainker: (17:20)
And then the final I think of an AI as a higher form of automation, if we are not able to use judgment rules, based logic based judgment, even if it's cumbersome, even if it's erroneous, but if we are not able to do a particular action in seven out of 10 times accurately, then there isn't an algorithm which will give you a big post. If it's already possible set of seven out of 10, takes a long time, and sometimes is error prone. Most of the algorithms will take you to 9.9, 9.7 out of 10. That's basically the boost. So last that 10 to 20, percent is the boost I have seen. Now. I'm not saying that's the only way to do it, but before you apply AI, all of that stuff to happen to answer your question, Carter, it varies case by case, if the input data is poor quality, there, isn't an algorithm that I have found that can fix that problem.

Carter Pape: (18:17)
Anything

Michael Hoffman: (18:17)
Add? Yeah. Supervised unsupervised. I think, you know, the big benefit you have right now with cloud, and how it works with AI and with your analytics group is you, you can spin up an environment, just for the purpose of doing some analytics queries and do massive simulation at a granular level, to see what these outcomes are gonna be. If anybody doesn't know, look even in Excel right now, if you have, eight periods of data, you could actually just throw it out. It'll actually go forward, propagate, you know, very kind of standard ways to do that. And if Excel is that smart at kind of doing analytics, and then we should be too, but the ability to go and take all that data, put it into environment. I've been encouraging clients to do really massive simulation when you're creating a new product or doing, doing some scenarios to look at where growth is.

Michael Hoffman: (19:07)
Now's a perfect time in the market, by the way, right? With the market one you had, COVID totally changed the game really, from a historical piece going and looking at how performance, if you go back five years, data doesn't make any sense, right? The whole world changed. Every, element changed, really all your background numbers. Now with the market, what's happened in the past, pretty much past three weeks, again, another, change of the deck. So historical data doesn't have as much value. What I wanna be able to do is start to estimate what the outcomes are gonna be. How are people changing? I can see where the forces are, but how our different groups and sectors and clusters are reacting going forward. And the real big piece of AI right now is, how good am I at predicting what's going to happen?

Michael Hoffman: (19:50)
So you have a best case, worst case, moderate case scenario that I wanna predict on everything. And if I lay that out, as part of my core AI structure, I wanna then continue to get better and better at estimating that should be like a foundational piece now that everyone has, that I think, is more groundbreaking than just about anything else is being able to predict if what's gonna happen and get better and better at predicting historical data, you know, questionable how, valuable it's always been. It is the basis of most of the things we do, but everything is changing at such a quick clip, and there's people that take advantage of that again, from a scam standpoint, when everything's going hot and heavy, they know, you can kind of work your way in there, and get away with, fraud. But right now, I think, the ability to go to on a granular level look at, I think you mentioned like the network of actors on things as well, that's what everybody should be focusing on. Yeah.

Carter Pape: (20:44)
I want to go to audience questions in just a second, but I, one last thing I wanted to run by you guys. So, you you've talked about sort of the, the importance of having clean data and just the, how important that is to, you know, creating models, artificial intelligence models that are actually useful and give good results, so the thing I wanted to ask was just about, you know, it seems like a maturity thing, you wanna be able to mature in your having very, clean data before you can actually apply artificial intelligence in a sort of useful way. What are the sort of other prerequisite things that you need to make sure you're doing? Well, maybe, you know, more fundamental things that will help you have a, you know, a good application of artificial intelligence.

Praful Mainker: (21:29)
Thanks for that question. Couple of items, I think it needs to be able to, be explained. So those were consuming, consuming it. They cannot be just surprised, or they cannot just, lose faith in it. So one of the challenges I faced was when customer complaints were coming in after cares act, and the law itself was knew the business was changing its process almost on a weekly basis. So the, the visualization I built out of the NLP, I, that I applied to, natural language to complaints using natural language processing, unsupervised learning was in a way that once you picked a pattern right away, it could actually tell you on what basis is that pattern coming together. So in there was marginal stuff, the simple stuff like lean release and all that stuff was just right, right away, apparent people were saying, Hey, I'm in foreclosure by doing, you know, late fees.

Praful Mainker: (22:18)
Majority of them were late fees by the late charges, late fees and stuff like that. And, so one aspect is that, you know, explainability, the other aspect is the usage, right? So if you remember the mortgage crisis, everybody said, moodys ratings were wrong. We weren't, our models were all right, moody was wrong. If you listened to moodys, they said, well, you were using the model wrong. We told you never to make investment decisions based on our ratings. They were just there to compare various securities. So, the model, one of the important aspect of model risk management is to understand who is using your model. Why are they using it and informing them saying, listen, it cannot be used in isolation is not NBO right. There is no model is perfect. They say, no model is perfect. Some are useful, right? All models are wrong. Some are useful. That's the way, it say. So that, that would be my answer. Thank you.

Michael Hoffman: (23:08)
Yeah. I guess, my question on clean data is, I've seen probably 90% of, AI and, data warehouse projects. All the money is gone while you're trying to clean your data and you don't get to answer questions, being like a forensic database guy, you, and also when you're doing models, you're usually doing samples, right? So I don't have to clean all the data to start to get intelligence, actually that process of going through and using the data will help me identify which fields need to be cleaned and where to invest to make those clean. Right. It may be inherent in the data, especially if we're doing ecosystem design, you may never have clean data, especially with new products and services, so, I would take that as a given, the big tip, I think too, is to look at, like I said before, some organizations have, tens of PhD statisticians at their disposal, either hired or in-house, some have one, look at the, I think the most important thing is what are the questions you're asking, not the tools you're using and that, I think everybody kind of misses that, but spend your time on, if you had the answer to a question, how would you act on it?

Michael Hoffman: (24:14)
Those are really the refining statements. Then look for the tool to get you there as quickly as possible and what data will get you there. It's a much, you'll have a much higher level of success than going any other way around. And again, work with, lots and lots and lots of, tools. But, the question is actually more important, more valuable than the tool. And then again, who the user is kind of your

Carter Pape: (24:35)
Point. Yeah. That's a good nuance to add. All right, I think we have time for one audience question. Possibly two, we have a mic available. If anyone has one, if not, I have more questions than I could ask. Okay. This guy right here, one

Praful Mainker: (24:48)
Right data.

Audience Member 1: (24:49)
My voice is pretty loud.

Carter Pape: (24:51)
We need it on the recording.

Audience Member 1: (24:53)
Recording better. Oh, hello, so talking about data railing, we're hearing a lot about knowledge graphs right now. And so the power of a knowledge graph to manage all these different business facts or different kind of elements of facts. Do you see, do you see kind of the rise of knowledge graphs kind of powering a lot of these models and curious how you would break that down?

Michael Hoffman: (25:17)
Yeah, I was gonna say so I would add that to the, tools. Yeah. I think it's inexpensive enough if when you have data to go and experiment with knowledge graphs and see how you're gonna use it and who are the users are gonna be. And I would make that additive to the solution act. I don't think that it's not as cost prohibitive as it used to be. So I would do it again. It depends on the question you're trying to ask, and then if you're going to automate it, that's kind of the other question was how do those answers actually fit into who's ever acting on those answers? I would look at that piece and look at knowledge graphs together, if that makes sense

Praful Mainker: (25:50)
And explain to me by knowledge graph, what do you specifically mean? Sorry.

Audience Member 1: (25:55)
Yeah, so I'm seeing it in a lot of language models with an NLP, NLU, so they're trying to kind of break down consumer questions, right? So just like looking at some simple questions as like the concept of, Hey, I wanna find a branch near me or like, what is a branch and financial professionals? So we need to, we kind of need to understand the first party, like, entities and then build language models on top of that. And then that kind of opens up these marketing personalization use cases is where I'm kind of seeing a lot of that go. So,

Praful Mainker: (26:27)
So, I have personally not worked in that field. My work has primarily been a risk and compliance, so I can't first, hand answer it, but it's very similar, to the application I had with complaints where you don't know what the customer will ask, but you want to be able to bunch it together in a way that, you direct them more or less in the simplest fashion. So they don't bounce around and go in wrong direction. So, that's my understanding of it.

Audience Member 1: (26:52)
And do you see yourself in like your tokenizing data, probably with a lot of year, so you don't really know what it's about, but you're kind of seeing those signals come out.

Praful Mainker: (27:02)
Yeah. Those, those methods are pretty standard now, of everything I've used has always been off the shelf. So it's sustainable. So there is no, you know, extraordinary algorithms, no unproven stuff, because the fact that it has regulatory impact and, it has regulatory scrutiny, I feel like in particular risk and compliance, when you're using a function, in fact, there is a lot of scrutiny to make sure, they're all standard, and they're sustainable. So they should be able to deploy them on server, run them automatically, et cetera, that pretty much rules out any, you know, back alley, coding and that kind of stuff. Yeah.

Carter Pape: (27:37)
So we gotta close now. So last thing I want to ask both of you is just to emphasize underline, highlight anything that we've talked about here that you think that, you wanna make sure people walk away with, or if we didn't get something that you wanted to, you can go ahead and mention that now.

Praful Mainker: (27:53)
Yeah, sure. I mean, the only message I'll have is the more you understand, and I echo what, Michael was saying. So the problem is exactly, you know, Is there a problem? Is there a capability that's missing or exists, but it's inefficient, it's error prone. That is the thing you need to ask, when I walked into my, one of my previous roles, the question that the challenge they gave me was actually not the right problem at all. Once I started fixing it. So they said to me, Hey, there's this, you know, complaints come with some categorization based on rules and some form of, text mining, fix it, use AI and fix it. I said, that seems pretty easy, as I started doing it, you know, of course it turned out it was not easy. And, even if I did fix it, they said, well, this is of no value. It was not actionable. So finally, what matters is you need to bring it by asking questions to the user as to why they're using the model or whatever you call it, visualization and it needs to come to the stage where a person simply looks at it and say, I need to make a phone call, or I need to actually download these and I need to go, do something with it. So it needs to be actionable. That's my message.

Michael Hoffman: (29:04)
Yeah. I was thinking one, one point is always, you know, automate what works. So you can spend your time innovating is probably this, the overall kind of design principle, I would use, the other thing that, just came up again recently is, kind of a proven best practice have like a decision room or a, I would call it a war room, but I know that's not politically correct, cuz also, you there's a lot of other connotations, but a place where the people that are making decisions go to that's isolated from everything else that has whiteboards, where you can have an expert kind of user, kind of walk through data, and have everybody kind of share the information in their interpretation because very few people understand analytics. Very few people understand how you got there.

Michael Hoffman: (29:50)
I've seen the, the greatest outcomes, just phenomenal types of work, bringing a CFO I've people get scared, bring a CFO and the CEO, the head of marketing and head of operations or branches into a room, and walk through, the results with a data person who understands everything there, and you can get real answers and move ahead very quickly, versus the, the latency that happens. When you ask a question, you get an answer back. Exactly what you're just saying. That's not really, oh, now that you answered that question, I have 50 more cuz every question has that, right. Get to the point of actionability. Explainability is kind of the other response, but the actionability, what are you gonna do if you have that answer, go through that. I have used design thinking by the way, to help kind of design and try to take everything down to mobile design, some things recently for a 20, 28 rollout that you know, what alerts your, watch, what alerts your phone, all the kind of profile interactions for alerts for anomalies and detection and how do people wanna be made aware of those types of things.

Michael Hoffman: (30:51)
So that include the people, but include the experts all together in one place.

Carter Pape: (30:57)
Michael Hoffman, Praful Mainker. Thank you so much.