Every year, a group of people in the innovation team at U.S. Bank go to the Consumer Electronics Show in Vegas, walk the floor and test products that might be useful to the bank someday.
"We kind of treat it as a future safari," said Todder Moning, head of applied foresights.
The group, led by Chief Innovation Officer Don Relyea, arrived with some megatrends in mind in the areas of embedded financial services, virtual economies, artificial intelligence and automation. But they were also looking for "unexpected or more guts technology, like capabilities that are going to start seeping into devices and machines and tech," Moning said.
"So it's really like being at the carnival," he said. "When we go to CES, it's a longitudinal look at these various technologies or futures or spaces and seeing how each year they progress. When we first started coming, the robots looked like science experiments; you saw the wires and stuff. Now they're really slick."
Moning and Relyea spoke with American Banker while they were still jet lagged from their trip back from Las Vegas. They shared some of their impressions.
What were some of the things that caught your eye or inspired you?
Moning: We saw an autonomous snowblower. I was a day late to the show because we had a 20-inch snowstorm in Minnesota, and I had been shoveling and snow-blowing for three days straight. And I cannot wait to someday just have that robot. In the long term, I might not need to own that. It might be something that's a business on its own. It just goes around and does people's driveways in the neighborhood and then charges their home hub.
Autonomous cars haven't arrived as quickly as everyone thought. And so a lot of these brands are starting to hire military folks or other folks to train them to remotely help the autonomous vehicles of all sorts when they get into trouble. And we saw an example of that in that Caterpillar had one of these giant mining trucks, but then they had these two seats in their exhibit, and you could get in them. And one was literally controlling a giant backhoe in Illinois. And another was controlling some other big piece of equipment in Tucson.
Is that something you could relate to banking in some way, or is it just interesting?
Relyea: To tie it back to banking, one of the big reasons why we do this is to catch the trends early enough so that we can get involved in the R&D work necessary to make something real. A good example of that is the story around our virtual assistant. It took us 10 years to make that virtual assistant, and we started prototyping based off of trends that we were seeing at CES. But the technology wasn't really ready until it was ready. So a lot of what we're looking for is to really understand where these technologies are and when they're going to be ready, and when we need to be ready with APIs and ways to connect into an evolving point of sale.
We see our normal channels eroding, and you're going to see points of sale happen all over the place, including in smart devices. One trend that we're catching right now is this convergence of AI and machine learning that is doing analysis of people or their behaviors, and sensors and personal devices. All of those things are converging to create highly personalized experiences. The real example of that is an AI engine that works off your phone or off of a smart-mirror-type device, and it will analyze your face and your skin and make customized skincare recommendations, which of course you can buy right there, from your mirror. There are several like that, everything from health and beauty to Incheon Airport, which showed facial scanning technology and personalized navigation through a robot only the user sees on their phone, that will take the person where they need to go in the airport.
Do you see using these sensors and devices to gather information about what customers are doing, or to insert some kind of loan offer or payment instrument?
Relyea: It's the payment part we really want to be involved in because where payments are happening could be in your car, it could be in your bathroom, and I think it's moving a little bit away from your phone. It's not just the digital experience on your phone. It could be experiential or environmental because the environments are becoming aware, and there's lots of technology that's understanding what humans are doing in a space that could create opportunities as well. Understanding where those payment experiences are going to happen is going to guide how we build and offer our APIs to plug into that space.
Moning: Over time, instead of staring into a couple of black screens, whether it's your laptop, your phone, your tablet, you have this entire constellation of devices, some in the background like internet of things and your smart homes, and some that you're interfacing with very directly. Over time, as these things get smarter, they might become businesses on their own behalf. We have financial services that have either been do it yourself or they've been do it together. And we're moving continually through automation and into autonomy into this world of, do it for me. And so it's not like folks are going to have to come back to the bank to get things they need and then go back into that experience. All of those apps, experiences, devices, they're going to need to have the financial part built in. And so it's almost like financial services are starting to not fade, but sort of distribute themselves into those devices via APIs.
For a while people were talking about smart refrigerators and smart washing machines that were going to order your food and detergent when you need it. But I've never seen those things take off. What do you think it would take for some of these internet of things use cases to become popular?
Moning: Part of it's a matter of time. The internet of things focuses on hardware right now, whether it's a car, refrigerator or appliance. But the model of sensors sending data using AI and crunching and finding new models, and then controlling those devices from a digital twin in the cloud, I think it's going to take a number of years before the Amazon kind of auto replenishment system will become a thing.
Relyea: When I was at Verizon, we had all kinds of IOT demos, but the bane of our existence was the lack of interoperability and standards between all of the different sensors and devices and how they connect and communicate. Until that actually happens, it's always going to struggle. But I think they're getting closer to standards now.
Moning: Yes, the consumer experience has been a nightmare. It's hard to have a successful internet of things if the things all can't connect smoothly. People had all these things in their homes from different manufacturers that didn't work together. But finally they got together and they've created this standard called Matter where once you plug it in, it finds your network and it doesn't matter what device, what network or brand it came from, it just works.
There's a technology underneath Matter called Thread that makes different brands, different devices work together to network in a more seamless way. Thread is a standard that lets those devices talk to each other and not need the internet. So the next iteration will be, you won't need to ping up to space and back again. They'll just talk to each other, even if the Internet's offline.
What was the coolest thing you saw?
Relyea: There was an off-grid building that's entirely powered by micro powered chips that derive their energy from liquids and moisture. They had a battery and a chip connected to a cup of dirt. And the cup of dirt over time will slowly fill the entire battery array. So if you have enough of these chips filling battery arrays, you can power your entire house. They're definitely using low-power LED light for everything. They had a demo where they had the chip connected to a glass of wine, and the wine was generating a ton of power versus a cup of water or a loaf of bread. But you could plug it into anything.
Most of the stuff we saw was kind of incremental improvements on existing technology, like augmented reality lenses getting really small in form factor and low power. When you look at the Samsung and LG glasses, they're not a whole lot different than your standard glasses. The little stems that go over your ear are a little thicker because they have to connect to a battery, but they're getting there. So, I think that mixed reality, augmented reality, at least from a prototype, customer experience level is getting pretty close to where people might begin to wear them.
Moning: We've hypothesized this coming, instead of having VR or AR be these giant goggles. In the future, you'll just have a pair of glasses and we'll look back someday and say, oh, remember we used to call it VR and AR and wasn't that dumb?
Do you see any practical use of virtual and augmented reality in banking? For a while people were talking about if you look at a house you're interested in buying, the bank could show you information about the house and how much credit you qualify for, overlaid on top of the picture.
Moning: We've done a lot of things over the last 10 years in that space. Everything from back when there was Google Glass being able to find our branches and ATMs, or where can I get deals from our customers or retailers. We find a lot of practical possibilities for banking. We're going to have all these new channels and new ways to reach people.
Relyea: Everybody's doing their own thing in that space right now. You've got Meta doing their own thing, Google doing their own thing, Apple doing their own thing, and there's hardware threads, as well as software threads in how the experience is going to be controlled. At this point, there are no clear leaders or winners in that space. So the thing we're trying to figure out is, is the metaverse going to be a centralized thing controlled by a big tech company like Google or Amazon or Apple or whatnot, or is a decentralized, community governed metaverse going to emerge? You see folks like Neal Stephenson, who was author of "Snow Crash" and coined the term metaverse, now exploring a decentralized version that's supposed to be open, that would belong to everyone. He's trying to put some white papers out by the end of this year on how that could be constructed as a layer one blockchain platform. I think we're going to continue coming up with prototypes in our R&D lab. But until we see who the winners are and how the standards come together, we have to wait to act.
Moning: You asked about the most "wow" thing we saw. One that was amazing but futuristic was BMW's i Vision Dee car that can change colors. Siemens had a VR experience where they were showing under-ocean farming. It was a giant air pocket under the ocean that they could grow plants in. The fans and the lights were run by ocean waves. That was a huge sustainability thing for me. And sustainability was all over the show.
Relyea: There was a solar tree, too — Panasonic had a fake tree, but all the leaves were solar panels. It generates power for commercial retail spaces.
Did you see anything that you would be ready to buy and use yourself? Either in your personal life or at work?
Moning: Oh, yes. 3M has done some really practical stuff, because Post-Its are the lifeblood of innovation designers. They have a new thing available for Mural, a remote whiteboard that we all use to do our work, and for Microsoft Teams. We could easily see using that right away. And again, for me, it was the autonomous mower and snowblower.
Relyea: I like the Jackery solar battery and solar panel. I'm into overlanding and fishing. So for me, that's a must if you're going off grid. Aira had a device that analyzes the blood flow in your face and it can read your heart rate, your blood pressure. Your stress level's right there.
We saw a bunch of AI companies, like Humanizing Autonomy. They're all about understanding the behavior of people and they can understand whether you're engaged, whether you're happy or sad. In a retail environment, you can tell what products people are reacting to as they're walking through a store, for example. Then they're also using that tech in gaming, to understand people's reactions.
Moning: Sony said they were going to come out with an electric vehicle with Honda. That's because when cars go autonomous, it's now an entertainment venue. Now it's a room on wheels. That's why when you hear brands like Apple and others say they're going to do an electric car, it sounds weird, but they're looking at the future.
Immediately my mind goes to that autonomous car in Tempe, Arizona, where the human observer was watching TV while the car
Moning: That's why you're seeing autonomy really only take off in those environments like mining and farming. John Deere came out with that autonomous tractor last year because in a farm field you're not going to typically have those problems.
Is there anything you haven't mentioned that you were impressed by or that you thought was weird or dumb?
Moning: I'm always amazed at the Indy race they have here, all autonomous. Last year was the first year they did it and it was a full size car racing on track. Last year they raced one at a time against time. This year it was head to head.
There was an electric cargo bike that can carry 595 pounds of cargo.
Relyea: One of my favorite weird ones, which is actually useful, is GutNote. It's an AI that listens to your bowel sounds and then makes customized diet recommendations based on what it hears. So apparently you can tell if you need to cut gluten or dairy out of your diet just by listening to your belly.
How does it listen to your belly?
Relyea: Through the microphone on your phone. You stick it one inch to the right of your belly button and it listens for a minute. And then it asks a bunch of questions about your bowel movements and some other super intimate questions.
Moning: There were a lot of light-based wearables that can reduce wrinkles and other stuff. Some you'd wear on muscles or knees for physical therapy or just to recover after a workout. There were some that help with depression. It's kind of a metaphor for what we were talking about earlier for banking, the way that healthcare technology is starting to distribute out to the consumer.
These devices sound like things a bank couldn't connect with because it's too personal and private.
Moning: Yes and no. Because some of them help you get better sleep, understand whether you're sleeping well. There was one company that said it didn't matter what device that you had, if it had a microphone, it could listen and diagnose your sleep and help you improve your sleep. A well-rested employee is a productive and happy employee. The same thing with customers. It all depends. You always have to look at the things that could go wrong. The black mirror stuff, the nefarious stuff. Then there's the positive part of, you wouldn't immediately think it impacts banking, but it absolutely does when you have 70,000 employees. The same thing with drones. You might think, what do drones have to do with banking? There are a lot of transactions that are happening with drones and people, but maybe someday between drones and drones. But those drones need a place to go. And we have a ton of bank branches. You get a place to land and recharge.
Relyea: Several drone nests [docking stations] were demoed at CES. We could put nests on top of every branch and when you get to a world where Amazon is actually leveraging drones to deliver things, that's a revenue opportunity if you've got a rooftop. We have thousands of rooftops that are within 10 miles of millions of people's homes.
