- Key insight: A study found that large language models endorse fintechs for small business banking far more than they recommend traditional banks.
- What's at stake: Banks could become invisible to entrepreneurs starting new businesses.
- Forward look: Companies can improve their chances of being recommended by an LLM by working on legibility.
Banks, as well as other businesses, have spent decades working on "SEO optimization," making their websites and online information easily accessible to search engines such as Google and Bing. A new study confirms that they now need to start thinking about "LLM optimization," making their websites and online information accessible to AI-powered chatbots such as ChatGPT or Gemini.
Researchers posing as small business owners recently had 2,160 conversations with ChatGPT, Gemini and Perplexity about banks and financial products. What they found was that their first question — for example, "What are the leading options for a business like mine right now?" — would surface a list of five to ten banks and fintechs. But as they asked follow-up questions and sought a product recommendation — for instance, "Based on everything we've discussed, what would you recommend I go with, and how do I open it?" — the AI models recommended fintechs Mercury, Bluevine and Wise far more than traditional banks.
"As the research clearly states, LLMs focus on products and services, not brands or banks," said John A. Thompson, a professor at the University of Michigan who teaches courses on AI and is not connected with AIVO or its study. "And the research shows that brand value and word of mouth mean nothing to an LLM. Banks and brands still have to serve customers, search engines, research organizations and now they have to add LLMs to their audience perspectives to be served."
Mercury was recommended in 383 of 1,440 ChatGPT and Gemini conversations in the study conducted by research firm AIVO and released Thursday — more than 30 traditional banks combined (299). Perplexity recommended Mercury 211 times, while it recommended a traditional bank 153 times. Across all three large language models, fintechs garnered 1,345 recommendations, compared to banks' 452. (The conversations were based on four small-business scenarios: forming a new LLC, switching providers, seeking a credit-led relationship and looking to meet international or multi-currency needs.)
In a related finding, fintechs' websites were cited more often than banks' by these large language models. For instance, among the 30,424 sources Perplexity cited across its answers, 32% were fintech companies' own websites, 20% were bank websites.
"Mercury, Bluevine and Wise are winning the 'who do I open an account with' conversation before a bank is ever considered," the report stated.
It's hard to say exactly why the models recommended fintechs more frequently. AIVO's leaders think product clarity and digital-first messaging are factors.
"My theory is that the online banks depend upon opening accounts online," said Tim de Rosen, CEO of AIVO, whereas some traditional banks still require small business owners to go to a branch and identify themselves with a driver's license to open an account.
Therefore, the fintechs have "spent a lot of time in the last few years making sure that their content is optimized in SEO terms," which seems to help with LLM optimization, de Rosen told American Banker.
Becoming 'legible'
Some of the difference between the way LLMs perceive fintechs and banks comes down to "legibility," experts say. Companies need to make themselves "legible" to models like ChatGPT, Gemini and Perplexity, "to make sure that if somebody has a conversation with an LLM, and their product, brand or bank is the suitable answer, it does get selected," de Rosen said.
A small business owner looking to open a new restaurant in a new town might be looking for a lender that can provide a specific rate and terms. Two lenders might meet all that criteria, but the LLM may only understand one of them, said Paul Sheals, co-founder of AIVO. The other might confuse the LLM with content that's not tagged or classified accurately.
"If an LLM gets confused, it won't recommend that particular brand because it hasn't got a degree of confidence that it can do what's asked," Sheals said. "Whereas if it's easy to understand, there's less content, it's been made legible, it's been made machine readable, it becomes more legible. The LLM can understand the context of what this bank does, what its offering is, based on its website, the content it produces and trusted sources."
One way to improve legibility is to use a structured data format called JavaScript Object Notation for Linked Data (JSON-LD) on a company's website. This makes the content easier for LLMs to read and understand.
Another is to make the content itself clearer and more readable for machines and people.
"Banks have an issue with the collection of historical information that they have provided and have on their sites and on the web, but that doesn't need to be an issue," Thompson said. "What they need to do is go on a project to align old and new content around products and services."
They need to focus on aspects of product offerings that people and businesses care about, he said.
When you can't control the source
A related theory about why fintechs performed better in this study is that fintechs have less "clutter," or misinformation about them flying around the internet.
Traditional banks are often recommended at the start of an LLM conversation because they have well-known brands, according to Sheals.
"But when it comes time for the LLM to make a decision on which one's best for certain criteria, there's so much out there about the big banks that it struggles to make sense of it all, and therefore goes for the option which is more provable, more legible and more understandable, which is often these new challenger banks," Sheals said.
The challenge for the big banks is to make whatever content exists about them more legible, "and get rid of any sort of negativity that should or that shouldn't be there," he said.
Thompson partly disagrees. He views having masses of information about a company available on the internet as an asset to an organization. But he agrees on the need for clarity and consistency.
"If the majority of the information that the bank or fintech puts out is consistent and clear, the LLMs will parrot back that or those messages consistently and reliably," Thompson said. "But if the messaging is contradictory or unclear, the LLMs will pick other offerings as more desirable or clearly the products and services to display."
LLMs rely heavily on public information sources such as Reddit and Wikipedia, where banks have limited control over how they show up. But they can set up
Where banks did well
Banks did well in some portions of the study. JPMorganChase was cited most often in answers to questions about small business credit. Huntington Bank and Synovus led answers to questions about SBA loans.
These banks may not have consciously done anything to perform well in these categories, de Rosen said.
Read more:
What comes next for banks in a post-CLARITY Congress Warsh: Rising bond yields a sign of strong U.S. economy Borrowing from Federal Home Loan banks jumped 20% in 2Q State bank examiners get a playbook for inspecting AI
"They may have done some really strong SEO work in terms of their content," Sheals said.
"I think this is a really interesting debate as to why certain banks were more legible and won in certain categories, and why some lost out," de Rosen said. The three banks did not respond to requests for comment.
"LLMs want to give the right answer," Sheals said. "Their whole purpose of being there is to give a more accurate, more comprehensive answer to people that use it. If they're getting it wrong, they're just not understanding the context or the information that's being given."
With legibility problems addressed, "the right banks will win," he said.
