- Key insight: Global bank regulators listened to a series of warnings about the havoc AI can wreak on financial markets and the banking system.
- Expert quote: " If the value creation and value capture of the new AI technology are concentrated in one or a few firms, these firms constitute choke points: their terms, prices and outages propagate everywhere at once. They also have the power to extract undue rents and exercise excessive bargaining power on the rest of the economy and society at large." —Markus Brunnermeier, professor, Princeton University
- Forward look: Brunnermeier would like bank regulators to be able to evaluate major new AI models before they're released to the public.
"Carpenters build tables. Bankers build trust; so do central bankers," said Markus Brunnermeier, a Princeton University professor, at the Federal Reserve Bank of Kansas City's Economic Policy Symposium this week. "AI has the potential to disrupt institutions and trust in a fundamental and qualitatively new way."
Brunnermeier laid out several scenarios in which AI could fundamentally disrupt banking and financial markets to an audience of about 120 global central bankers, economists and academics at the conference in Jackson Hole, Wyoming. This year's attendees included Federal Reserve Chairman Kevin Warsh, Bank of England Governor Andrew Bailey and Bank of Canada Governor Tiff Macklem.
"I hope these concerns and recommendations will be taken seriously by the Exchanges and Regulators!" Ken Tower, CEO of research firm Quantitative Analysis Service, wrote on LinkedIn.
Colluding AI agents
When AI trading agents successfully coordinate with each other, they can conduct undetectable pump-and-dump schemes, Brunnermeier warned.
"In illiquid markets even small orders move prices, which makes the pump cheap, and if many AI traders tacitly coordinate, no individual trader is large enough to be identified as the manipulator or to lean against the scheme," he said. "Manipulation can hence emerge without communication or explicit intent."
This concern resonates with Sultan Meghji, senior technical expert at the Center for Strategic and International Studies and former Chief Innovation Officer at the FDIC.
"If you have a number of models that are seemingly independent, but built or trained in similar ways, you could get manipulation 'without communication or explicit intent,'" Meghji told American Banker. "This undoes the standard operating model about how the law enforcement and regulatory communities operate. There's nothing to subpoena, no data to track and no one to charge with a crime."
Central bankers outsmarted
Central banks and regulators may be outmaneuvered by AI-equipped market participants who understand regulatory reaction functions better than the regulators understand the market, Brunnermeier argued.
"AI agents understand the central bank well, while the central bank understands the AI-driven market less," he said. "The asymmetry arises because market participants' AI tools make the central bank even more legible — its speeches, minutes, and entire history are training data — while the AI ecology is not."
An AI-blinded central bank becomes vulnerable to trickery.
"Gaming by market participants becomes much more sophisticated and, importantly, it hides in non-explainable reaction functions and cannot be detected, even ex post," Brunnermeier said. "Market participants can more easily collude and hence more credibly threaten financial havoc."
Kill switches thwarted by AI
The financial markets have historically had "circuit breakers" that can bring trading to a halt, Brunnermeier noted. In the past, these have been used to contain flash crashes.
"They buy time for humans to restore common understanding and allow traders suffering from inertia to enter and act as shock absorbers," Brunnermeier said. But in a world where most trading is done by AI agents, a trading kill switch could "trigger more chaos," he said.
"An abrupt shut-off severs hedges mid-stream, cascades margin calls and evaporates liquidity, since algorithmic market makers supply the bulk of it," he said. "It might even be impossible once society depends on AI trading; the kill switch is then an illusion. Just as cash could not absorb a shutdown of electronic payments, human traders could not replace algorithmic liquidity."
Concentration of power
Brunnermeier also spoke of the concentration risk caused by many banks and market participants using foundational AI models controlled by a few companies.
"If the technology can be supplied by many competitive firms, downstream users, including the financial sector, can switch and adjust: when one provider stumbles, its clients migrate," he said. "This enhances the resilience of the financial sector and the real economy. If instead the value creation and value capture of the new AI technology are concentrated in one or a few firms, these firms constitute choke points: their terms, prices and outages propagate everywhere at once. They also have the power to extract undue rents and exercise excessive bargaining power on the rest of the economy and society at large."
Also, a flaw in a widely used model hits all its users simultaneously, Brunnermeier said.
"Hence, it is important to foster some commodification of AI models in the U.S., ideally before concentrations arise," he said. "Early regulation that leans against product differentiation by AI firms and ensures low switching costs across versions and vendors is an urgent step. Switching to rival models and to previous-generation models should remain possible to minimize systemic AI dependency risk."
Meghji also worries about concentration risk. "If the entire industry rests on fewer than three models, you get something similar to the concentration risk we see with banking cores and cloud providers except it is far riskier than either, in my opinion," Meghji said. "If something goes wrong, it affects everyone at the same second. There is no examination hook, process or model that supports that currently."
New forms of fraud and manipulation
Brunnermeier also touched on the cybersecurity and fraud risks brought on by AI adoption.
"Fake news, misleading advice and fraudulent products can be generated as cheaply as the real thing, and the same capabilities power AI-driven cyber attacks and phishing, as well as fabricated evidence," he said. A prompt injection or hallucination can become an unauthorized trade or funds transfer. "More subtly, AI providers may gain the power to shape or manipulate financial products in ways no outside party can observe," he said.
He also addressed the challenges raised by validating AI model outputs and actions with other AI agents, for instance to verify a robo-adviser's recommendation, screen a novel structured product or flag a manipulated transaction.
"The appeal is that validation, though it requires understanding, need not be human understanding; if enough capable models concur, the human principal can act on the consensus without following the reasoning," Brunnermeier said.
Read more:
- These are the
Top Performing Banks of 2026 in four asset tiers - New York bank
regulator says vendor flaw hit its firms - The
best credit unions to work for in 2026 Tokenized deposits are here . Banks need to manage the risks.
But having bots validate other bots raises questions, he said. "First, how do the validators interact, and do they share a common understanding among themselves? If they are near-copies, drawn from the same foundation model, they will tend to make correlated errors and miss the same manipulations, so apparent agreement conveys little: a hundred near-identical validators are close to a single validator voting a hundred times."
The second question is, will the validators collude? "Because the check now runs machine-to-machine at machine speed, tacit coordination is easier to sustain and harder to observe than among human auditors: a joint misalignment problem in which the swarm converges on passing what it should flag, whether through explicit game-play or through shared blind spots."
Third, he asked, who validates the validators? "The scheme presupposes that the principal understands the swarm well enough to trust its aggregate verdict, which simply reproduces the original asymmetric understanding problem one level up," Brunnermeier said.
Bank regulators should be able to evaluate new AI models before they are released to the public, he concluded. An
