- Key insight: A push to using large language models to generate software code is causing service disruptions at Amazon.
- Quote: "Amazon had four critical incidents in a week, and their own memo said safeguards 'aren't yet fully established.' Are you kidding me?" —Bradley Leimer, consultant
- Forward look: At banks and other companies that use vibe coding, guardrails and human review need to be in place.
According to the
"Amazon is holding a mandatory meeting about AI breaking its systems," security researcher Lukasz Olejnik wrote on X. "The briefing note describes a trend of incidents with 'high blast radius' caused by 'Gen-AI assisted changes' for which 'best practices and safeguards are not yet fully established.' Translation to human language: we gave AI to engineers and things keep breaking?"
An Amazon spokeswoman said the Tuesday meeting was a regular weekly operations meeting "where we review operational performance across our store. As part of normal business, the meeting will include a review of the availability of our website and app as we focus on continual improvement."
A person familiar with the matter said Amazon Web Services, the company's cloud computing division, was not involved in the outages. "Only one incident was related to AI, and none of the incidents involved AI-written code," this person said. "It is not accurate that junior and mid-level engineers require senior engineers to sign off any AI-assisted changes."
More than a dozen current and former Amazon employees recently told
A software developer told The Guardian that programmers are expected to use an internally created AI tool called Kiro that's designed to automate software development from concept to production. The model frequently hallucinates and generates flawed code, according to the software programmer, who did not give her full name and was fired shortly after the article was published. Programmers have to comb through and correct the sloppy code, or just start over again, she said.
"I and many of my colleagues don't feel that it actually makes us that much faster," she said. "But from management, we are certainly getting messaging that we have to go faster, this will make us go faster, and that speed is the number one priority."
Among seasoned software engineers and bankers, the reaction to the reporting on vibe-coding at Amazon was scathing.
"Were juniors just vibe-coding and pushing straight to [production] without oversight or review?" wrote Nick VanGilder, senior vice president of offensive security at Regions Bank, in a LinkedIn post. "Maybe Amazon gained some efficiencies and cost savings by introducing AI and laying off a bunch of people, but now it sounds like the new bottleneck will be humans reviewing all the potentially dangerous AI vibe code."
Vibe-coding and AI-assisted coding issues show what's wrong with the tech industry's move-fast-and-break-things ethos, according to Bradley Leimer, principal consultant at Leimer One Advisors.
"Amazon had four critical incidents in a week, and their own memo said safeguards 'aren't yet fully established.' Are you kidding me?" Leimer told American Banker. "Amazon cuts 30,000 people and tells employees to have AI to pick up the slack basically. Then they're surprised when no one is left to catch the snippets of code that AI got wrong."
If a similar vibe-coded-related outage had occurred in a bank, it would be a regulatory event, Leimer said.
"In banking, this would end the careers of entire teams," Leimer said. "Or at least it should. Imagine if this type of system outage becomes common in banking, in payments? Money has zero tolerance for downtime. Customers might forgive an online retailer, to a certain extent. It's inconvenient. But they will not forgive a bank brand. Amazon has an AI governance problem, that's the lesson for banks."
Theodora Lau, founder of the investment fund Unconventional Ventures, told American Banker that the Amazon case "validates what we've been saying all along: The coding tool is in fact, a tool, not a replacement. The problem isn't so much so with the tool itself; rather, it's how it's being used and the expectations on what it can do."
Better guidelines and guardrails, processes and procedures are needed to ensure humans remain in the loop, to validate and verify, before changes are committed to a live environment, she said.
"I do wonder, however, if the challenges that we see with Amazon and AWS could be partially attributed to the massive layoffs that they have had, leading to the loss of crucial knowledge and expertise with the systems, which have also grown more complex over time," Lau said. "And then when you let your guard down and put all the faith in a tool … you end up with a perfect storm. Vibe-coding is great for creating prototypes. But human expertise and oversight are still essential."
