Behind Visa's outage: A clash between innovation and basic expectations

Legacy payment companies are under severe competitive and regulatory pressure to innovate, a trend that places increasing stress on older older parts of the payments system.

This innovation, spurred by PSD2 and broader e-commerce trends relies on platforms that, while frequently updated, are centralized and decades old. Companies like Visa and Mastercard talk openly about their concerns of being thought of as a dumb pipe for newer innovators, and have taken steps to open their networks to third parties to see them as partners instead of mere utilities.

Visa has vaguely blamed its Friday outage on a hardware issue, leaving the world to speculate on what single point of failure could have affected systems across Europe. But one thing is clear: This outage, however brief, undermines the card networks' ability to market themselves as the fundamental platforms for a new era of payments technology.

Andrew Harrer/Bloomberg

"Centralization is being challenged by the growth of open banking, with hundreds if not thousands of APIs pinging and putting further strain and vulnerability on older centralized systems," said Richard Crone, a payments consultant.

As if on cue, Visa this week announced a $100 million investment to support European fintechs. Starting in July, fintech startups in Europe can onboard to Visa's global network in about four weeks.

In another program, Visa will support businesses that are innovating in open banking and others that support open banking and new commerce experiences. This follows Visa's earlier investments in Klarna, Payworks and other companies that are designed to expedite Visa's digital migration.

"[The announcement] seems ironically positioned with the European outage, announced on the European stage of Money 20/20," Crone said. "The program and the API approach is also very strategic for protecting the Visa processing assets as the investment program and API method acknowledges, by default, the central role and power of Visa as the legacy central provider of the payment network."

Mastercard has also been active in this market, rearranging its management and making more investments in its developer platforms.

Visa, which didn't return a request for comment by Tuesday morning, has said little more about the outage other than to address perceived concerns over whether it was hacked. Its explanation did not provide enough information to blame external stress from new technology developments, but the lack of detail could plant enough seeds of uncertainty to worry Visa's next generation of fintech partners.

Paul Lomax, CTO of the rewards marketing app maker VirginRed, questioned Visa's stated explanation, signaling a lack confidence in the ability of legacy payments networks to handle the modern mobile-driven payments world.

A hardware failure should not cause an outage if you have modern infrastructure. Either VISA’s systems are not fit for purpose, or they’re run incompetently, or they’re lying... https://t.co/kAFzvc8RKA

— Paul Lomax (@PaulLomax) June 4, 2018

"I’m guessing it still runs on some old IBM system/360 hardware with many new layers on top, like most banks," Lomax said.

A Quora analysis of the networks' core systems suggests IBM technology under the hood. This case study from Visa shows its European system uses IBM AIX to power its server, WebSphere for business logic and IBM DB2 for its database. According to the Visa case study, recent improvements have shortened the time to make changes, and allow for a greater volume of changes. These changes have also improved Visa's ability to manage variable payment types. Visa's European business, which used to be a standalone operation, also has a reputation for encouraging innovation.

Another Visa document says the card brand has invested "hundreds of millions" of dollars to improve its processing capabilities to 24,000 transaction messages per second for a system that can trace its roots to the early 1970s.

Mastercard has been processing payments since 1966, according to a Mastercard spokesperson, who referred other questions to Mastercard's website. The Quora analysis linked to a Mastercard document that also suggests IBM DB2 as the power behind its database.

"We're all asking ourselves if there is a systemic vulnerability of older systems," Crone said. "This is an innovator's dilemma. You have a centralized core processing system and scale advantage. It's a barrier to entry for new companies, but it's also a chokepoint."

In today's digital era, organizations like Visa and many other financial organizations like them cannot tolerate any downtime, said Doron Pinhas, CTO of Continuity Software, adding that these companies invest heavily in High Availability technology, which supports continuous availability and resiliency architecture to ensure 100% uptime. "In other words, such [outages] should not happen," Pinhas said.

The underlying architecture that supports continuity includes at least two and typically three data centers with failover capacity; redundancy of power, networking, communication, software and storage; and verification software for manual processes, Pinhas said.

"It is clear, though, that as the hybrid IT environments become more complex with more hardware and software technologies and more interconnections between them, it becomes more difficult to avoid single points of failure across IT stacks and maintain the highest levels of IT resilience," Pinhas said.

The war on hackers and fraudsters is pushing the networks to never stop working to minimize the risk of an outage, said Thad Peterson, a senior analyst at Aite Group.

"Another risk mitigator is the explosive growth of data management that will help them predict and manage any outage risk, along with increasing levels of bandwidth, which will help to minimize latency," Peterson said. "The fact that this significant issue was essentially resolved within 24 hours also shows the resiliency of the networks."