'Not a Conspiracy, Just a Bad Day': Bank Reactions to Glitchfest
The New York Stock Exchange halted trading in all securities as it addressed a computer malfunction and said it canceled all outstanding orders.
The company is addressing a problem more institutions will face as consumers increasingly bank from mobile devices. Application performance management software also helps Comerica deal with problems throughout its infrastructure.
With an airline, a national business newspaper and the New York Stock Exchange all suffering outages Wednesday, it was no surprise when regulators checked in with a West Coast bank.
"It came across quickly, we had people internally get the news and the regulatory community reached out as well to see how it was impacting us," said the bank's chief information officer, who did not want to be identified. "We got our business continuity team to report. They were executing their plan. They were able to determine that the disruption was manageable."
Like many others in banking and the media, this CIO wondered if there was a connection between the glitches.
"What's really happening?" he recalled thinking. "We see what's on the media, but you look at the other events happening in the same time period and it does cause you to be vigilant around what might be said and what might not be said. In this world of skepticism, you don't ever want to underestimate the possibilities."
Business went on as usual at his and other commercial banks Wednesday. But the confluence of events served as an unnerving reminder of the cyber risks facing the financial services industry.
United Airlines grounded 4,900 flights due to "a network connectivity issue," The Wall Street Journal's website went down for a short time, and the NYSE was forced to shut down for three and a half hours due to an unspecified computer glitch.
The Department of Homeland Security, the White House and the Financial Services Information Sharing and Analysis Center all issued statements saying they had no evidence of a cyberatttack. Despite the reassurances, speculation that the events were somehow connected was probably inevitable.
"In this day and age, my initial thought was, is this security related?" said James Gordon, chief technology officer at the $1.7 billion-asset Needham Bank in Massachusetts. "Once is an accident, twice is a trend, and three times is a sure thing. Either we're trending in the wrong direction or maybe today is just a bad day. You don't know that until you know."
And bankers in his state had one more reason to be suspicious. Gordon said some of his colleagues mentioned the outages to him, observing that the Massachusetts Bankers Association website has also been down for close to a week.
"It starts to add up in people's minds 'I don't buy this anymore, what's really going on?'" he said. (For the record: The regional trade group said it's been replacing an old firewall.)
Gordon said he wasn't too concerned that his bank would be a target for a coordinated cyberattack, due to its small size. "If you're going to pull a great train heist, you start with the car in front first," he said.
But incidents like this make him think about the service-level agreements he has with his technology vendors, which set expectations for performance.
"The only thing that has a 100% SLA is the sun rising or death and taxes," he said. "Everything else is a percentage thereof."
But he also thought, "Before the grace go I rushing to judgment won't calm down any markets."
The CIO at the West Coast bank said it, too, was not dramatically affected.
"It's a disruption to the business but one that's not as meaningful as for the big trading houses," he said. "Right now I'd call this minor."
Adam Honore, CEO at consulting firm MarketsTech and Wall Street veteran, also sounded unfazed.
"You can still trade everything, that's the whole point of Reg NMS," which allows NYSE trades to be carried out on other exchanges, such as the Nasdaq. "It's weird. It doesn't look like there's any correlation to United or The Wall Street Journal, but it's definitely not a gold-star day for IT." United said its problem is a router issue, he noted.
"It's a bunch of weird of coincidences," he said. "Even as a libertarian, I'm telling you, this is not a conspiracy. It's just a bad day. The markets have been relatively calm about it, although it sucks that all the trades got canceled for the people who were trading. It is what it is. Welcome to the new world."
The NYSE could be moving code into production, he speculated, pointing out that electronic exchange BATS had a similar problem on Monday.
"Every exchange has had outages in the last few years," he said. "The great thing about United having their outage today is that it shows it's not industry-specific. You can't pick on capital markets for having an outage."
Former NYSE CIO Steve Rubinow put the day's events in perspective.
"As important and well-designed as those systems are, your objective is to make sure disruptions are as infrequent as possible," said Rubinow, who is now CTO of Catalina, a digital media company in Chicago. "The biggest problem is human error, which will happen from time to time."
The long downtime is unusual, he acknowledged.
"My guess I that they either don't know the cause of the problem or they're not sure," he said. "You don't want to bring the system back up unless you know it's sufficiently bulletproof. They wouldn't bring it back up unless it would have all the integrity they need."
It's impossible to test every condition that could happen in the universe, Rubinow pointed out.
"You do the best you can. There will always be something you didn't anticipate," he said.