Inside Wells Fargo's cybersecurity war room
To keep their cyberteams fit and ready to defend against a cyberattack at any time, large banks have begun using “cyber ranges” — virtual environments where real cyberattacks are launched on replicas of their actual IT systems.
“Special forces in the military train with live bullets shooting around them, so when and if they're in the time of battle, they're not ducking and covering because of these loud banging noises,” said Rich Baich, Wells Fargo's chief information security officer. “A cyber range is the same thing — your machine is actually being attacked. It’s no longer theoretical.”
Baich was a member of the U.S. Navy for more than 20 years and worked as a Naval Information Warfare officer for the National Security Agency. He is also chairman of the Financial Services Sector Coordinating Council, a group set up to help the financial services sector prepare for and recover from cyberattacks.
He said he believes cyber ranges are what’s next for banks when it comes to dealing with the asymmetric cyberattacks.
In some ways, cyber ranges are an extension of banks recruiting people like Baich. It's kind of like the movie "The Last Samurai," in which an elite, dedicated group of warriors spend their days training for war and getting ready for the next attack.
“As financial institutions continue to grow expertise among individuals that have advanced cyberwarfarelike capabilities, those warriors need a place to practice and play and test their skills,” Baich said.
In the past, banks have primarily run paper or desktop cyberattack simulations that focused on who would call whom in the event of a cybercatastrophe.
“It’s now emerged to a cyber range, a cybersimulation that allows those cyberwarriors to respond to real-life infections and malware, strengthen their skills, improve the controls in their environment and get ready for what may come one day from a malicious or nation-state actor,” Baich said. “In a cyber range, you take real action and, since it's a virtual environment, it will not impact production systems.”
Most large banks are using cyber ranges. But the technique has yet to reach the midtier and smaller banks.
Chris Thompson, senior managing director and head of financial services cybersecurity and resilience at Accenture Security, said many organizations can’t afford it.
“It’s expensive to build a cyber range or to have a sophisticated red team, and the skills needed to build those ranges are scarce,” Thompson said. A red team is an independent group that challenges an organization to improve its effectiveness.
“The people who run those exercises are demanding high salaries and are hard to get hold of. So there’s a danger the midtier banks can get left out,” Thompson said.
There are broader industry efforts to bring cyber ranges to others.
From his FSSCC chairman podium, Baich advocates the idea that the industry should create shared cyber ranges all could use, perhaps hosted by the Financial Services Information Sharing and Analysis Center, the industry’s cyberthreat information-sharing hub. The FS-ISAC already develops and participates in cyberexercises, such as the “Hamilton Series” cybersecurity tabletop simulation exercises sponsored by the U.S. Treasury Department. It also runs a table top exercise for hundreds of financial institutions called the Cyber-Attack Against Payment Systems or CAPS. It’s exploring the use of cyber ranges.
“By adding cyber range exercises to our suite of table top exercises that the FS-ISAC already provides, FS-ISAC hopes to provide our 7,000 financial institutions members with additional technology and cross-training tools to help the financial sector more effectively mitigate cyberattacks,” the group said in a statement. It turned down requests for an interview.
The Securities Industry and Financial Markets Association is said to be planning to use a cyber range in the Quantum Dawn cybersecurity attack simulation this year for the first time. The exercise, which runs every October, brings Wall Street firms together to practice how they would cope with a cyberattack.
Wells Fargo’s approach
Wells Fargo was one of the first banks to build a cyber range. It quietly installed cyber ranges in a geographically diverse group of its cyberthreat fusion centers two years ago.
The bank runs fake cyberattacks every quarter — each one a little harder than the one before it. Team members learn more from practicing in the cyber range than they do from years of reading books, Baich said. “It’s immediate, hands-on, real combat.”
To try to replicate an entire IT infrastructure within a sandbox would be extremely expensive, Baich said. What banks typically do is choose an area of focus and add on over time.
“It’s almost like building blocks, because you wouldn’t go and try to do the entire network at first — it might take you years,” Baich said. “But maybe you do payment systems, maybe you do ATMs, maybe you do Swift, maybe you do routers.”
Most banks will need help, Baich said.
“It’s not something you do yourself. You work with vendors, they develop and control the range,” he said. Cyberbit, Raytheon, Lockheed and SimSpace are among the vendors that offer this.
IBM’s $200 million cyber range
IBM does not sell cyber-range software, but it spent $200 million to build a cyber range of its own that opened in Cambridge, Mass., in November. It’s a data center similar to what you’d find in a Fortune 500 company, with the ability to simulate the types of technology used in various types of companies, from health care to energy to financial institutions.
Client company executives and board members come to the cyber range to test their ability to handle a data breach. 800 executives have been through the program so far, most of them in financial services, and the cyber range is typically booked three months out, according to Caleb Barlow, vice president of threat intelligence at IBM.
“We like to bring in a whole board or an entire C-suite, and we can simulate for them their worst day and give them the opportunity to learn how to get through it,” Barlow said. “When people come here, they break into a sweat. We put them through a completely immersive scenario in which they’ve got to deal with press, regulators, law enforcement, technology things that are going to hit them that they never anticipated.”
Some clients come with specific situations they want to test their response to. Others say, “Throw your best at us and let’s see how we do,” Barlow said.
The simulation trains the executives to do “System 1” thinking — to make quick decisions based on limited information, versus the more typical System 2 thinking: slow, deliberate, data-centric decision-making, Barlow said.
“When you’re under a breach you have to make decisions with limited information and they can have significant implications,” Barlow said. “We teach how to make those decisions rapidly, to test and rehearse them to the point where they become muscle memory. You’re up against a human adversary, so that human adversary can pivot, can jog and adjust. You, too, must be able to pivot and jog and adjust, and do it faster than they do.”
IBM developed its cyber range with help from the U.S. Air Force. Some exercises practice the use of what’s called an OODA loop, a method of decision-making that fighter pilots use that’s based on four things: observe, orient, decide and act.
Use of the cyber range is free. But while IBM doesn’t sell cyber-range technology, it does sell security solutions.
The next level of cyberwarfare preparedness, in Thompson’s view, is simulated attacks on the actual enterprise that are handled by red teams.
“In military terms, this would be your live-fire training in the field rather than a range,” Thompson said.
“Our clients don’t want people in security operations to know there’s a simulated attack going on, they want it to feel like a real attack,” he said.
While no bank would launch malicious software into itsr real environment, banks are launching carefully modified versions of malware, with malicious parts removed, Thompson said.
“If you’re simulating Wiper, it’s a very destructive malware,” he said. “Clearly the virus you use doesn’t erase machines.” The modified version would put a benign file on hundreds of machines that the red team would have to find and eradicate.
The security team doesn’t get any advance warning. “All they see is malware in the environment,” Thompson said.
The advantage of using real malware in a production environment is that often in real life,
bad guys sneak through things like misconfigurations or a lapse in two-factor authentication. Red team tests find these vulnerabilities, where a cyber range may not.
Editor at Large Penny Crosman welcomes feedback at email@example.com.