A growing number of credit union IT executives are turning to cloud-based disaster recovery solutions to help maintain or restore operations after disasters ranging from simple power outages to earthquakes and hurricanes.
Those events – no matter how severe – can have a major impact on operations, causing downtime and financial loss, sometimes to the tune of thousands of dollars per hour.
At Mid-Hudson Valley FCU in Kingston, N.Y., VP of Risk Management Jessica Fox said the credit union has utilized a variety of disaster-recovery platforms, “From big fancy software that created hundreds of pages of documentation automatically to self-formatted excel worksheets. Both were housed locally.”
According to Zetta’s 2016 State of Disaster Recovery report, which polled 406 IT executives in 20 different industries, 96 percent of respondents have a disaster recovery (DR) solution with 50 percent using cloud providers. However, only 9 percent of respondents are operating exclusively on a cloud platform.
“Our culture is to constantly evolve and improve, and the next logical step was to a cloud-based solution that provided a standard framework while allowing us to customize our existing documentation in a meaningful and available way,” said Fox.
While Fox said her CU hasn’t experienced a significant disaster, her team spent “several years” brainstorming solutions to recover from the possibility of one occurring. This included instituting a functioning “hot site,” an offsite location greater than 75 miles from the CU’s primary site mirroring its headquarters. The cost, however, far outweighed the possibility of needing it, she said.
“We researched reciprocal scenarios leveraging out of market credit unions and we considered large organizations that offer hot sites,” said Fox. “In the end, the scenarios that would most likely land us in a recovery situation were not feasible in most of these options, or the cost was just overwhelming.”
The $913 million Mid-Hudson Valley FCU, which supports 73,000 members and 13 branches, was recently also in need of a “planned recovery.” This was due to a new branch office that was going to be completed ahead of the scheduled opening. So the CU partnered with the Westminster, Col.-based Agility Recovery, which deployed a mobile branch recovery unit for three months.
“Our disaster recovery solution enabled us – in within approximately a 10-day period – to open a temporary location equidistant from the old location and the new location,” said Fox. “It was fully functioning branch location complete with a teller line and MSR desks.”
Confidence in the cloud
For the $140 million Tucson Old Pueblo CU, which supports 12,7000 members at three branches, finding a suitable disaster-recovery solution required a good deal of vendor vetting. After due diligence, the Layton, Utah-based CUProdigy and its disaster-recovery-as-a-service (DRaaS) was selected.
“The days of carting tapes to a hot site are long gone,” said CUProdigy CEO Anthony Montgomery. He explained that Tucson Old Pueblo CU is the firm’s first cloud-based disaster-recovery client that doesn’t operate on its core system. Rather, the CU uses Symitar’s Episys, which is provided by Member Driven Technologies.
“We were able to utilize our existing Internet connection and replicate the data, near real time, to CUProdigy’s data center in Salt Lake City and utilize their virtual data center for our needs, if a disaster arose,” said Beausoleil.
Prior to rolling out DRaaS in April 2016, a process that took nearly two months, Tucson Old Pueblo CU was using a Hyper-V cluster and SAN replication to a DR site.
“Due to limits of our SAN equipment we did nightly replications,” said Beausoleil. “This meant that in the event of a disaster we had the possibility of having to recreate data between the replication time and recovery time, which was unacceptable for us.”
From a technology and training perspective, Beausoleil said the process was “pretty painless,” although by adding a cross hypervisor replication, he had to install System Center VMM on site to support the Zerto client. This process added time to the launch date.
“We were able to retire aging equipment at our [disaster-recovery] site and not have to replace the equipment, saving the credit union some unneeded expenditures,” said Beausoleil. “We went from hours of downtime to recover, to minutes. As an IT staff, we can focus on staff and members in a disaster now, instead of hardware.”
