Lessons The Wind Blew In
Tim Beeson is convinced that backup-to-disc solutions-not conventional backups-to-tape-are the way to go for disaster and data recovery.
That's just one of the technology practices the Data Processing Manager at $4.4-billion Suncoast Schools Federal Credit Union recommends after being threatened by four Gulf Coast hurricanes in August and September.
The season's hurricanes coupled with the 9/11 attacks have lent a sense of urgency to disaster recovery. A Gartner, Inc. survey last year reporting that one-third of U.S. businesses were still ill-prepared for disasters reiterated the need for business continuity planning and technology-and testing.
Three CUs interviewed exclusively by The Credit Union Journal put their disaster recovery plans to the test as a result of the recent hurricanes.
"Don't do back-ups from your core system to tape," Beeson advised. "If at all possible, you need to get an EMC Symmetrix networked storage system."
The Symmetrix allows Beeson to make disc and tape backups from a business contingency volume (BCV), rather than taking down Suncoast Schools' core system for hours to make tape backups.
Symmetrix is offered by information storage and management solutions provider EMC Corp. of Hopkinton, Mass.
Better yet, Beeson can restore his transaction processing system in about 30 minutes, instead of the eight hours it took using tape backups.
During the hurricanes, Beeson had three copies of all his data-one on tape and two on BCV disc.
And instead of shutting down completely, the nation's eighth-largest CU was also successful running in store-and-forward mode during the hurricanes. After the CU went on alert, all transactions were recorded and stored but not forwarded and posted to the CU's Spectrum processing system until after the storms passed.
Spectrum is a real-time information processing application provided by Corvallis, Ore.-based Summit Information Systems, a Fiserv, Inc. subsidiary.
"During store-and-forward, nothing posts to the system, but to the outside world it still looked like we were up and running," Beeson explained. "Basically, we warehouse transactions, which makes for a clean and easy recovery process later."
Store-and-forward is "an epiphany in terms of system state at time of disaster," agreed Don Stewart, Summit's Disaster Recovery Coordinator.
Suncoast Schools FCU successfully posted 475,000 transactions coming out of four days of store-and-forward mode after one of the storms, Beeson said.
"With store-and-forward, my disc and tape backups, and my hot site in Philadelphia, I'm pretty comfortable that my member is protected," he said.
Suncoast Schools FCU suffered no damage from the hurricanes, Beeson added.
Alabama CU in Tuscaloosa, Ala. and America's First FCU of Birmingham, Ala., also successfully tested their disaster recovery plans during the hurricanes.
"Fortunately, we did not need to activate our recovery plan but this was an excellent test of our disaster recovery contingency plan in a real-life environment," said Vickie Yaeger, senior vice president at $610-million America's First FCU.
All three CUs confirmed that Summit's CloneLog solution offered additional peace of mind. Summit developed CloneLog in partnership with Beechglen, a Cincinnati, Ohio-based technical support and consulting firm.
The new solution automatically creates an off-site copy of critical Spectrum transaction and activity log files for all network systems. "That way, we not only had a full-system tape back-up but a copy of the logs right up to the point of disaster," explained Michael R. Shults, vice president of IT at $187-million Alabama CU.
For more info:
* Suncoast Schools FCU at www.suncoastfcu.org
* Alabama CU at www.alabamacu.org
* America's First FCU at www.amfirst.org
* Disaster Recovery Journal at www.drj.com
* Summit at www.summitsite.net
* EMC at www.emc.com