As Wachovia Mortgage Group's director of operations, Tom Wood watched the refinance boom gather steam in recent years and wondered whether the bank was doing all it could to capitalize on it. "We believe we lost a lot of opportunities," he says.
So Wachovia Mortgage started migrating to a Web-based loan-origination system more than two years ago in an effort to create an end-to-end, scalable fulfillment process by early 2006. Under the old system, loans took about 50 days to close. Under the new system, the goal is 14 days, he says. Wachovia hopes to double productivity over the next two years as the new system rolls out across all sales channels. And David Aach, COO at Englewood Cliffs, NJ-based Palisades Technology Partners, which is working with Wachovia on its IT overhaul, says that's an important yardstick. "The true measure of return on investment is to improve your productivity," he says.
Of course, the question for banks is how to effectively predict those benefits before spending millions on a project. That's the crux of ROI. Whether defining productivity or assessing how much a project might drive top-line growth, each institution must figure out which factors are measurable and which are not. And much of that can vary, depending on the bank and the scope and/or nature of the project.
For example, Wachovia didn't even try to predict how much shorter loan closings will actually increase mortgage sales-even though that's the assumed result. "The gravy is that our sales folks can increase their sales dramatically," Wood says. "But that's not in the business case." After all, interest rates could climb higher, creating less overall demand for mortgages. Or conversely, the hot housing market could get even hotter, driving new overall demand. In either case, it would be difficult to know how much of each scenario resulted from macroeconomic factors and how much was tied to Wachovia's internal IT improvements. "We could put an artificial number in there, but there are just so many things that could change," Wood says.
Going beyond the simple "money-in, money-out" metrics that typify traditional ROI remains easier said than done. But it also remains a common goal as banks try to better understand the full benefits of undertaking often expensive and daunting IT projects. "Banks are not just looking at the hard-dollar cost in measuring IT effectiveness," says Ellen Joyner, SAS's global industry strategist for financial services. "Due to a continued increase in mergers and acquisitions, IT departments are being put on the spot to ensure the business processes are transparent and the flow of information is being properly analyzed for quality and performance."
As a result, CIOs are finding they can no longer justify IT projects based simply on "fear, uncertainty and doubt," says Gregory Hedges, managing director of technology risk consulting at Protiviti. "That has been quite a successful way to get money for a long time," he says. "IT departments that don't quantify results become more like a utility and make themselves a great case to be outsourced." Overcoming that fate requires IT managers to drill deeper into ROI metrics than they have done in the past. "It's the intangible things," Hedges says. "We've got to measure them. There's a lot of room to improve on metrics."
The trick, of course, is figuring out exactly how to effectively measure those intangibles. And there's no real consensus. Many debate whether it's smart to assign metrics to everything under the sun when some business results are unpredictable. A new IT system may indeed enable workers to complete tasks more quickly, but employees won't necessarily spend all of their new-found free time trying to increase sales. In some cases, more efficient IT systems might actually give employees more rather than less time to socialize at the water cooler. "Anyone who's going to buy a multimillion-dollar system based on water-cooler savings is crazy," says Frank Florence, svp of marketing for Dorado Corp. "That's kind of kindergarten ROI."
That's not to say that banks shouldn't consider potential "soft savings" before undertaking an IT project, but Florence says trying to create ROI metrics for factors such as improved customer satisfaction can get firms in trouble quickly. "What's the value of a more satisfied customer?" he says. "You can't say, 'We know this will make a customer seven percent more happy.'" Of course, while a soft metric like customer satisfaction may be difficult to gauge, "it might be more important for the long-term health of the company" and, therefore, something that should at least be in the mix, notes Tom Westcott, founder and CEO of Marlboro, MA-based Project Solutions Group. "You need to invest in short-term returns as well as long-term returns."
Banks, however, are often skeptical of ROI models that include too many factors. And in many cases, banks devise IT projects based more on general business goals than ROI metrics-especially when undertaking smaller projects. For example, ESSA Bank & Trust, a 12-branch bank in northeastern Pennsylvania, recently finished the successful installation of an internal intranet system from Norristown, PA-based software firm Mindbridge, but the bank didn't run any ROI numbers beforehand. "We really have never had to base any hard numbers on why we needed it," says Suzie Farley, ESSA's corporate secretary. "We're more of the mindset of trying to constantly improve ourselves and improve customer service." Of course, Farley adds that the bank is now documenting its workflow process, and "perhaps we'll glean some kind of return on investment from that."
ESSA Bank & Trust's take on ROI isn't necessarily unusual within the banking industry. According to vendors, even large banks often undertake complicated IT projects with few or no ROI measures in place. "I don't really see a lot of ROIs going on-even just the 'money-in, money-out' types," says Anthony Jabbour, evp for products at Fidelity Information Services. "It's really a shame because ROI is critical." Westcott says many banks "know they want to start measuring; they just don't know how." Ashwin Goyal, vp and general manager of retail banking at Siebel Systems, adds that he's sometimes amazed that "people who have spent millions of dollars on an IT deployment haven't done the simple blocking and tackling to understand what it all means."
Siebel Systems, for example, has devised a complex ROI program that sets baseline metrics, defines and achieves consensus, creates a solution, delivers an enterprise-wide implementation plan and only then determines what ROI metrics come into play. "You have to figure out if it's just 'money-in, money-out' or whether there are other factors," Goyal says, noting that Siebel Systems has come up with some 1,200 detailed ROI metrics based on its years of experience running IT projects. "For almost every situation you're trying to measure, there is a metric," he says. "More often than not, you'll have a metric you can use. Or at least you'll have a proxy to use." According to Goyal, proxy metrics can combine two or more established metrics to estimate an outcome. For example, it's difficult to specifically measure employee morale, but a company might be able to combine its employee- satisfaction survey results with various productivity metrics to determine whether a new IT system improved or degraded overall morale levels.
Jay Simons, vp of product marketing and strategy at Plumtree Software, agrees that "you should measure soft benefits as well as hard benefits, but it does take study. It takes a concerted effort. You really have to poll the business." For example, determining how much a streamlined IT system will lower employee training costs can be difficult, but "there are some factual things you can plug into the formula," he says, such as the average amount of time it takes an employee to train on the existing system. The resulting metric-when combined with other more tangible metrics-can help form a larger ROI picture. "You want to be hyper-conservative," he says, "but that's a calculation we're willing to put forward." He acknowledges that soft metrics require "making some assumptions," but adds, "I think you've got to start somewhere. ...You want to provide some general guidance. ...And when customers go through the planning exercise, they're better positioned to go into it after the fact and make a measurement."
For example, Plumtree client MassHousing, a state-sponsored bank that makes low-interest loans to low- and middle-income families in Massachusetts, recently ran some ROI metrics before upgrading its IT system, but found out only after a post-project evaluation that final results had exceeded expectations: lowering transaction costs from $1,600 to $600 and cutting the processing workload in half. "That was a pleasant surprise," says MassHousing IT Director David O'Connor, who notes that the reduced workload has freed up employees for other tasks. "We took those analysts and re-purposed them," he says. "Instead of sitting there on the phone entering information, now they're out on the road drumming up business."
Of course, many institutions frown upon post-project evaluations because they are time-consuming and can seem pointless when the money has already been spent. "But that's kind of a nutty perspective," says Simons, noting that projects can often be tweaked or improved even after completion. "And if the project isn't contributing to the business, kill it," he says. "That's why it's important to measure after the fact." Todd Amsley, managing director of financial services at Buffalo, NY-based CTG, an IT solutions company, says banks can even take a phased approach to IT projects to get a "proof of concept" before going further. "You're better off knowing that on a small scale," he says. "You've got to test it."
According to Fidelity's Jabbour, another benefit of IT post-mortems is that "when you've got the accountability built within the organization, there's no loss of focus throughout the project." In other words, an IT department that knows final results will end up under the microscope will work even harder to meet objectives. "Everyone needs to be accountable," he says.
Furthermore, an institution might do a post-analysis review to learn from any mistakes, increasing the chance that planners will create more accurate ROI models for future projects. "Unless you do the post-investment analysis, you're really nowhere," says Dorado's Florence. At the same time, banks may also want to be careful when applying lessons learned to new projects that contain different variables. "Looking back is good, but it's not enough," says Westcott. "By the time you have applied the changes, the conditions have changed anyway." Instead, he says banks should put more emphasis on critical-thinking skills that can anticipate unforeseen events and lead to smarter ROI models. "We need to focus less on the past and get better skills for looking to the future," he says.
