Financial services companies spend huge sums of money on technology projects that they hope will make some kind of improvement.
Some are designed to cut costs, some to drive up revenue, others to increase productivity - all are supposed to help the bottom line. So are they accomplishing these things?
As it turns out, many companies don't know.
Though the tech groups generally keep track of how much they need to complete a specific project, and the accountants monitor disbursements, in many cases oversight stops once the development team hands a new system over to the unit that will put it to use.
So a company may know precisely what it spent designing and building the next great thing but be far less certain that it really is all that great, this despite the fact that return on investment has been the industry mantra for years.
A PricewaterhouseCoopers LLP survey of 227 financial services companies in North America, Europe, and Asia found that 30% have no mechanism at all to determine if technology initiatives are doing what they were originally meant to do. The executives who took part in the survey were generally not satisfied with their tech departments - on a 5-point scale, with 5 being the worst, they gave their IT management an average score of 2.97.
The poll, conducted in October and November, covered all types of financial companies, though the results are applicable for just the banking industry, the company said.
"There are one or two banks that do this really well, but they are in the minority," said Andrew Gray, a partner with Pricewaterhouse and the head of the consulting company's U.K. banking group.
Mr. Gray, who worked on the report, said the vast majority of banks segregate the development teams from the business lines. Once the tech people finish installing, debugging, fine-tuning, and testing their creations, they move on to the next job.
When they do, they leave behind complicated new systems that may or may not live up to expectations. Most of the time, they do not, according to M. Arthur Gillis, the president of Computer Based Solutions Inc., a Dallas research and consulting firm.
"Practically all big projects fail, because they don't deliver what was promised," he said.
So how can banks ensure that they are getting what they paid for? One differentiator, at least at JPMorgan Chase & Co., has to do with the accountability that resides with business-unit technology executives. Austin Adams, the company's chief information officer, [by way of Bank One Corp.], said he tries to make the top technology executives in each business unit accountable, first to the head of their unit and ultimately to him.
Any IT project with a budget of $250,000 or more has several review stages at JPMorgan Chase, Mr. Adams said. First, executives with a good idea need to get approval even before spending the time and money to develop a full-blown business case that details the pros and cons. The concept is reviewed again after the business case is completed, showing the budget and anticipated benefits.
During the development process, any initiative that goes more than 10% off-budget will automatically get reexamined, Mr. Adams said. And finally, the company does a post-implementation review.
One key to making sure he gets the full return on his investment is to incorporate long-term projections into each business case and then link each department's budget to those proposals, Mr. Adams said.
For example, if the investment banking operation installed a new order-tracking system last year that was originally promised to supply $10 million in savings in 2006, there are people tasked to keep track of those figures. When the corporate budget group sits down with the head of the investment bank to plan the 2006 budget, those savings will be accounted for in the budget process.
That doesn't mean the investment bank is being penalized, Mr. Adams noted, adding that often, units do get the money back so they can spend it on other initiatives. The process simply reflects the fact that the operation has promised certain savings. Regardless of whether the system actually provides those savings, the company is now making plans based on those estimates.
Mr. Adams said this approach has motivated executives to make sure their IT projects fulfill their promise.
"I find the system works best when the great scorekeeper in the sky, the budget head, keeps a record," he said.
Mr. Gray said this type of project ownership is not the norm.
"Most project managers are incentivized on implementation," he said, so their main goal is to complete a project and then hand it over to the business unit that will actually be using it. Few companies assign anybody to monitor the effort all the way from concept to real-world use.
Without such a sponsor, the business unit might be pleased with a system that increases throughput by 15%, never realizing that it was originally intended to deliver a 30% bump. "The people who own the system," Mr. Gray said, often "have no performance metrics in place to measure the impact of a system."
Leslie M. Muma, the president and chief executive of Fiserv Inc., a technology outsourcing company in Brookfield, Wis. that works primarily with banks far smaller than JPMorgan, said that frequently "there is not enough time spent up front trying to define what gains are expected, and then tracking the project afterwards to see if you meet that."
Still, about a quarter of the banks that Fiserv works with do this pre- and post-project work versus about 10% a decade ago, Mr. Muma said. "More and more banks are looking more carefully at the return on their investment," he said.
Danny Peltz, the executive vice president of wholesale Internet and treasury solutions at Wells Fargo & Co., said that in his group the tech people sit right next to the business people, and they work together to identify the unit's IT needs and implement the systems. "We have business people sponsor a project from the beginning," he said, and he always schedules post-implementation reviews.
"Most of the time, it's pretty obvious" if the technology is working as expected, Mr. Peltz said.
That is, if someone is looking.
William Wray, the chief information officer at Citizens Financial Group Inc. in Providence, R.I., a subsidiary of Royal Bank of Scotland Group PLC, said his project management efforts always include measuring performance when a system is up and running. It's important to chart improvements, he said, but it's not always necessary to translate every gain into a dollar value.
For example, Citizens finished upgrading its branch sales and servicing platform last year, and as expected the upgrade has helped increase the number of sales per session by about 15%.
The same technology has also allowed branch staff spend less time on those sales - 10 minutes per session instead of 45.
It would be possible to calculate the exact value of saved labor, Mr. Wray said, but that misses the point. The most important gain in this case is improving the experience for both the customer and employee.
Valuable? Certainly, but harder to quantify. And in this case, he says, just knowing it is better is enough. "There is no need to make a fetish about ROI."
