We would do well to remember that "market projection" and "real-world results" are not interchangeable terms. The distinction between them is inconvenient, however, for software vendors, equipment makers, and service providers that would have us believe the most aggressive projections for every new Internet banking offering.
This view is based on two closely related observations. First, even the best market research can be - and is not infrequently - wrong. Second, sponsored research - studies funded by companies with a vested interest in swaying buyers - has a funny way of falling well short of the standard implied by the term "best."
Most of us can recall an infamous product launching - one that produced a spectacular failure despite exhaustive research and compelling analysis. Marketing and merchandising paragon Coca-Cola reformulated Coke because its research concluded that consumer tastes had shifted.
But the so-called New Coke ranks with the Ford Edsel among business' most notorious marketing debacles.
Lesser misadventures in marketing are seen daily, illustrated by a common occurrence: the clearance sale.
To understand what can go wrong with market analysis, it is necessary to remember that projections are based on at least one hypothesis - an educated guess that research tries to validate. As we all know, such assumptions - and attempts to validate them - can be problematic.
Take adoption-rate forecasts, for example. When they are off, the error is not so much that no one will use the product or service as that far fewer people than predicted will use it. This flaw is compounded by the fact that people who do buy will take longer than anticipated to do so - and only after pricing concessions.
The failure of these assessments frequently manifests itself as a product used by a small but unprofitable cadre of dedicated customers. Almost every bank can point to a handful of accounts still on the books that were generated from an unsuccessful product launching.
If nothing else, experience, not forecasting, has taught us that a certain number of consumers and businesses can be counted on to try just about anything. Even New Coke captured consumers who liked it better than the original, and Ford sold a few Edsels.
None of this is intended as a blanket condemnation of companies that have tried and failed. For example, no one would suggest that the people who launched Iridium, the global wireless communication company that blew through billions of dollars before failing, lacked reasonable market research. They were just dead wrong about the timing of demand and consumer willingness to pay for the service. Too bad it didn't turn out as projected.
Examples abound of inaccurate forecasts and subsequent misadventures in banking service launchings.
Consider the research and analysis that predicted people would bank with Sears if it put a financial center next to the tool department. It turned out that Sears had trouble just selling tools. Today Sears no longer owns any financial services business.
The faults inherent in market research are compounded by the involvement of commercial firms with vested interests in a study's outcome.
How credible is research claiming that wireless banking is the coming rage when the research is sponsored by a cellular phone maker?
Is research claiming that Internet sales effectiveness is compromised by poor sales follow-up believable when that research is funded by a sales management software vendor?
Are evaluations of banking Web sites reliable when the evaluator derives income by generating customer referrals to specific sites?
An interviewer can guarantee just about any response by carefully wording the question.
There is a science to determining a statistically valid sample size, and there is a rigorous and time-consuming methodology for validating the effectiveness of a given question for testing whatever you are trying to test. The structure of the underlying questions is fundamental to successfully eliciting useful, objective responses.
Unfortunately for the banking industry, the researchers looking at Internet banking routinely trumpet top-line survey results without releasing the detailed survey instrument or explaining the statistical significance of variances in responses. The audience is left to accept or reject the conclusions drawn by the researchers with little insight as to the efficacy of the research design or the possibility of alternative conclusions.
Unfortunately, what is presented today as market research is all too often nothing more than thinly veiled sales propaganda. Bankers would be well advised to research the researcher's methods, conclusions, and sponsors before accepting forecasts of consumer or business adoption of Internet banking and well in advance of actually investing in the latest wireless wonder, aggregation alternative, or bill payment proposition.
Tom McGrath is a managing partner of Bank Earnings International LLP, a consulting firm in Orange, Va.
