Last in a four-part series.

On the surface, the payday lending business looks vulnerable to the threat of disruptive innovation.

The industry operates today in largely the same way it has for decades — through an inefficient network of retail stores. Incumbents face a slew of more forward-thinking new entrants. And breakthroughs in underwriting may render obsolete the industry's old model of using sky-high interest rates to offset soaring default rates.

And yet, for Americans with poor credit histories, the price of a small-dollar loan hasn't moved much in recent years. In California, prices barely budged between 2006, when the average annual percentage rate among licensed lenders was 429%, and 2011, when it was 411%.

"This isn't a perfect market, where you would expect, with perfect competition, the costs to go down," explains Shane Hadden, chief executive of Float Money, a startup that offers 0% interest loans to consumers who make purchases at specific retailers.

In a four-part series, American Banker is exploring the opportunities and challenges facing companies that are looking to upend the bottom of the U.S. consumer credit market. That segment is currently dominated by payday lenders, bank overdrafts and other expensive forms of credit.

The central problem in the market is a lack of price competition among small-dollar lenders. In well-functioning markets, competition drives prices down. But there's little evidence of that dynamic among small-dollar lenders.

"Is this a product that is expensive because it has to be, or because it can be?" asks Raj Date, an industry consultant and the former deputy director of the Consumer Financial Protection Bureau.

What follows is a discussion of five key reasons the cost of borrowing remains as high as it does for tens of millions of Americans at the lower end of the credit spectrum. If the prices are to come down — as consumer advocates, regulators and some startup lenders would all like to happen — these entrenched forces will have to be overcome.

Desperate Customers

People who walk into a payday loan shop tend to need cash right away, whether for a car repair or a rent payment or another immediate financial obligation. So they are not well positioned to find the best deal available. Their desperation may help explain why expensive loan providers have proven hard to dislodge.

Payday lenders also offer convenience by operating stores that are located in their customers' neighborhoods. "If it's really easy to do, and it's really high rate, that's what consumers will do," Hadden says.

Moreover, payday shops provide an implicit assurance that returning customers will be able to walk out of the store with the cash they're seeking. "And the fear of rejection is a powerful motivator for consumers," says Gregory Mills, a senior fellow at the Urban Institute's Center for Labor, Human Services and Population.

"Businesses compete along many different dimensions, including price, convenience, friendliness, and perceived usefulness," adds Jeremy Tobacman, an economics professor at the University of Pennsylvania. "So if for some reason price is less salient to the borrowers, you can imagine competition shifting toward these other features."

One more factor: for those customers who are price-sensitive, it can be hard to compare costs between various lenders. Some reform advocates, including former chairman of the Federal Deposit Insurance Corp. Chairman, Sheila Bair, hope to remedy that by requiring uniform disclosures nationwide.

"If the information isn't really accessible, and the disclosures aren't consistent, you're not going to get a lot of price competition," says Bair, who is now a senior advisor to the Pew Charitable Trusts.

One-Size-Fits-All Pricing

One of the most striking aspects of the lower end of the consumer loan market is the fact that everyone pays the same price, no matter how creditworthy they are.

That's true for an old-fashioned payday loan store, where, as Date, the former regulator turned industry consultant, says: "The only thing on the menu is a two-week loan or a four-week loan. And you can have it at any price you deserve, just as long as it is the absolute maximum permitted by state law."

It's also true at the handful of banks that offer deposit advances, which bear a strong resemblance to payday loans. For example, a 35-day deposit advance of $300 from U.S. Bancorp (USB) costs $30, no matter who the customer is.

And it's true at high-tech companies such as ZestFinance, which are investing heavily in efforts to understand which borrowers are most likely to repay.

Credit cards, of course, do offer variable pricing. But even outside of the segment of consumers who qualify for plastic, some customers are significantly more likely to repay their loans than others, experts say.

"Surely we should be able to find and attract the more creditworthy of those who are operating in the alternative market," says the Urban Institute's Mills.

One-size-fits-all pricing has perverse consequences. Consumers with a higher likelihood of repayment are essentially dissuaded from seeking small-dollar loans because they aren't getting rewarded for better behavior. That drives up aggregate default rates and, in turn, puts upward pressure on pricing.

Today certain companies, including LendUp and Emerge Financial Wellness, are trying to address the pricing problem. The idea is that customers who establish a solid track record of repayment will eventually be able to graduate into cheaper loans. But these firms have yet to make a significant dent in the marketplace.

Banks' Fear: Cannibalization

Five years ago, the FDIC conducted an experiment aimed at encouraging banks to make affordable small-dollar loans to consumers.

As long as participating banks agreed to set annual percentage rates at 36% or less, the regulators agreed to give them more leeway on underwriting. The banks also received credit for the loans under the Community Reinvestment Act.

By most accounts, the project was a dud. Thirty-one banks made a total of 34,400 loans. That's about two loans for every 10,000 that payday lenders make each year. "We got some mileage out of that, but it wasn't as much as I had hoped for," acknowledges Bair, who was the FDIC's chairman at the time.

Banks have some big advantages over payday lenders — a lower cost of funds, lower operating costs, and a large existing customer base. So why did the program fall flat?

One obvious reason is that banks have a strong financial incentive not to make small-dollar loans, since those loans would likely cut into their overdraft revenue, as Bair herself wrote in a 2005 paper, before she went to the FDIC.

"Banks and credit unions benefitting from overdraft fee income may not want to cannibalize this income by offering their customers lower-cost, small-dollar credit options," Bair wrote at that time. Overdraft fees generated total revenue of $32 billion for banks and credit unions in 2012, according to an estimate by Moebs Research. And though regulators treat overdraft fees differently than small-dollar loans, they function similarly from the consumer's perspective, and are often even more expensive than payday loans. Both products offer ready cash to meet immediate spending needs.

In a report issued Tuesday, the Center for Responsible Lending recommends that banks be required to disclose the cost of overdraft fees as an annual percentage rate, consistent with requirements for loan products.

"There has got to be a recognition that all of these products are economically equivalent," Bair says.

Finders' Fees

Much as Amazon has driven brick-and-mortar bookstores to the brink of extinction, Internet-based lenders hold some key advantages over payday loan stores. They don't have to rent retail space. They need far fewer employees. Their lending processes are more automated.

And yet, an online payday loan is usually much more expensive than one from a payday store. The average cost of an online loan is about 73% higher than a storefront loan, according to a 2012 report by the Pew Charitable Trusts.

"Enormous, astronomical APRs," is how Eric Wright, a staff attorney at the Maine Bureau of Consumer Credit Protection, characterized Internet payday rates in recent congressional testimony. "Eighteen hundred percent, in some cases."

Why are prices so high online? One key contributor is the large cost of acquiring each customer.

Internet lenders often rely on lead generation companies to bring in new business. These middlemen use online ads and direct marketing to sign up potential customers. They may require the prospective borrower to provide personal financial information, including bank account numbers. And then they sell the customer's information to the highest bidder.

"It's very difficult to acquire that customer for less than $200," says Ken Rees, chief executive of ThinkFinance, an online lender that does not use lead generators.

Complaints about lead generators have generally focused on the lack of privacy safeguards for sensitive information provided by consumers as well as the potential for fraud. But given the high prices that lenders are paying for that data, it's also not hard to see how these companies might be driving up the cost of credit.

If the lender's up-front acquisition cost is $200, and the borrower pays $50 for a two-week, $300 loan, which translates to 435% APR, each loan would have to roll over an average of four times just for the lender to cover the lead generation fee. And that's assuming a zero default rate.

"This is adding yet another layer of cost," says Tom Feltner, director of financial services at the Consumer Federation of America.

A Regulatory Thicket

A small-dollar consumer loan is generally a relatively simple product. The same cannot be said about the rules that govern its availability.

The complex stew of regulations in this area has not only become increasingly ineffective at controlling the prices paid by consumers, it is also burdening potential new entrants to the market.

Each of the 50 states, of course, has different consumer credit laws and licensing requirements. Twenty-eight states allow payday lending without major restrictions, another eight allow it but impose requirements that cut into its profitability, and the 14 remaining states don't allow payday lending, according to last year's Pew report.

For startup nonbanks trying to compete against the industry's incumbents, the patchwork of regulations serves as a barrier to entry. LendUp and FairLoan Financial, two new entrants that offer much lower interest rates than payday lenders, are currently licensed in only one state.

Perhaps the most successful new entrant into the small-dollar loan market is Progreso Financiero, which makes installment loans with annual percentage rates of 25% to 28% to Latino customers. This well-financed company has close links to Wal-Mart Stores, including an equity investment by the Walton family's venture capital firm, according to a recent Bloomberg story. Still, Progreso is licensed to operate in only three states — California, Texas and Illinois.

But there are also opportunities for regulatory arbitrage. Some Indian tribes claiming sovereign immunity from state laws offer online loans to customers across the country. Other lenders have set up shop offshore or in states with lax laws and virtually dared other states to come and get them.

Wright, the Maine regulator, all but acknowledged that his state is powerless to take action against out-of-state online lenders. "The powers that we have such as licensing and so on, while we would like to think them to be real, are largely ignored by these companies," he said in recent congressional testimony.

On top of the state laws are federal rules governing short-term consumer credit offered by banks. Federal regulators recently proposed new restrictions on the ability of banks to offer so-called deposit advances, which have many similar characteristics to payday loans. Six banks, including Wells Fargo (WFC), currently offer the product.

Finally, there are bank overdraft fees, which federal regulators do not consider to be small-dollar loans, even though the two products are functionally equivalent.

Without more uniform rules, it will be hard to encourage price competition among lenders, according to analysts. "There has to be a consistent set of rules," Bair says.

The CFPB, which has jurisdiction over both banks and nonbank lenders, appears to hold a similar view.

The consumer agency recently released a report on payday loans and bank deposit advances, and a CFPB official told a congressional panel last week that an upcoming report on overdraft fees is also part of the agency's overall examination of small-dollar consumer credit.

"A competitive, fair marketplace can't work if not everyone is governed by the same set of rules," David Silberman, an associate director at the CFPB, testified. "So we would attempt to look holistically at all products to make sure we're not squeezing air out of one part of the balloon to another."

Previously in this series:

Part 1: Downfall of Subprime Cards Spawns Opportunity

Part 2: Employers Hold Key in Efforts to Undercut Payday Lending

Part 3: Using Big Data to Reduce Risk in Small-Dollar Lending