Mortgage customers continued to pay down their residential loans in the second quarter, a trend that spells trouble for servicing firms dependent on fee income.

Data compiled by National Mortgage News and the Quarterly Data Report showed that servicing balances nationwide fell to $9.77 trillion at June 30, the second consecutive quarterly decline. Servicing balances peaked at $10.13 trillion at Dec. 31. This includes both first and second liens.

Historically, mortgage debt outstanding hardly ever declines, but the recent housing crash and recession have changed the way people spend and save.

Amy Crew Cutts, a deputy chief economist at Freddie Mac, noted that the servicing shrinkage comes not only as some people pay down their balances when they refinance — to achieve a better rate — but also as foreclosures remove defaulted mortgages from the calculations.

"We're seeing a lot of all-cash purchases" of homes, she said. Relying on data from the National Association of Realtors, she said that in some cases 20% to 30% of home purchases are all-cash deals, which do not involve mortgages. Historically, cash deals have accounted for about 10% of home purchases.

Many of the cash purchases occurring today are by investors who buy deeply discounted homes at foreclosure auctions. In some cases properties are selling at 50% to 70% of their prices at the peak of the market. Jay Brinkmann, chief economist of the Mortgage Bankers Association, said that, in states where the market was once red-hot — like California and Florida — it is not unusual to see a house that sold for $1.3 million a few years back sell at foreclosure for less than $500,000.

When it comes to declining mortgage debt, he noted, among the biggest problems is "a lack of meaningful cash-out refis."

For mortgage servicers, lower debt balances translate into reduced earnings on their processing fees, a phenomenon that is not lost on MBA. Second-quarter servicing rankings compiled by National Mortgage News showed that most of the top 10 servicers had anemic or negative growth during the period, with the exception of U.S. Bank Home Mortgage in Bloomington, Minn. The unit of U.S. Bancorp serviced $199 billion of home mortgages at June 30, a 17% gain during the preceding 12 months.

It is an open question when balances will stop shrinking and resume growth. MBA's Brinkmann said household formations and an improvement in the unemployment rate are keys.

"There are people in their 20s and early 30s living in apartments," he said. "They are OK with one kid, but when that second kid comes along, they might start looking for a house."

Subscribe Now

Access to authoritative analysis and perspective and our data-driven report series.

14-Day Free Trial

No credit card required. Complete access to articles, breaking news and industry data.