Financial firms from Charles Schwab to Banco Popular may face tougher negotiations with their mortgage outsourcer, PHH Corp., if that company's dissident shareholders get their way this week.

Pennant Capital Management LLC, which owns 9.94% of PHH, is campaigning to oust its nonexecutive chairman, Alvin B. Krongard, and its chief executive, Terence W. Edwards, from the board at the annual shareholders meeting Wednesday. The hedge fund wants to replace them with its two nominees, including Greg Parseghian, the former chief executive of Freddie Mac.

Since 2006, Pennant has complained that the Mount Laurel, N.J., company has been too slow to reduce mortgage production costs during the housing downturn and that it spends too much in order to keep its private-label customers happy.

Industry experts said the proxy fight underscores the difficulty of mortgage outsourcing, particularly during a housing downturn. Customers demand specialized service, but the business depends on economies of scale.

"Every client wants some sort of customization, which can eat into profits," said Craig Focardi, a senior research director in the consumer lending unit at TowerGroup Inc., an independent research firm owned by MasterCard Inc.

Neither PHH nor Pennant returned calls for comment.

Pennant began meeting with PHH executives early last year. The hedge fund said in a proxy filing last month that during those early discussions the management team "conveyed to Pennant its suspicion that some clients were insufficiently profitable across the business cycle.

"Without understanding profitability, how can PHH pursue profitable growth and evaluate existing clients and cost-cutting efforts?" Pennant said.

PHH countered that keeping customers happy is crucial to long-term growth.

"If we take care of our clients in … the mortgage business … they are our best salespeople," Edwards said in a presentation to investors last week. "So by taking care of our clients today as we go through the ebbs and flows of business, they help us sign new accounts."

PHH also likes to point out that it has outlived 17 other mortgage competitors, which augurs well for profitability in a less-crowded field.

"Because people still have to make money in the mortgage business, because all of the bad competitors have disappeared, you're left to the three large banks and then we're the fourth-largest retail player right behind those three large banks," Edwards said. "They price appropriately — everyone wins."

In August, Pennant proposed to PHH that Parseghian get a board seat that was about to become available. The management team rejected this idea, and since then things have turned contentious.

PHH has pointed out in proxy filings that Parseghian was ousted from Freddie in the fallout from its 2003 accounting scandal. PHH has argued in proxy filings that his presence would hurt the company's ability to land third-party mortgage clients and to gain regulatory approval for its application to buy a thrift. (The application, submitted in November, is pending.)

Pennant in a proxy filing has called PHH's criticisms of Parseghian "character assassination" and noted that he "was never charged with improper activities and never even received notice that he was a target of an investigation." His mortgage expertise would help PHH, the hedge fund said, particularly when it comes to managing servicing rights, which have produced losses in recent quarters.

Aside from Parseghian, Pennant has nominated Allan Loren, the former chairman and CEO of Dun & Bradstreet Corp., to the seven-member board. The hedge fund has cited Loren's experience in running a public company. Though PHH has said Loren lacks experience in mortgages or fleet management (its other business line), Pennant said the same criticism "could have been leveled against any of the board's nonmanagement directors as well," including Krongard.

Krongard, a former executive director of the Central Intelligence Agency, is no stranger to financial services, however. He is the former chairman and CEO of the investment bank Alex. Brown & Sons, which is now a part of Deutsche Bank. He joined PHH as the chairman in 2005.

The last few years have been rough for PHH. The company lost its biggest mortgage client last year when Bank of America Corp. bought Merrill Lynch & Co., which accounted for 21% of PHH's 2008 loan production, according to its annual report.

Charles Schwab Corp. now is PHH's largest client, with 16% of its loan production last year. Other clients include Allstate Corp., BankAtlantic Bancorp Inc., Comerica Inc., Northern Trust Corp. and Popular Inc.'s Banco Popular. (Schwab and Northern Trust declined to comment; the others did not return calls.)

"The reason people sign up to do business with PHH is, they don't want to incur the capital expense of starting up a mortgage company from scratch," said Terry Wakefield, the chief executive of Wakefield Co., a Grafton, Wis., mortgage consulting firm.

Wakefield, who founded the first mortgage outsourcing firm, Private Label Mortgage Services Corp., in 1988 (it is now a part of Wells Fargo & Co.), nevertheless called the business model "tenuous."

"If you're tailoring your offering to every customer, it would be very hard to make money," he said. "The whole strategic notion of third-party outsourcing is to get a single platform producing as much volume as possible."

Mortgage outsourcers need to have labor costs that are 40% to 60% below the industry average to be profitable, he said.

"PHH is a very people-dependent enterprise," said Wakefield. "The only way to manage the ups and downs of the business is by hiring and firing people, which is a 30-year-old model."

Though PHH's mortgage production unit posted a first-quarter profit of $113 million on $9 billion of origination volume — the unit's first profit in 11 quarters — the mortgage servicing unit lost $118 million due to prepayments and paper losses on the value of servicing rights.

Paul Miller, a managing director at Friedman, Billings, Ramsey & Co. Inc., wrote in a report last month of the "considerable uncertainty in the long-term earnings power" of PHH's core businesses.

"The continued deterioration in housing and credit markets is pressuring PHH's business model," Miller wrote.

"PHH has lost money in four of the last eight quarters. … It needs to prove to the Street that its business model can deliver a consistent and favorable return for investors, which is not clear, given recent results."

Yet PHH has won an important battle in the proxy contest.

Three proxy advisory firms — RiskMetrics Group, Glass Lewis & Co., and Proxy Governance Inc. — have recommended that shareholders vote for the incumbent directors.

In a 34-page report on the proxy fight, RiskMetrics cited PHH's success in weathering the mortgage crisis and "the absence of prevailing evidence that change is needed."

But the firm made its own criticism of PHH.

"The company actually took credit for being more sound than Countrywide, IndyMac and Washington Mutual, as though avoiding bankruptcy or a forced fire sale was a mark of success for PHH," RiskMetrics wrote. "The company's poor communications strategy and execution has hurt stockholders in a number of ways, including … by making it harder for PHH to recruit and retain outsourcing clients."

Subscribe Now

Access to authoritative analysis and perspective and our data-driven report series.

14-Day Free Trial

No credit card required. Complete access to articles, breaking news and industry data.