The highest court in Massachusetts rejected, in part, a challenge from a Boston homeowner who had contested the validity of her foreclosure, in a widely watched case that threatened to cause a wave of new legal problems for banks seeking to repossess homes.

The ruling avoided a decision that the real-estate industry had feared would cloud titles on thousands of foreclosed properties in the state, even as it created a new documentation requirement for lenders in certain foreclosure cases. The same court issued two rulings last year that reversed foreclosures.

"The real-estate bar up here is breathing a great sigh of relief," said Edward Bloom, a partner at Sherin & Lodgen in Boston and the former president of the state's Real Estate Bar Association. A decision challenging the industry's practice of repossessing homes "would have screwed up real-estate titles for the last 50 years," he said in an interview Friday.

Consumer advocates said that the ruling would help homeowners by ensuring banks provide clearer documentation of loan ownership, and said it opened new avenues for potential challenges if banks have trouble fulfilling new standards laid out by the court. "This opens up a lot of issues that weren't there before," said O. Max Gardner III, a lawyer in Shelby, N.C., who represents borrowers in bankruptcy cases.

At issue was whether lenders needed to possess both the mortgage and the underlying promissory note when foreclosing on delinquent homeowners.

In 2009, a company called Green Tree Servicing LLC, which collected payments on behalf of Fannie Mae, foreclosed on Henrietta Eaton after she stopped making payments on the $145,000 loan that she refinanced in 2007.

Ms. Eaton challenged the foreclosure because Green Tree possessed the mortgage but not the underlying promissory note, making what is sometimes called a "naked" mortgage. A judge halted the foreclosure because Green Tree didn't have possession of both the mortgage and the note.

Friday's decision from the Massachusetts Supreme Judicial Court agreed in part with the lower court, but said that a foreclosing entity didn't need physical possession of the underlying note to foreclose. Foreclosing entities still had to show that they were acting on the owner's behalf, which could hamstring lenders that have difficulty documenting a complete ownership chain.

It would be a "mistake" for banks "to think this is a victory," said Adam Levitin, a law professor at Georgetown University who had submitted a brief in support of Ms. Eaton. That is because foreclosing entities still face challenges if it turns out that loans weren't properly assigned to trusts of pooled mortgages. "You still have to show that someone else is actually the noteholder."

The court said the ruling only applied prospectively, meaning that already-completed foreclosures couldn't be reopened. It also vacated the foreclosure injunction against Fannie Mae, sending Ms. Eaton's case back to the lower court to determine how the foreclosure should proceed.

Banks often separate the mortgage from the note when loans are bundled into securities and sold to other investors. Shortcuts designed by lenders to facilitate that process, called "securitization," have spawned myriad legal challenges now that banks are foreclosing on hundreds of thousands of those mortgages every year. Banks' attorneys have struggled at times to meet state requirements designed to ensure clear title to repossessed homes.

The Eaton case attracted national attention. Consumer advocates said that a ruling supporting Ms. Eaton would ensure due process for homeowners. The Federal Housing Finance Agency, which regulates Fannie Mae, warned that a ruling in favor of the homeowner and applied retroactively represented a "direct threat to orderly operation of the mortgage market."

Subscribe Now

Access to authoritative analysis and perspective and our data-driven report series.

14-Day Free Trial

No credit card required. Complete access to articles, breaking news and industry data.