How well do consumers understand arbitration clauses? A recent op-ed by Jeff Sovern argued that most Americans fail to grasp them, using the results of a St. John's University School of Law survey to support the conclusion that arbitration clauses are "tricks and traps" that "enable wrongdoers to avoid taking responsibility for their actions." He goes on to urge the Consumer Financial Protection Bureau as well as Congress to ban the use of pre-dispute arbitration clauses in consumer contracts.

In our view, readers should consider both the survey and Sovern's conclusion with a healthy dose of skepticism. We took issue with several aspects of the survey's methodology on our blog,

The authors of the survey did not interview consumers who actually had participated in arbitration, whom we believe would likely have positive comments about their experience. The survey ignored well-established contract law with respect to the understanding of a signatory: parties are generally responsible for contracts they enter into unless they were tricked into entering into the contract.

Moreover, the survey did not inquire into consumer understanding of litigation and class actions as compared to individual arbitration. This is a particularly critical point since studying arbitration in a vacuum does not shed nearly as much light on the topic as comparing the relative benefits to the consumer of individual arbitration versus court litigation and membership in a putative class.

Some questions asked by the survey authors contained non-neutral terms that could influence results, such as asking whether a court might "throw out" an arbitration clause. And questions suggestively focused only on arbitration clauses and none of the other provisions of the hypothetical contract. It would be helpful to know if the participants understood more or less about arbitration than other terms of the contract, so that the arbitration results could be evaluated in context.

Other academic studies have shown that arbitration actually does benefit consumers. For example, a 2009 Northwestern University School of Law study of 301 American Arbitration Association consumer arbitrations determined that consumers won relief in 53.3% of the cases examined and recovered an average of $19,255 (52% of the amount claimed). In addition, arbitrators awarded attorneys' fees to prevailing consumers in 63% of cases in which the consumer sought such an award, with an average award of $14,574. Clearly, arbitration is not a way for a company to exculpate itself from liability to consumers.

The Northwestern University study also concluded, with respect to the arbitrations studied, that the upfront cost of arbitration for consumer claimants was quite low: an average of $96 for claims of less than $10,000 and $219 for claims between $10,000 and $75,000. The study further found that arbitration is an expeditious way to resolve disputes, taking an average of seven months. Compare that to the $400 federal court filing fee for commencing a new case, and to the years that many, if not most, cases linger in the court system.

Unfortunately, the St. John's survey avoided the critical issue of asking whether consumers benefit from arbitration as compared with court and class action litigation. "We will not attempt to resolve the debate over the comparative advantages of arbitration and litigation in this article," the authors write. Therefore neither the CFPB nor Congress should rely on its results. Is it a "trick" for consumers to be able to resolve their disputes with companies more economically, promptly and efficiently than they could in court? Is it a "trap" for a prevailing consumer to receive a substantial financial award, as opposed to a $5 check or a coupon towards a future purchase, while attorneys for the class obtain millions in fees?

Any meaningful analysis of consumer arbitration requires a far broader scope than the St. John's survey. It must consider the relative benefits of arbitration as compared with court and class action litigation and other dispute resolution options available to consumers. Hopefully the CFPB's long-awaited study of consumer arbitration, which it expects to complete by early 2015, will take a serious look at how consumers actually fare in arbitration. That, to us, is the most compelling and meaningful data point.

Alan Kaplinsky, Mark Levin and Daniel McKenna are partners in the Consumer Financial Services Group at the law firm of Ballard Spahr LLP. Their clients include financial institutions.