Making bank complaint data more useful

Banking is a data-intensive industry, perhaps more so than any other. This is not only because banking is a numbers business, but also because it’s one of the most heavily regulated industries, with countless reports and statements.

Banking data, however, is a two-edged sword. It can be very useful and helpful to consumers, investors and regulators on one hand, but if misused or improperly analyzed, it can be damaging not only to banks but also to the broader public.

My mentor explained the pros and cons of banking information by creating a two-by-two information matrix in one of his banking lectures at the Wharton School decades ago. On one axis he sorted banking information as either likely to cause a bank run or not, and on the other axis as either relevant or irrelevant.

Irrelevant information may appear to be harmless on its face, but it can be misinterpreted or improperly analyzed and immediately spread via newscasters and social media. The greatest public policy danger is irrelevant information that can cause a bank run.

There are other less extreme problems that can result from the disclosure of irrelevant information — for example, a bank may be accused of racial discrimination based on disparities in reported HMDA data, yet a proper analysis including FICO and other more relevant underwriting data may reach the opposite conclusion. By the time that analysis is done, however, the reputational damage has likely been done.

The current debate over the disclosure of complaint data by the CFPB is befuddled by similar problems, since the data may be improperly reported and analyzed, thereby causing unnecessary reputational damage. The most common problem in this regard is when the CFPB itself or consumer and other groups simply report the absolute number of complaints without properly adjusting them for a bank’s customer base in a specific geography.

Virtually every well-publicized “bank complaint analysis” results in the same “name and shame” conclusion that the most complained-about banks are the biggest banks, namely JPMorgan Chase, Bank of America, Wells Fargo and Citibank. But that is a foregone conclusion, since these banks have the most customers and usually the most complaints simply because of their size. This scale issue exists with almost any company that is a dominant firm in an industry.

As a longtime Florida resident, I have been collecting and analyzing complaint data for both banks and credit unions from the Florida Department of Financial Institutions since 2008, and I believe this work can serve as a guide for others looking to properly analyze the CFPB’s complaint database. My last “Florida Bank Complaint Analysis” was for the year 2015, because that department ceased reporting comparable data after that.

The most important part of this analysis was the creation of the “Bank Complaint Index,” or BCI, which is defined as a bank’s relative portion of all complaints in a given year divided by its deposit market share as of the midpoint of the year for a specific geographic area. Midyear deposit data by geographic branch location is readily available from the FDIC, while comparable credit union share data must be estimated based on their size and number and type of branches.

Critically, this analysis helps to control for market size in a way that simply reporting the absolute number of complaints for a specific institution does not.

The results of the Florida Bank Complaint Analysis were published every year and reported in most major Florida dailies and even in national credit union publications. Banks and credit unions with elevated BCIs — those doing an average or worse job, as measured by market-weighted complaints — apparently took notice and worked very hard to lower them, and those with very low BCIs or no complaints did the same to keep their outstanding records.

The clear winners from these analyses were existing and prospective bank customers who could use these data to make more informed banking decisions — based on all the relevant data.

While it is much easier to simply list the number of complaints by bank, as is most commonly the case, such irrelevant information may lead to improper and potentially reputation-damaging conclusions.

I support the continued collection and publication of bank complaint data by the CFPB and similar sources that confirm the legitimacy of complaints and the existence of a business relationship. That raw complaint data, however, is only as good as the organization or analyst reporting them. Including an analysis along the lines of the “Bank Complaint Index” could go a long way to providing more relevant data.