The Heartache Of Bad Data

Print
Email
Reprints
Comment
Twitter
LinkedIn
Facebook
Google+

Has this ever happened to you? It happened to a friend of mine. You open a performance report, one that's been sent to your boss and all the top executives in your company, and it states that your group is failing to meet company mandated quotas and therefore its performance is poor. Your heart sinks and you think, I don't need this.

Then, knowing how hard you and your team have been working in this particular area, you take a closer look at the numbers and realize that all the data points in the report are wrong. In some instances, the real-life numbers are multiples of what's stated in the report. The creators of the report are not willing to make any changes to their data sourcing or methodology. You understand the need for quotas and the drive to "move the needle," but the inaccuracy and apparent unfairness of the reports leave you feeling helpless and betrayed.

It's a side effect of data-driven decisionmaking. Data-based decisions sound so good, so democratic and fair. But much depends on what is being measured and how, and the soundness of the underlying data. Without solid and well identified data sources, numbers can easily be miscounted, misallocated and misattributed.

And methodologies are often flawed. In a recent New York Times article, Steve Lohr wrote, "The problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according to the laws of physics. In so many Big Data applications, a math model attaches a crisp number to human behavior, interests and preferences." The article quotes Claudia Perlich, chief scientist at online ad company Media6Degrees, saying, "You can fool yourself with data like you can't with anything else. I fear a Big Data bubble." That bubble, she says, would involve a rush of people calling themselves "data scientists," doing poor work and giving the field a bad name.

Four efforts would improve data-driven decisions.

1. Vet the performance metrics. Are your key performance indicators the right ones? Say you measure employee performance by number of tweets posted per day. One person might meet the quota with irrelevant or poorly craft tweets that fail to advance the company's goals, or worse, an automated feed. Another might tweet three times a day but create engagement, generate followers and boost the brand.

2. Check the accuracy of the underlying data. If the data are wrong, people will sooner or later begin to notice and ignore the reports. Have an unbiased, independent party conduct a reality check against original sources before you take a report live.

3. Use your intuition. If numbers feel wrong, they probably are and at the very least are worth a closer look.

4. Be the squeaky wheel. Someone has to speak up for the truth, or an ugly precedent will be set. Ideally - and this is the hard part - one does this with discretion and good will.

Without such efforts, false reports become a cultural illness that can breed misunderstandings and poor decisions.

JOIN THE DISCUSSION

SEE MORE IN

Top 10 Tech Companies to Watch
To identify the fintech startups deserving of special recognition this year, a team of American Banker editors, BAI staff and consultant Jeanne Capachin debated the merits of self-nominees and companies we know. These are the ten that matter for 2015.

Image: Fotolia

Comments (0)

Be the first to comment on this post using the section below.

Add Your Comments:
Not Registered?
You must be registered to post a comment. Click here to register.
Already registered? Log in here
Please note you must now log in with your email address and password.
Already a subscriber? Log in here
Please note you must now log in with your email address and password.