If your organization is like most banks and credit unions, you probably use data from an outside data vendor in your back-end collection departments to locate customers. You may need to find out where a customer has moved, or the best phone number for contacting a customer—or you may need to determine compliance red flags, such as bankruptcy or a deceased customer. But how can you know that you’re getting the best data from an outside vendor? Are you comparing data vendors in a head-to-head test? Are you doing the testing right?
When choosing a data provider, the first rule of thumb is to realize not all data providers are created equal. During the evaluation process, it is important to peel back the layers on the prospective partner’s processes, technology and databases in order to choose the vendor that best meets your individual needs.
Cheaper isn’t always better. Prospective vendors are always knocking on your door, claiming they can offer better quality data, lower prices and/or faster results. But beware, many vendors will make claims that seem attractive at first, but have hidden costs—or they can’t deliver on their promises. The proof will be in the results – that’s why testing is important.
Setting up your test
Your testing should match real-world scenarios, and should align closely with real processes that transpire in your day-to-day world. For example, if you are testing bankruptcy, and know that every day, you will be sending in a file and expecting data back, and that you will continue to monitor that file for updates, then you should test that exact process over a 30-day period to make sure that not only the data meets your needs, but also that the process is something that works for you and your IT department.
Another very important component of testing is ensuring that all vendors are given the same input data with the same parameters for the test. Each should get the same input file, on the same day at the same time. The conditions of the test should be equal and consistent as far as matching expectations, filtering expectations and output file fields. If the various vendors set up your test differently, you will get different results, and it will be difficult to compare apples to oranges.
Additionally, consider the size of your test file. While it may be nice to have a large number of results to compare, remember, you need to actually look at all of the results from each vendor in order to find out if the data returned is correct or not. However, too small of a file may not result in enough hits for you to test. A good file size to test is 500 to 1000 records. You can’t just run the raw numbers and say that vendor 1 returned 30 hits, and vendor 2 returned 20 hits so vendor 1 is better. You could be seeing false positive hits that will skew the results and ultimately make unnecessary work for your company.
Here is a checklist for best practices in batch testing:
• Provide the same file of 500 to 1000 records to each vendor at the same day/time
• Require all vendors to return the file back to you at the same day/time
• Require all vendors to return the same information back in the file (the same fields)
• Set up all vendors with the same testing requirements as far as the product being tested, your expectations for a return date/fields, your matching expectations (for example, will you accept just a SSN match on a bankruptcy hit?)
• If filters are being requested, make sure all vendors are using the same filters—two examples being date filters (only return a bankruptcy on or after the open date of your account) and matching filters (only return certain confidence levels of matching)
Reviewing your results
Depending on what you are testing, you may need to involve several people in your organization to review the test results, and confirm whether or not they are correct. Below are some best practices based on the most common data sets tested:
Phone Data: You will need to call the numbers provided to determine if the contact is correct. All numbers should be called within the same time frame. Remember, a telephone disconnection can happen overnight. Look for the following stats:
• Was the consumer reached via the phone number provided?
• Was a 3rd party who knows the debtor reached via the phone number provided?
• Was the number disconnected?
• Was it an incorrect number?
• Was it a possible correct number (e.g., got a voicemail, no answer, busy signal)?
If you have requested multiple numbers for each consumer during the test, make sure to call ALL numbers provided—don’t stop after the first number, even if you reached the consumer. Remember you are testing the validity of all numbers received.
Bankruptcy Data: The best source for testing the results of your bankruptcy data is PACER, but note that it will cost you to use the PACER service ($0.10 per page for bankruptcies viewed, but no cost if your search comes up with no hits). You don’t want to call the consumer to verify bankruptcy, because that constitutes a violation of the automatic stay. You can call the attorney for the consumer and verify certain criteria with them, but PACER is the fastest way to identify correct and incorrect results.
Deceased Data: Since there is no one online source available to confirm a death, you may need to use several sources, calling the last known phone number of the consumer, calling relatives, or looking online for obituaries.
Scores: If you are testing a scoring product, it may be more difficult to determine results based on a numeric score. The best way to determine scoring results is to look at the scoring bands overall to see if the various vendors you’re testing grouped your results similarly (e.g., 500 in the top band, 200 in the middle band, 300 in the lowest band, etc.).
Partnering with a data vendor can help you collect on outstanding accounts more quickly and cost-effectively than performing these functions on your own. To repeat: Not all data vendors are created equal. Different batch testing parameters like time frames, definition and instructions can lead to inconsistent processes and skewed results. Walk through all of your expectations with your potential vendors prior to the test. Make sure you have the time and staff to review your test results before you enter into the test. Proper testing can help you to find the right data vendor fit for your company, and as a result you should see increased collections, stronger compliance and a better bottom line.
