Comment: Lending Fairly? Mystery Shopping Will Clue You In

During a recent speaking engagement, a bank president asked me why his institution should consider self-testing for fair lending. The president said he knew the bank did not discriminate, and that he didn't believe in managing his bank to satisfy every regulator's whim. My response to this president was simple: Self-testing is unnecessary if you know your bank is in full compliance. But how do you know?

You know that your institution has a variety of loan products designed to penetrate your entire community, but how do you know if your officers are offering the products to qualified applicants? You changed your underwriting standards to accept as credit histories such nontraditional items as utility payments, but how do you know if your lenders are requesting the bills? Your entire loan staff completed diversity training, but do you know if the lender in your branch office really understood it?

Knowing what your staff is doing is the important factor in fair lending. Developing objective standards, modifying antiquated policies, offering creative products, and training your staff are all important aspects to a strong fair-lending program. But these efforts do not test what matters: fair-lending performance. Testing is the key to really knowing.

Two methods of testing

Federal regulatory agencies agree that testing is an important component of a strong fair-lending program. However, the agencies have not prescribed a chosen method.

Two methods of testing exist in the marketplace: post-application mystery shopping and match-paired testing. More and more financial institutions are choosing post-application mystery shopping, as opposed to match-paired testing, as the method to effectively test their fair lending performance.

In a match-paired testing, an outside consulting firm hires professional testers to pose as applicants for credit. The firm sets up a pair of minority testers and a pair of white testers with similar credit histories and applications. The testers compare the level of service given each set of applicants to detect disparate treatment.

The results of match-paired testing can reveal much about a financial institution's lending. However, institutions have found that match-paired testing has its pitfalls. The largest is cost. Structuring a match-paired test involves coordination of many variables and is ultimately a high-cost project for a relatively small output of analyses. Using fake testers also may cause an entrapment issue, resulting in a decrease in employee morale. Further, statistical significance is nearly impossible to achieve economically.

Finally, the number of tests seldom covers all loan officers. Thus, the tests often yield more questions than answers.

Post-application mystery shopping uses actual customer experiences to provide insight into lender performance. The process involves contacting customers who have applied for a loan and asking them about their perceptions of the transaction. Applicants are asked a series of questions and their answers are scored by professional phone shoppers. Scores are entered into a data base where the questions are merged with application information, including race and sex.

In post-application mystery shopping professional phone shoppers don't know that they are testing for potential disparate treatment. Phone shoppers don't have access to customer records other than the date of application and name of customer, and only ask basic service-related questions.

Three-phase project

Post-application mystery shopping is generally a three-phase project involving questionnaire design, shopping, and analysis.

Questions that are posed in a post-application mystery shopping project involve the applicant's perceptions of the entire lending process. Questions may attempt to discover the amount of coaching the loan officer provided, the level of information provided on different products, and the level of service provided in the application stage.

The key is designing questions that will result in valid data. The applicant must be asked questions that are not leading, are relevant to your institution's practices, and provide responses that are measurable.

Three considerations in performing mystery shops are the method of collecting data, competency of interviewers, and sampling techniques.

You generally have two methods of collecting the information on your questionnaire, either through phone interviews or mail surveys. Phone interviews are the most recommended and effective because they provide higher response rates in a controlled environment. However, mailed surveys have one principal benefit over phone interviews: cost.

If you opt for phone interviews, the key is to engage professional interviewers. Professional phone interviewers are aware that they can skew data by the way they phrase questions. As a result, a professional tester will take steps to ensure that applicants are not prompted for a specific response. Professional interviewers also realize that when they call your applicants, they represent your institution and must do so with integrity.

A simplified analysis

Samples are pulled from your normal application base. When drawing a sample it is important to ensure that it is fully representative of your applicants and that you have enough minority applicants to perform analysis. It is also important that certain sample data - the applicant's race, sex, and income - not be provided to the interviewers.

The shopping process also involves analyzing questionnaire data with respect to the applicant's race, sex, and age. An example of a simplified analysis is presented here:

Question: On a scale of 1 to 5 - 1 being the lowest and 5 the highest - how would you rate the amount of information the lender provided you on the home loan process?

Lender A A's branch Institution

Overall 3.75 3.35 4.10

Minority applicants 2.80 3.50 4.12

White applicants 4.50 3.20 4.09

An institution's response to this data would reveal that:

Lender A has a disparity in the perceptions of minority applicants versus the perception of white applicants. Lender A's branch appears to be satisfying minority applicants at a higher level than applicants. However, the branch's overall satisfaction numbers are low compared to those of the entire institution.

The financial institution overall appears to have no disparity in applicants' perception of this question.

As a result, the institution may decide to require Lender A to attend an additional diversity training session and require employees at Lender A's branch to get further training on providing information on loan processes.

This simplified example shows how mystery shopping is used as a management tool. However, once the shopping is completed, a financial institution is not done with the data. Information from past shopping is used to determine if training was effective and if improvements have been made.

For example, if Lender A's scores among minority applicants improved, the institution would know that training was effective. If the branch continued to have low scores, the institution might want to reevaluate its training program.

A number of uses

Institutions that use mystery shopping quickly realize that the information they collect is not solely for the use of the compliance department. Most importantly, the compliance department needs the program to test for disparate treatment. But branch managers can use the program to assess lender performance, marketing departments can use it to gauge knowledge of products, and senior managers can use it to assess the performance of branch manager .

Post-application mystery shopping becomes a tool for the entire institution.

Mr. Schriner is a bank regulatory consultant with McGladrey & Pullen's national financial institutions consulting group in St. Paul.

For reprint and licensing requests for this article, click here.
MORE FROM AMERICAN BANKER