- Key insight: Companies that use AI to screen job candidates should keep an eye on a class-action suit against Workday.
- What's at stake: Banks are heavy users of this technology. They may have to vet HR software more carefully or implement more human review.
- Forward look: The case is heading into legal discovery.
A class-action lawsuit against human resources software company Workday has survived a series of motions to dismiss and is proceeding to the discovery phase in California's northern district federal court. Its outcome will affect banks that use Workday's AI screening tools and others like it.
An estimated 80% of U.S. employers use AI software to filter job applicants, including
The use of AI in job screening has escalated because job seekers increasingly use automated tools including AI to apply for jobs, which has caused the number of applications per job posting to explode into the thousands, according to Dina Taylor, chief evangelist at HireVue. "No human has the capability to read all of those resumes and make informed decisions based on them," she said. "It's literally impossible."
Derek Mobley is an African-American man over 40 who suffers from depression and anxiety, according to the complaint. He has a bachelor's degree in finance from Morehouse College and an associate's degree in network systems administration from ITT Technical Institute. Since 2018, he has applied for at least 100 positions at companies that use Workday's software. He's been rejected every time, sometimes within minutes in the middle of the night, though Mobley says he was qualified for all these jobs.
Workday's AI screening tools "rely on algorithms and inputs created by humans who often have built-in motivations, conscious and unconscious, to discriminate," the complaint states. The company "unlawfully offers an algorithm-based applicant screening system that determines whether an employer should accept or reject an application for employment based on the individual's race, age and or disability."
Workday says the suit's claims are false.
"Workday's AI recruiting tools don't make hiring decisions and our tools are designed with human oversight at their core," Connor Spielmaker, principal, corporate communications at Workday, told American Banker. "Our technology looks only at job qualifications, not protected traits like race, age, or disability. We rigorously test our products as part of our responsible AI program to confirm our tools do not harm protected groups."
A Bank of America spokeswoman said the company does not use AI screeners in its hiring process. U.S. Bank and PNC declined a request for comment. Keybank, Wells Fargo and Fifth Third did not respond to a request for comment.
In a somewhat similar case,
What the Workday lawsuit alleges
The Mobley complaint does not claim that Workday itself is discriminating. It says Workday gives employers tools that they can configure in ways that have a negative effect on job applicants who belong to protected classes.
"What's being alleged is disparate impact, meaning that it's a neutral policy on its face, but it still results in an adverse impact on a protected group," attorney Robert B. Hinckley Jr., managing partner of Buchalter, told American Banker. Hinckley and his firm have no connection to the case. "It examines the consequences in use of the algorithm rather than the intent of the tool."
AI screening tools are typically designed to not consider race, age or disability. However, some models have been known to be affected by proxy indicators — data that correlate strongly with a protected characteristic without naming it directly. A candidate with large gaps in employment, for instance, could be someone with a disability, a person who has been in prison or a woman who has taken time away from work to raise children.
"On its face, [not hiring someone with a large employment gap] is a neutral policy, but what ends up happening is you may also have screened out people who had cancer or took time to take care of a family member," Hinckley said. "You end up screening out people that have abilities that have precluded them from being in the workforce."
Proxies for age discrimination include terms like "energetic," "eager" and "excited" — "these are all code words for 'young,' and so if the AI tool is not properly vetted, it might also interpret that as you are looking for someone young, and it may have an implicit bias," Karina Sterman, partner at Greenberg Glusker, which is unconnected with this case, told American Banker. "Even without that, by virtue of describing this as 'high energy,' 'really fast-paced,' 'requires-a-lot-of-stamina' work, [an AI model] might assume that you want somebody younger to comply with these kinds of criteria, and it might automatically dump somebody who, based on their resume, is more than 10 years out of college."
AI is like having a third person in the room, Sterman said. "There's the applicant, there's the employer that's describing what they're looking for in an ideal candidate, and then the AI tool is adding its own thought process, is reading between the lines, and that does change the potential exposure for the employer, because the AI tool has no morality. It's not thinking of, is this good or bad, or morally right or wrong, or even ethical or unethical. It's just getting to the efficiency that it thinks the employer wants. So it will make assumptions. If the job description inadvertently says 'looking for an energetic' whatever, they will take the word energetic and assume it means young. AI assumes it understands you and runs with it, long before you realize that it's completely misinterpreted."
The plaintiff's attorneys are likely to look for flaws in Workday's algorithms that allow such outcomes.
"I think there's going to be a deep dive and discovery as to the Workday products," Hinckley said. Specifically, Workday offers a candidate skills match tool and an assessment connector that will both face close scrutiny, he said.
The written discovery process is starting in this case, and depositions will likely follow, Hinckley said.
Takeaways for bankers
The important thing when using AI to screen job candidates is to "keep the candidate experience in mind, leveraging ethical, transparent AI, and focus on skills validation as much as possible," Taylor said. "Nothing is ever perfect at any point in time, humans aren't either, but the technology can be a really wonderful thing if it's used properly and in the right way and with the right intentions."
But the AI needs to be ethical and can't be a black box, Taylor said. "The science that is used needs to be validated and backed and predictive," she said.
Banks that use AI for application screening need to look closely at the process, just not the final shortlist that the human decision-maker sees, Hinckley said.
"You have to be intentional about the programs you're using, and look at each automated step in the AI process to make sure it's not leading to a disparate impact result unintentionally," he said.
Another issue is the vendor contract, according to Hinckley. Some vendors cap their liability at a low figure like $10,000. In such cases, "you've now lost the ability to offset your liability with the vendor."
Some vendors restrict audits, while others do not warrant regulatory compliance, he said. Sometimes companies can defer liability for a software program's unwanted outcome to the vendor: "We bought this product, we're using this product, it's not our fault that it doesn't function right," Hinckley said.
This case also tests a novel legal theory that other courts will be watching, that a vendor can serve as an agent and stand in the shoes of the employer, Hinckley said. This may not hold up.
"You can't take your hand off the wheel and replace it with AI," Hinckley said. He noted that for large banks, this can be difficult, as they receive tens of thousands of applications a month.
"You've got to have a strategy that allows for the inclusion of AI processes, but allows for human final decision-making to remain," he said.
