> This is the markdown version of https://www.classet.ai/blog/adverse-impact-analysis > Learn more at https://www.classet.ai # Adverse Impact Analysis for AI Screening: A Worked Example How to run an adverse impact analysis on an AI screening tool: the four-fifths rule, a worked example by funnel stage, and where the math misleads you. [![Paul Jones](/_next/image?url=https%3A%2F%2Fassets.basehub.com%2Fe0b5701f%2F6599306507912123f90f150a8bfaaf6c%2Fscreenshot-2026-01-28-at-10.53.16-am.png%3Fwidth%3D100%26height%3D100%26quality%3D100&w=96&q=75) Paul JonesHead of Growth at Classet ](/blog/authors/paul-jones) October 19, 2026 AI Recruiting, Guides & Insights ![Low-poly funnel splitting into streams around a balance scale weighing two groups](/_next/image?url=%2Fimages%2Fblog%2Fadverse-impact-analysis.png&w=3840&q=75) At an HR conference last week, one HR leader summed up her legal team's worry about AI screening in a sentence: what if the tool screens out a protected group, and no person ever looks at it? Two other enterprise teams raised the same fear without us asking, and one of them cited the Workday lawsuit. What they wanted was a way to check the tool themselves instead of taking a vendor's word for it. That check is an adverse impact analysis, and the math is simple enough to run in a spreadsheet this afternoon. Quick answer An adverse impact analysis compares how often each demographic group passes a step in your hiring process. Divide each group's pass rate by the rate of the group that passes most often. Under the federal Uniform Guidelines, a ratio below 0.80 (the "four-fifths rule") is generally treated as evidence of adverse impact. For AI screening, run it at every stage the tool touches, not only on final hires, and pair it with a significance test. ## What is adverse impact? **Adverse impact** is the legal term for a selection practice that looks neutral but filters out one group at a much higher rate than another. Intent doesn't matter. A knockout question asked the same way to everyone can still create it. The standard test comes from the [Uniform Guidelines on Employee Selection Procedures](https://www.law.cornell.edu/cfr/text/29/1607.4), which federal agencies adopted in 1978. Section 1607.4(D) says a selection rate for any group that is less than four-fifths of the rate for the highest group is generally regarded as evidence of adverse impact. That 80% line is the **four-fifths rule**. Two details in the same section are easy to miss: - Smaller gaps can still count if they're significant in both statistical and practical terms. - Bigger gaps may not count when they're based on small numbers. Treat the 80% line as a first check, and confirm any gap with a significance test before you act on it. ## How do you run an adverse impact analysis? Run these seven steps on one stage of your funnel. 1. Pick the decision point. For example: "Advanced past the phone screen." 2. Count the people who reached that step, split by group (sex, race and ethnicity, and age 40+ are the usual starting set). 3. Count how many in each group passed. 4. Divide passes by total for each group. That's the **selection rate**. 5. Find the group with the highest selection rate. 6. Divide every other group's rate by that highest rate. That's the **impact ratio**. 7. Flag any ratio below 0.80, then test whether the gap is statistically significant. ### A worked example Say a distribution center receives 1,000 applications for warehouse associate roles in a month. Applicants who pass the basic eligibility questions get a phone screen with five must-have questions. The group labels come from voluntary EEO self-identification in the ATS. Here are the results for that screen, by sex and by age. Group Screened Met all must-haves Selection rate Impact ratio Men 600 330 55% 1.00 Women 400 180 45% 0.82 Under 40 700 399 57% 1.00 40 and over 300 132 44% 0.77 The sex comparison clears the four-fifths line at 0.82, though not by much, while the age comparison fails it at 0.77. With 300 and 700 people in the groups, that age gap is also statistically significant: a standard two-proportion z-test puts it near 3.8, well past the usual cutoff of about 2. Next, take the five must-have questions one at a time and rerun the math on each. In this example, say four questions show no meaningful gap and one does: "Have you used an RF scanner in the last 12 months?" Applicants over 40 were far more likely to answer no, often because they were returning to work after time away. Now ask whether the requirement is job-related and necessary. If new hires learn the scanner in their first shift, it isn't. Drop the question, rerun, and the age ratio moves back above 0.80. The analysis points you to the criterion, and a person on your team decides whether it belongs. If a requirement shows a gap but is necessary for the job (a forklift certification for a forklift role, say), the Uniform Guidelines expect you to show it's job-related and that you looked for an alternative with less impact. Write that reasoning down when you set the criterion, not after someone asks. ## Which hiring stages should you test? Most teams run the analysis once, on hires. For AI screening, that misses the stages where the tool acts. Break the funnel into steps and run the table at each one: Stage Question to ask Invited to screen → completed screen Do some groups answer the call or finish the screen less often? Completed screen → met must-haves Is one criterion driving a gap? Met must-haves → advanced by a recruiter Are people treating the AI output differently by group? Advanced → hired Is the gap appearing in interviews instead? The first row is easy to overlook. If one group completes the screen less often (because of call times, language, or how the invite is worded), the tool can produce adverse impact without asking a single biased question. Offering the screen in the candidate's language and leaving the line open around the clock both help here, and both are measurable. The third row is the human part of the process, and it deserves the same scrutiny. A clean AI stage followed by a skewed recruiter stage is still a skewed funnel. ## When does the four-fifths rule mislead you? The 80% line is a rule of thumb, and it fails in two predictable ways. Small numbers make it too sensitive. Take a single store where 12 applicants over 40 and 30 under 40 went through the screen. If 3 of the 12 pass (25%) and 12 of the 30 pass (40%), the ratio is 0.63. It looks alarming, but a single applicant moving the other way would change the picture. The Uniform Guidelines let you combine data over a longer period or across similar locations when samples are this small. Do that before you act on one store's numbers. Big numbers make it too lenient. At 50,000 applicants, a ratio of 0.85 can be highly significant, so passing the four-fifths rule doesn't mean you're clear. Run a significance test alongside the ratio at high volume. The safe habit is to report both numbers every time: the impact ratio and whether the gap is significant. ## Where does the demographic data come from? You can't run the analysis without group labels, and you shouldn't collect them inside the screen itself. For most employers, the source is voluntary EEO self-identification on the application in your ATS. Keep it separate from anything a hiring decision uses, and join it to screening outcomes only for the analysis. That means the analysis lives where both pieces of data live: in your ATS or HR data warehouse. If your screening vendor writes each candidate's outcome back to the ATS record, you can run the math yourself without the vendor ever holding demographic data. Expect a large "unknown" group, because many applicants skip self-ID. New York City's bias audit rules for [automated employment decision tools](https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page) require the audit to report how many people fell into an unknown category, and let an auditor leave out a category that is under 2% of the data. Report your unknowns instead of quietly dropping them. ## Does adverse impact still matter after Executive Order 14281? Federal enforcement has stepped back. [Executive Order 14281](https://www.ebglaw.com/insights/publications/new-executive-order-addresses-disparate-impact-liability-key-implications-for-employers), signed in April 2025, told the EEOC to deprioritize disparate impact enforcement. But an executive order can't change Title VII, and private plaintiffs can still bring these claims. _Mobley v. Workday_ is the case most TA leaders have heard of. It's a private lawsuit alleging disparate impact, and as of [late September 2026](https://ailawsuittracker.com/cases/mobley-v-workday-3-23-cv-00770-rfl/) it was still in discovery. We wrote up [what the Workday case means for employers](/blog/workday-ai-hiring-lawsuit) separately. State and city rules are adding requirements too, including [Colorado's AI Act](/blog/colorado-ai-act-hiring), [California's ADMT rules](/blog/california-admt-hiring-rules), and New York City's Local Law 144. The exposure moved from a federal agency to plaintiffs' lawyers and state regulators. An analysis you ran and documented is the first thing your counsel will ask for either way. ## What to ask your AI screening vendor If your legal team is stalled on an AI screening deal, start with these five questions: 1. Does the tool screen anyone out beyond objective yes/no requirements we set, or score people on anything else? 2. Are the screening criteria ones we write and can see, or a model's inferred score? 3. Who runs your bias audit, how often, and can we read the results? 4. Does every screening outcome write back to our ATS, so we can run our own adverse impact analysis against our EEO data? 5. What happens to candidates who opt out of the AI screen? Our [guide to AI recruiting bias audits](/blog/ai-recruiting-bias-audits-compliant-platform) covers vendor selection in more depth, and our post on [what legal teams ask about AI screening](/blog/legal-teams-ai-screening) covers the review itself. ## How we handle this at Classet Joy, our AI phone screener, asks the must-have questions you write and returns a structured summary, transcript, and recording to your recruiters. If a candidate answers no to an objective yes/no requirement you set, like a valid driver's license or availability for overnight shifts, Joy can end the interview and route them to the stage you choose, the same way a yes/no question on an application works. She doesn't score or rank anyone on subjective traits, and your team makes every other decision. Classet doesn't collect self-reported demographic data from candidates. Candidates who opt out of the AI screen are flagged so your team can follow up with them directly. Warden AI, an independent auditor, tests Joy for sex and race/ethnicity bias every month and [publishes the results on a public dashboard](https://trust.warden-ai.com/classet/ai-phone-interviewer). When Classet is connected to your ATS, the summary writes back to the candidate record, so your team can run the stage-by-stage analysis above alongside the EEO data you already hold. How rules like Local Law 144 apply to your rollout is a question for your counsel. ## Frequently asked questions ### What is the four-fifths rule in an adverse impact analysis? The four-fifths rule says that if a group's selection rate is less than 80% of the rate for the group selected most often, that's generally treated as evidence of adverse impact. It comes from Section 1607.4(D) of the federal Uniform Guidelines on Employee Selection Procedures. It's a first screen, so pair it with a statistical significance test. ### How do you calculate an impact ratio? Divide the number of people in a group who passed a step by the number in that group who reached it. That's the group's selection rate. Then divide each group's selection rate by the highest group's selection rate. A result below 0.80 fails the four-fifths rule. ### How often should we run an adverse impact analysis on AI screening? Run it before launch on pilot data, then at least quarterly, and any time you change the screening questions or add a new role family. Frontline hiring moves a lot of applicants, so quarterly usually gives you enough data to see real patterns instead of noise. ### Does passing the four-fifths rule mean our AI screening tool is legal? No. Passing the rule doesn't prove a selection procedure is lawful, and failing it doesn't prove discrimination. Courts and agencies also look at statistical significance, whether each criterion is job-related, and whether a less discriminatory alternative was available. Talk to employment counsel about your specific process. ### Can an AI screening tool cause adverse impact if the questions are the same for everyone? Yes. Adverse impact is about outcomes, not intent or consistency. A neutral-sounding requirement, such as recent experience with a specific tool or availability for one shift, can filter out one group at a much higher rate. That's why you test each criterion as well as the overall result. ## Key points - An adverse impact analysis compares each group's pass rate to the highest group's rate. Below 0.80 fails the four-fifths rule. - For AI screening, run it at each stage: completing the screen, meeting must-haves, and advancing past a recruiter. - When a stage fails, test each criterion on its own, because a single question often drives the gap. - Small samples exaggerate gaps and large samples hide them, so report a significance test alongside every ratio. - Federal enforcement has pulled back, but private lawsuits and state rules haven't, so document the analysis. ## See how it works If your legal or HR team needs to see the audit trail before an AI screening pilot goes ahead, we'll walk through the Warden results, the candidate groups, and what lands in your ATS. [Book a demo](/demo?ref=blog-adverse-impact-analysis) and bring your counsel's questions. [![Paul Jones](/_next/image?url=https%3A%2F%2Fassets.basehub.com%2Fe0b5701f%2F6599306507912123f90f150a8bfaaf6c%2Fscreenshot-2026-01-28-at-10.53.16-am.png%3Fwidth%3D100%26height%3D100%26quality%3D100&w=128&q=75) Paul Jones Head of Growth at Classet Paul comes from an operator background running an Alpine-owned company, and brings firsthand experience with the hiring challenges Classet was built to solve. He's driven by a belief that the right technology can make meaningful work more accessible. ](/blog/authors/paul-jones) Follow Classet in Google Add classet.ai as a preferred source and our hiring research is more likely to show up in your Top Stories and AI Overviews. [Add as preferred source →](https://www.google.com/preferences/source?q=classet.ai) ## Explore More ### Use Cases - [RPO / BPO Recruiting](/use-cases/call-centers-bpo) - [Data Centers Recruiting](/use-cases/data-centers) - [Healthcare Recruiting](/use-cases/healthcare) ### Integrations - [Greenhouse](/integrations/greenhouse) - [Bullhorn](/integrations/bullhorn) - [Lever](/integrations/lever)