What 'Bias Audit' Actually Means for an AI Hiring Tool

“Bias audited” appears in marketing materials for AI hiring tools with increasing frequency. It is not a standardized certification. It does not mean the same thing across vendors. Under NYC Local Law 144, it has a specific legal definition. Everywhere else, it means whatever the vendor decides it means.

Understanding what a real bias audit tests, and what it does not, is necessary for evaluating whether an AI tool is actually compliant.

What NYC Local Law 144 requires

NYC Local Law 144, which applies to employers using automated employment decision tools (AEDTs) to screen candidates in New York City, requires an annual bias audit conducted by an independent auditor.

The audit must:

1. Calculate selection rates (or score distributions) across race, sex, and intersectional categories
2. Compare those rates using the impact ratio: the selection rate for each group divided by the selection rate for the highest-selected group
3. Include the number of applicants in each category
4. Be published publicly on the employer’s website before the tool is used

The independent auditor requirement means the audit cannot be conducted by the employer or the vendor. It must be a third party with no financial relationship to either.

What the audit actually tests

The bias audit is an adverse impact analysis. It tests whether the tool’s outcomes (selection rates or score distributions) differ significantly across demographic groups. This is a statistical test of what the tool produces, applied to a specific dataset.

What the audit does not test:

Intent: Adverse impact analysis measures outcomes, not intent. A tool can produce adverse impact without any discriminatory design.

Validity: The bias audit does not assess whether the tool predicts job performance. A tool can pass a bias audit and still be invalid, producing equal-opportunity noise rather than equal-opportunity signal.

Disparate treatment: The audit tests whether the tool treats groups equally in outcome. It does not test whether the tool is being applied differently to different groups by the employer.

All protected classes: NYC LL 144 specifies race, sex, and their intersection. The audit does not necessarily cover age, disability status, national origin, or other protected characteristics under federal law.

What a more complete evaluation looks like

A thorough evaluation of an AI hiring tool for bias goes beyond what any single bias audit requires:

Validity evidence: Does the tool predict job performance? An adverse impact analysis without validity evidence tells you the tool treats groups equally, but not whether it is measuring anything meaningful.

Construct analysis: What is the tool measuring? If it is measuring speech patterns, fluency, or presentation style rather than job-relevant competencies, equal outcomes do not make it appropriate.

Intersectional analysis: Impact ratios at the intersection of race and gender can reveal disparities that are invisible when race and gender are analyzed separately.

Cross-role generalizability: A bias audit on a dataset from one role or industry may not generalize to a different application.

The distinction that matters for employers

An employer using an AI hiring tool takes on compliance responsibility for that tool. If the tool produces adverse impact, the employer, not the vendor, faces the legal exposure under EEOC enforcement.

“The vendor told us it was bias audited” is not a legal defense. Understanding what the audit covered, what it did not cover, and what validity evidence supports the tool’s use is the employer’s obligation.