A bias audit is an independent test of whether a hiring tool selects or scores one group of people at a lower rate than another. New York City requires a bias audit within one year before using a covered automated hiring tool, plus a public summary of the results.
AItopic
2 minread
3related terms
In one line
An independent test of whether a hiring tool selects one group at a lower rate than another. New York City requires one each year for covered tools.
An auditor takes data on how the tool scored or selected people and compares outcomes by group, such as sex and race or ethnicity. The core measure is the impact ratio: each group's selection rate divided by the rate of the most selected group. A low ratio flags possible adverse impact. Under the US federal Uniform Guidelines, a ratio below four fifths, or 80 percent, is generally treated as evidence of adverse impact.
Under New York City's Local Law 144, employers and employment agencies that use an automated employment decision tool for candidates or employees in the city need a bias audit by an independent auditor within the past year, must publish a summary of the results, and must give notice to candidates before the tool is used. Other places are adding similar rules, so the practice is spreading beyond New York.
Why it matters to recruiters
A bias audit gives you evidence about whether a tool is fair before it affects real people. It helps you meet legal requirements where they apply, choose between vendors and catch problems after an update. Agencies placing candidates into New York City roles need to know whether their clients' tools, or their own, are covered.
An example from a recruiting desk
An agency uses an AI screening tool on roles in New York City. Before renewing, the ops lead asks the vendor for its latest independent audit. The summary shows impact ratios by sex and race or ethnicity, and one category falls below 0.8 for a specific job family. In this example the agency pauses the tool on those roles, asks the vendor to investigate and posts the audit summary on its careers page.
Tips that help
Ask every vendor for its latest independent audit and the date it was done.
Check that the audit used data similar to your own roles and candidate pool.
Calendar the renewal so you never use a covered tool with an audit older than a year.
Treat an audit as a starting point and keep monitoring outcomes between audits.
Mistakes to avoid
Relying on a vendor's marketing claim of fairness instead of an actual audit report.
Forgetting the notice requirement and only completing the audit.
Frequently asked questions
Who needs a bias audit?
In New York City, employers and employment agencies using an automated employment decision tool to substantially help decide hiring or promotion for city based roles need one. Other places may have different or new rules. Even where not required, many companies audit tools to manage risk. Check current law. This is not legal advice.
How often do you need a bias audit?
Under New York City's rules, the audit must have been done within one year before you use the tool, so in practice it is yearly. It is also wise to retest after major model updates, new job types or big changes in your applicant pool, since results can shift.
What does a bias audit measure?
It measures selection rates or average scores for different groups and compares them using impact ratios. A ratio well below 1.0 means one group is selected less often than the most selected group. The audit flags gaps for review; it does not decide on its own whether a tool is unlawful.
