Training data is the set of examples an AI model learned from. If past hiring decisions in that data were unfair, the model can repeat the pattern. That is why buyers ask vendors what their models were trained on and how they test the results for bias.
AItopic
2 minread
2related terms
In one line
The examples a model learned from. If past hiring decisions in that data were unfair, the model can repeat the pattern, which is why buyers ask vendors what their models were trained on.
General AI models are trained on very large amounts of public and licensed text. Recruiting tools then often add their own layer, training or tuning on resumes, job descriptions, interview outcomes or past hires. Each layer shapes what the model treats as a good candidate, and the model has no way to know whether past decisions were fair. A model trained on a decade of hires from one narrow group may learn to favor schools, gaps or phrasing that track that group rather than skill.
Training data is also a privacy question. Some tools use customer data, including candidate records and prompts, to improve their models unless you opt out. Contracts and settings should state clearly whether your candidates' data is used for training, how long it is kept and whether it is shared across customers. Get those answers in writing before any candidate data goes in.
Why it matters to recruiters
Laws in some places now require bias audits of automated hiring tools, such as New York City's rules on automated employment decision tools. Even where no such law applies, US anti discrimination law still covers outcomes produced with software. Knowing what a tool learned from helps you spot risk before it shows up as a pattern of unfair rejections.
An example from a recruiting desk
An AI screening tool is on trial with an in house team. Before signing, the TA lead asks the vendor three questions: what data the model was trained on, whether customer candidate data is used for further training, and for its latest bias audit results by sex and race. The vendor answers the first two but has no audit, so the team limits the tool to ranking, never auto rejecting.
Tips that help
Ask every AI vendor what data trained the model and whether your data will be added.
Request bias audit results and the method used, not just a summary claim.
Turn off training on your data in tool settings where the option exists.
Keep a human decision maker for any rejection an AI tool recommends.
Mistakes to avoid
Assuming a tool is neutral because it does not see names or photos.
Pasting candidate data into consumer AI apps without checking their training settings.
Frequently asked questions
Can I stop an AI tool from training on my candidate data?
Often, yes. Many business plans exclude your data from training by default, and consumer apps usually have a setting to turn it off. Check the plan terms and settings, and get it in writing for any recruiting vendor that handles candidate records.
What is a bias audit?
A bias audit tests whether a hiring tool's results differ across groups, such as by sex or race, often by comparing selection rates. New York City requires one for certain automated hiring tools. This is not legal advice, and rules vary by location and change often.
Does removing names from resumes fix training data bias?
It helps but does not solve it. Models can still pick up proxies such as zip codes, school names, clubs or career gaps. Testing outcomes across groups is the only reliable way to see whether a tool treats people fairly.
