Resources/Glossary/Cold email and lead gen
An A/B test, also called a split test, means sending two versions of a subject line or message to similar groups to see which gets more replies. You change one thing at a time, so when one version wins you know what caused the difference.
In one line
Sending two versions of a subject line or message to similar groups to see which gets more replies. Change one thing at a time.
A fair test needs three things. The two groups must be similar, so split one list at random rather than sending version A to engineers and version B to sales leaders. Only one element should change, such as the subject line, the opening line or the call to action. And each version needs enough sends that the result is not just luck. Small tests with a handful of replies can easily mislead.
For cold outreach, reply rate is the best measure. Open rates have become unreliable because some email apps preload images and count emails as opened that nobody read. Positive replies, such as interested candidates or booked calls, are better still. Most outreach tools can split a list automatically and report results for each version side by side.
Small wording changes can change how many people answer. Testing replaces guesswork with evidence, so recruiters and agency business developers learn what works with their audience rather than copying templates. Over time, steady testing improves response rates, saves time and protects sender reputation, because better messages get fewer spam complaints.
Four hundred nurses get the same outreach, split at random into two groups. Version A opens with the hospital's name; version B opens with the schedule, three 12 hour shifts a week. Everything else is the same. In this example version A gets 14 replies and version B gets 27. She makes the schedule opening her default and tests the call to action next.
A few of the outreach tools in our directory, including our picks. Prices are the published starting price on the date we last checked.
There is no single number, but more is better. With small lists, the gap between versions has to be large before you can trust it. If each version gets only a few replies, treat the result as a hint and run the test again before changing your approach.
Start with the parts that most affect replies: the subject line, the first sentence and the call to action. Then test length, personalization and send time. Testing the role's main selling point, such as pay, schedule or remote work, often gives the biggest lesson.
Some email apps load images automatically, which counts emails as opened even when nobody read them. That makes open rates unreliable for comparing versions. Reply rate, and especially positive reply rate, shows whether the message actually worked with real people.
Written by recruiters at Work Insiders. Part of the recruiting and HR tech glossary. Updated October 2026.
