Home / Glossary / A/B testing in cold email
Glossary entry

A/B testing in cold email

The practice of sending two variants of a cold email to randomized halves of a list and measuring which variant outperforms. The technique most SDR teams claim to do and few do well: real A/B testing requires sample sizes above 200-300 per variant for statistical confidence, holding all other variables constant. Most 'A/B tests' on 30-send batches are pattern-matching, not testing. Test one variable at a time (subject vs body vs CTA), measure for at least 7 days, treat ≤2-point reply rate differences as noise.

Category: Outbound Reading time: 2 min

01Definition

TLDR
The practice of sending two variants of a cold email to randomized halves of a list and measuring which variant outperforms. The technique most SDR teams claim to do and few do well: real A/B testing requires sample sizes above 200-300 per variant for statistical confidence, holding all other variables constant. Most 'A/B tests' on 30-send batches are pattern-matching, not testing. Test one variable at a time (subject vs body vs CTA), measure for at least 7 days, treat ≤2-point reply rate differences as noise.

02Why it matters

Without disciplined A/B testing, 'what works' becomes whatever the loudest rep believes worked last week.

03Example

Worked example
Test subject A vs B on 500 recipients each → A wins by 4 percentage points → roll out to the next 5,000 sends.
Found something off? Glossary entries are reviewed in batches. If a definition is wrong, incomplete, or missing the angle that would help your team, email [email protected] and reference this entry — we'll revise and credit your input.
Signal-anchored outbound, no theater

Stop guessing. See every signal firing on every account in your CRM.

Connect a CRM. Mama detects 47 signal types, scores every account daily, and writes the first email. Free up to 100 accounts. No card.

Start free → Back to glossary