A low baseline costs traffic
The rarer the conversion, the more visitors it takes to separate a real change from noise. Roughly speaking, halving your baseline conversion rate doubles the traffic each variant needs.
Free tool
A/B Test Sample Size Calculator
Before you launch an experiment, find out how many visitors each variant needs so the result is actually trustworthy. Enter your numbers and the calculator updates instantly, all in your browser.
31,232
visitors needed per variant
62,464 visitors total across two variants
What drives the number
Why the sample size moves as much as it does
The detectable effect is the big lever
Minimum detectable effect enters the formula squared, so it dominates everything else. Asking to detect a 5% lift instead of a 10% one costs about four times the traffic — decide what size of win would actually change your decision, and test for that.
Set the duration before you start
Convert the number above into whole weeks of traffic, then run for that long. Stopping the moment a result turns significant is the most common way teams ship changes that do not hold up.
FAQ
About sample size
How is A/B test sample size calculated?
This tool uses the standard two-proportion formula based on your baseline conversion rate, the minimum detectable effect, your chosen confidence level, and statistical power to estimate the visitors needed per variant.
What is a minimum detectable effect?
The minimum detectable effect is the smallest relative change in conversion rate you want the test to be able to detect reliably. Smaller effects require larger samples.
Why does my test need so many visitors?
Lower baseline rates, smaller effects, higher confidence, and higher power all increase the sample size required to avoid false positives and false negatives.
Related
More from PulsePanda
Run the test, then watch the sessions
PulsePanda pairs your experiments with session replay, heatmaps, funnels, and feedback so you understand the why behind the win.
Start for free