How to Use the A/B Test Sample Size Calculator
Running an A/B test without enough data leads to unreliable conclusions — either seeing a win that doesn't exist (Type I error) or missing a real improvement (Type II error). This calculator uses the standard two-proportion z-test formula to find the minimum sample size per variant needed to detect your target lift with statistical confidence.
The Minimum Detectable Effect (MDE) is the smallest relative improvement worth detecting. If your baseline is 5% and a 10% relative lift (to 5.5%) would justify a creative change, set MDE to 10%. Smaller MDE values require larger sample sizes.
The industry standard settings are 95% confidence and 80% power. Raising to 99%/90% substantially increases the required sample but is appropriate for high-stakes decisions.
Frequently Asked Questions
Yes. Each variant (control A and treatment B) gets the per-variant sample size n. The total is 2n. If traffic is limited, raise the MDE or lower confidence to 90% to reduce the required sample.
Run the test for at least 7 full days regardless of when you hit the sample size target. This accounts for day-of-week variation in user behavior and helps prevent peeking bias.