A/B Test Sample Size Calculator
Calculate the required sample size for statistically significant A/B test results. Get your test duration and avoid common mistakes.
Test Parameters
Current Performance
Current % of visitors who convert
Average daily traffic to your test page
Expected Improvement
Relative improvement you want to detect (smaller = longer test)
Statistical Requirements
Confidence level (95% is industry standard)
Probability of detecting a real effect (80% minimum recommended)
Configure your test parameters and click
"Calculate Sample Size" to see results
Statistical Concepts Explained
Statistical Significance
The probability that your observed results aren't just due to random chance. A 95% confidence level means there's only a 5% chance your results are a false positive.
- •95% (recommended): Industry standard for most tests
- •90%: Acceptable for exploratory tests
- •80%: Too low risk - may lead to false conclusions
Statistical Power
The probability of detecting a real effect if it actually exists. Higher power means you're less likely to miss a true improvement.
- •80% (minimum): Standard threshold for reliable tests
- •90%: Better for detecting smaller effects
- •70%: Risky - high chance of missing real improvements
Minimum Detectable Effect
The smallest improvement your test can reliably detect. Smaller effects require larger sample sizes and longer test duration.
- •20-30%: Reasonable for most tests
- •10-20%: Requires larger samples
- •<10%: Very large sample needed
Sample Size
The number of visitors needed in each variation to achieve your desired statistical power and significance level.
- •Equal split: 50/50 between control and treatment
- •More visitors: Needed for smaller effects
- •Higher conversions: Need fewer total visitors
Common A/B Testing Mistakes to Avoid
Stopping Tests Early
Don't stop when you see promising results. Wait for your calculated sample size to avoid false positives.
Running Multiple Tests Simultaneously
Test one variable at a time. Multiple simultaneous tests can interact and skew results.
Ignoring Seasonal Effects
Run tests for complete business cycles. Weekend vs. weekday behavior can be very different.
Testing Too Small Effects
Don't test micro-changes. Focus on meaningful improvements that justify the testing effort.
Not Having a Hypothesis
Always start with a clear hypothesis about why your change will improve conversions.
Unequal Traffic Splits
Unless you have a specific reason, always use 50/50 traffic splits for maximum statistical power.
Ready to track your A/B test results like a pro?
Try SkunkCRM - Track Conversions Automatically