Calculators
A/B Test Significance Calculator
Check whether your A/B test result is statistically significant and plan how many visitors a test needs.
- Relative uplift
- +14.00%
- p-value
- 0.0278
- z-score
- 2.2
- Confidence reached
- 97.2%
95% confidence interval for the absolute difference in conversion rate (B − A): +0.08% to +1.32%.
What is this tool?
Did variant B really win, or is the difference just noise? Enter visitors and conversions for each version to get the conversion rates, relative uplift, p-value, and a clear verdict at your chosen confidence level. Planning a test? The sample size calculator tells you how many visitors and days you need.
Significance uses a two-proportion z-test with a pooled standard error, the standard method for comparing conversion rates. A result is significant when the p-value is below 1 − confidence, for example below 0.05 at 95%.
Sample sizes use the normal approximation for two proportions at the chosen power (80% by default). Decide the sample size in advance and avoid stopping as soon as a result looks significant — peeking inflates false positives.
How to use the A/B Test Calculator
- Enter the number of visitors and conversions for A (control) and B (variant).
- Choose a confidence level (95% is standard) and a two-sided or one-sided test.
- Read the verdict, uplift, p-value, and the confidence interval of the difference.
- Before a test, open Sample size: enter your baseline rate, the smallest lift worth detecting, and daily traffic to estimate how long to run it.
Examples
A significant win
A: 10,000 visitors, 500 conversions B: 10,000 visitors, 600 conversions
Uplift +20% · p ≈ 0.002 · significant at 95%
Frequently asked questions
What confidence level should I use?
95% is the common default. Use 99% when a wrong decision would be costly, or 90% for quick, low-risk experiments.
One-sided or two-sided?
Two-sided tests detect a difference in either direction and are the safer default. One-sided tests only ask whether B is better, need fewer visitors, but can't tell you if B is worse.
What is the minimum detectable effect?
The smallest relative improvement you care to detect. Smaller effects need far more visitors: halving the effect roughly quadruples the sample size.
My test isn't significant. What now?
Either the difference is too small to matter or you need more data. Check the sample size calculator to see how many visitors are needed for the effect you expect.
Want more power later?
Premium tools, higher limits, and API access are on the roadmap. V1 stays free and private in your browser.