🧪 A/B Test Significance Calculator
Enter conversions and visitors for each variant to test whether the difference in conversion rate is significant — the A/B-test z-test.
Variant A
Variant B
Rate A
20%
Rate B
30%
z statistic
-2.309
p-value
0.0209
Significant difference at α = 0.05
The 10 percentage-point difference is statistically significant (p = 0.0209 < 0.05) — variant B converts higher.
The two-proportion z-test compares two conversion rates (the standard A/B-test significance test), using a pooled proportion for the standard error. It assumes independent visitors and enough conversions per group (a common rule: at least ~5–10 in each cell). Statistical significance isn't the same as a meaningful business effect. 🔒 Computed in your browser.
How the a/b test significance calculator works
The tool runs a two-proportion z-test: it computes each variant’s conversion rate, a pooled proportion across both, the standard error, the z statistic and the exact p-value from the standard normal distribution. It compares the p-value to your significance level α and reports whether the observed difference is unlikely to be due to chance.
This is the significance test behind most A/B testing tools. Two cautions matter: significance depends on sample size, so run the test to a pre-planned sample rather than stopping the moment it turns green (“peeking” inflates false positives); and statistical significance isn’t the same as a difference big enough to matter commercially. The test also assumes independent visitors and enough conversions per group. Everything runs locally, so your experiment data never leaves your browser.
Frequently asked questions
How do I know if my A/B test result is significant?
Enter the conversions and total visitors for each variant. The tool computes a two-proportion z-test and a p-value; if the p-value is below your chosen α (commonly 0.05), the difference is statistically significant.
What test does an A/B test use?
The standard approach for comparing two conversion rates is a two-proportion z-test, which uses a pooled proportion to estimate the standard error. That’s exactly what this calculator runs.
Why shouldn’t I stop the test as soon as it’s significant?
Repeatedly checking and stopping the moment p < α (“peeking”) greatly inflates the false-positive rate. Decide the sample size in advance and evaluate the result once you reach it, or use a method designed for sequential testing.
Does a significant result mean the change is worth shipping?
Not necessarily — significance says the difference is unlikely to be chance, not that it’s large. Look at the actual percentage-point lift and whether it justifies the change for your business.
Is my experiment data uploaded?
No — the z-test runs entirely in your browser and works offline, so your conversion data stays on your device.