Loading…
Loading…
SigmaCalc
Find out whether your variant's improvement is real, or just noise from a small sample.
Checking results daily and stopping as soon as you see significance inflates your false-positive rate dramatically — decide on a sample size or time window in advance and stick to it, rather than stopping the moment p dips below 0.05.
A tiny, real lift can still be statistically significant with enough traffic, but not worth the engineering cost to ship. Always weigh the lift's size and business impact alongside the p-value, not instead of it.
p-value
0.15
Not statistically significant at α = 0.05
What you entered
Conversion rate A
50 ÷ 1000= 0.05Conversion rate B
65 ÷ 1000= 0.065Pooled conversion rate
(50 + 65) ÷ (1000 + 1000)= 0.0575Standard error of the difference
√[p(1−p)(1/1000 + 1/1000)]= 0.0104z-statistic
(0.065 − 0.05) ÷ 0.0104= 1.4408Two-tailed p-value
from z = 1.4408= 0.1496Result
p-value: 0.15
Variant B converts at 6.5% versus 5% for A, a 30% lift — with a p-value of 0.15, that difference is not statistically significant at the conventional 0.05 threshold.