What is Statistical Significance?
Statistical significance is the probability that the difference in conversion rates between your Control and Variation is not due to random chance.
In the industry, 95% Confidence is the gold standard. This means there is only a 5% chance that the result is a "False Positive".
Why use a calculator?
Human intuition is bad at probability. A variation might look like it's winning (e.g. 50 conversions vs 40), but mathematically, that difference could easily happen by luck.
Zyro Tip: Manual calculators are slow. Zyro's platform calculates this automatically in real-time and uses Bandit Algorithms to automatically shift traffic to the winner, so you don't have to check a calculator every day.