Bayesian A/B Test Calculator
Stop guessing if your A/B test results are significant. Get a complete Bayesian analysis with probability to win, expected lift, risk quantification, and actionable recommendations - all calculated instantly in your browser.
How it works
Bayesian Approach
Uses Beta-Binomial conjugate priors for exact posterior distributions. No p-hacking or multiple comparison issues.
Monte Carlo Simulation
100,000 samples from posterior distributions ensure accurate probability estimates.
Expected Loss
Quantifies the cost of making the wrong decision, helping you make risk-aware choices.
Built for marketers, product managers, and data analysts who want statistically sound A/B test analysis without the complexity.
Enter your test data
Input visitors and conversions for each variant
Control Group
Variation Group
Frequently Asked Questions
Why Bayesian instead of frequentist statistics?
Bayesian analysis gives you direct probability statements like "There's a 95% chance variation beats control." It also allows for continuous monitoring without inflating false positive rates, and naturally incorporates prior knowledge when available.
What does "expected loss" mean?
Expected loss quantifies the potential cost of making the wrong decision. If the expected loss of implementing the variation is 0.05%, it means on average you'd lose 0.05 percentage points of conversion rate if the variation is actually worse. This helps you make risk-aware decisions.
What is Sample Ratio Mismatch (SRM)?
SRM occurs when the actual traffic split differs significantly from the expected split (e.g., 50/50). This can indicate bugs in your experiment setup, bot traffic, or data collection issues. The calculator checks for SRM automatically and warns you if detected.
How much data do I need?
There's no fixed minimum, but Bayesian analysis becomes more reliable with more data. As a rule of thumb, aim for at least 100 conversions per variant for meaningful results. The calculator will suggest continuing testing if there isn't enough data to make a confident decision.
Is my data stored anywhere?
No. All calculations happen entirely in your browser. Your data never leaves your device and is not stored or transmitted anywhere.