Sequential Testing Duration and Boundary Calculator
Plan group sequential A/B experiments that permit continuous monitoring and valid early stopping without inflating false positive Type I error rates. Uses O’Brien-Fleming alpha spending bounds.
Current conversion or click-through baseline metric
Desired relative uplift to detect with 80% statistical power
Total daily traffic allocated equally across control and variant
Maximum sample needed per branch before mandatory test conclusion
How to Use This Calculator
- Enter your baseline conversion rate.
- Set your target minimum detectable relative effect (MDE).
- Enter expected daily visitor volume.
- Review the required variant sample size and maximum duration.
Formula & Calculation Methodology
Sequential designs require a slight ~8% sample inflation over fixed-horizon tests to accommodate interim looks while strictly preserving overall family-wise Type I error α = 0.05.
Worked Example
5% baseline conversion with 10% MDE and 2,000 visitors/day
10% baseline with 15% MDE and 5,000 visitors/day
Frequently Asked Questions
Why not simply check A/B test results every day on a standard test?↓
Continuous peeking on a fixed-horizon test causes extreme alpha inflation; checking results daily can increase the true false positive rate from 5% to over 30%.
How does O’Brien-Fleming early stopping work?↓
It sets very conservative significance thresholds early in the test (e.g. p < 0.0001) that gradually relax toward p < 0.05 near test completion.
Can sequential tests stop early for futility?↓
Yes. Sequential boundaries can declare early victory for winners or early termination for futile variants that have negligible chance of reaching significance.
What is a limitation of sequential testing?↓
If an experiment does not stop early, it will run approximately 8% longer than an equivalent fixed-sample test.
Related Calculators
False Positive Risk Calculator for A/B Test Peeking
Open calculator →Minimum Reliably Detectable Effect (MRDE) Calculator
Open calculator →Dunnett's Correction A/B/n Sample Size Estimator
Open calculator →Sample Size Calculator
Open calculator →Sample Ratio Mismatch (SRM) Diagnostic Analyzer
Open calculator →Conversion Rate Calculator
Open calculator →Source Citation & Disclaimer
Informational tool only; not licensed tax, legal, or financial advice.