A/B Test Statistics Calculator API
Frequentist, Bayesian, sequential and CUPED A/B test analysis, sample ratio mismatch, sample size and MDE, multiple-testing correction, delta-method ratio metrics, bandit allocation, deterministic bucketing, feature-flag rule evaluation, novelty detection, segment lift, guardrail and holdout checks. Pure computation on the numbers you send.
Two-arm A/B scorecard for conversion (two-proportion z) or continuous (Welch t) metrics: absolute and relative lift, confidence intervals, p-value and significance. Pure computation on supplied aggregates.
Posterior, probability to beat control, probability of being best, expected loss and equal-tailed credible intervals for 2-10 conversion arms. Deterministic: exact Miller sum or quadrature, no Monte Carlo.
Lan-DeMets group-sequential boundaries (O'Brien-Fleming or Pocock spending) for planned looks with a stop/continue decision, or an always-valid mSPRT p-value for continuous monitoring.
CUPED adjustment with a pre-experiment covariate on unit-level data: theta, variance reduction, and Welch tests on raw and adjusted values.
Chi-square goodness-of-fit test of observed arm counts against expected traffic shares, with an exact binomial p-value for tiny two-arm counts and generic causes when a mismatch is flagged.
Required users per arm and duration for a target minimum detectable effect, or the MDE for a fixed sample, for proportion or mean metrics with power, multi-arm Bonferroni and unequal allocation.
Bonferroni, Holm and Benjamini-Hochberg adjusted p-values and reject decisions across metrics or arms, returned in input order.
Confidence interval for a ratio metric whose analysis unit differs from the randomisation unit, plus the treatment-vs-control difference and relative lift. Accepts unit rows or aggregates.
Next-period traffic weights from arm outcomes: Thompson-sampling probabilities by deterministic quadrature, or UCB1 indices, with an exploration floor.
Stateless SHA-256 bucketing of unit ids into an experiment with traffic gating, ordered group shares and an optional layer, documented so any SDK can reproduce it.
Evaluate ordered feature-flag rules (attribute operators, semver comparison, deterministic percentage rollout) for one user context and explain which rule matched.
OLS trend test on daily lift with early-versus-late means, classifying the pattern as novelty, primacy or stable.
Per-segment conversion lift with Benjamini-Hochberg adjusted p-values, a Cochran Q heterogeneity test and suppression of small segments.
One-sided non-inferiority tests of guardrail metrics against caller-supplied relative or absolute margins, with bounds and an overall pass flag.
Exposed-versus-holdout impact with confidence interval and p-value, plus diluted population-level impact for partial exposure.