A/B Test Statistics Calculator API

A/B TestingExperimentationStatisticsBayesianSequential TestingCUPEDFeature Flagsx402MCPOpenAPI

Frequentist, Bayesian, sequential and CUPED A/B test analysis, sample ratio mismatch, sample size and MDE, multiple-testing correction, delta-method ratio metrics, bandit allocation, deterministic bucketing, feature-flag rule evaluation, novelty detection, segment lift, guardrail and holdout checks. Pure computation on the numbers you send.

POST/api/v1/experimentation/ab-test-frequentist$0.02 USDC

Two-arm A/B scorecard for conversion (two-proportion z) or continuous (Welch t) metrics: absolute and relative lift, confidence intervals, p-value and significance. Pure computation on supplied aggregates.

POST/api/v1/experimentation/bayesian-ab$0.03 USDC

Posterior, probability to beat control, probability of being best, expected loss and equal-tailed credible intervals for 2-10 conversion arms. Deterministic: exact Miller sum or quadrature, no Monte Carlo.

POST/api/v1/experimentation/sequential-test$0.04 USDC

Lan-DeMets group-sequential boundaries (O'Brien-Fleming or Pocock spending) for planned looks with a stop/continue decision, or an always-valid mSPRT p-value for continuous monitoring.

POST/api/v1/experimentation/cuped-adjust$0.03 USDC

CUPED adjustment with a pre-experiment covariate on unit-level data: theta, variance reduction, and Welch tests on raw and adjusted values.

POST/api/v1/experimentation/sample-ratio-mismatch$0.01 USDC

Chi-square goodness-of-fit test of observed arm counts against expected traffic shares, with an exact binomial p-value for tiny two-arm counts and generic causes when a mismatch is flagged.

POST/api/v1/experimentation/sample-size-mde$0.01 USDC

Required users per arm and duration for a target minimum detectable effect, or the MDE for a fixed sample, for proportion or mean metrics with power, multi-arm Bonferroni and unequal allocation.

POST/api/v1/experimentation/multiple-testing-correct$0.01 USDC

Bonferroni, Holm and Benjamini-Hochberg adjusted p-values and reject decisions across metrics or arms, returned in input order.

POST/api/v1/experimentation/delta-method-ratio$0.03 USDC

Confidence interval for a ratio metric whose analysis unit differs from the randomisation unit, plus the treatment-vs-control difference and relative lift. Accepts unit rows or aggregates.

POST/api/v1/experimentation/bandit-allocate$0.03 USDC

Next-period traffic weights from arm outcomes: Thompson-sampling probabilities by deterministic quadrature, or UCB1 indices, with an exploration floor.

POST/api/v1/experimentation/bucket-assign$0.01 USDC

Stateless SHA-256 bucketing of unit ids into an experiment with traffic gating, ordered group shares and an optional layer, documented so any SDK can reproduce it.

POST/api/v1/experimentation/flag-rule-evaluate$0.01 USDC

Evaluate ordered feature-flag rules (attribute operators, semver comparison, deterministic percentage rollout) for one user context and explain which rule matched.

POST/api/v1/experimentation/novelty-effect-detect$0.01 USDC

OLS trend test on daily lift with early-versus-late means, classifying the pattern as novelty, primacy or stable.

POST/api/v1/experimentation/segment-lift$0.02 USDC

Per-segment conversion lift with Benjamini-Hochberg adjusted p-values, a Cochran Q heterogeneity test and suppression of small segments.

POST/api/v1/experimentation/guardrail-check$0.02 USDC

One-sided non-inferiority tests of guardrail metrics against caller-supplied relative or absolute margins, with bounds and an overall pass flag.

POST/api/v1/experimentation/holdout-impact$0.01 USDC

Exposed-versus-holdout impact with confidence interval and p-value, plus diluted population-level impact for partial exposure.