Run a frequentist A/B test scorecard API

Two-arm A/B scorecard for conversion (two-proportion z) or continuous (Welch t) metrics: absolute and relative lift, confidence intervals, p-value and significance. Pure computation on supplied aggregates.

The Run a frequentist A/B test scorecard operation is part of the A/B Test Statistics Calculator API on BRAINIAC.software. Two-arm A/B scorecard for conversion (two-proportion z) or continuous (Welch t) metrics: absolute and relative lift, confidence intervals, p-value and significance. Pure computation on supplied aggregates. It is exposed as POST /api/v1/experimentation/ab-test-frequentist and priced at $0.02 USDC on Base. Payment uses the x402 exact scheme: an unpaid call returns HTTP 402 with payment requirements; after paying the exact USDC amount, retry the same request with a payment-signature header and Idempotency-Key. No account and no API key are required. Data sources for this product: NIST/SEMATECH e-Handbook of Statistical Methods (Welch t-test, chi-square goodness of fit, sample size, regression) Miller (2015) Formulas for Bayesian A/B Testing Lan & DeMets (1983) alpha-spending boundaries; Johari et al. (2017) always-valid mSPRT Deng et al. (2013) CUPED and Deng et al. (2018) delta method in metric analytics Holm (1979); Benjamini & Hochberg (1995); Higgins & Thompson (2002) heterogeneity Auer et al. (2002) UCB1; Russo et al. (2018) Thompson sampling NIST FIPS 180-4 SHA-256; Semantic Versioning 2.0.0 Limitations: Every endpoint is a stateless calculation on caller-supplied aggregates or unit rows. No events are stored, no database or network is used, and results are for human review. Normal-approximation tests are unreliable for very small counts; inputs that are structurally invalid are rejected, but design problems such as peeking, post-hoc segments or biased assignment are not detected beyond the checks documented per endpoint. Bayesian and Thompson results use deterministic quadrature, not Monte Carlo, so repeated calls return identical values; they depend on the supplied Beta prior. Group-sequential boundaries are computed numerically (Simpson grid, tolerance about 1e-3 against published tables) and assume the planned information fractions are met. Bucket assignment and flag rollouts are SHA-256 based and reproducible, but they are not compatible with any vendor SDK hashing scheme. Flag regex conditions use the JavaScript engine behind a safety guard, not RE2. Outputs are compliance prechecks, drafts, or data lookups for human review. They are not official determinations, legal advice, identity confirmation, or certified filings. Related operations in this group include Compare A/B arms with a Bayesian Beta-Binomial model, Compute sequential testing boundaries or an always-valid p-value, Apply CUPED variance reduction and test the adjusted effect, Detect A/B sample ratio mismatch. Open the group documentation for the full request schema, field notes, and response examples before integrating.

POST$0.02 USDC
/api/v1/experimentation/ab-test-frequentist

Example request

curl -i -X POST "https://www.brainiac.software/api/v1/experimentation/ab-test-frequentist" \
  -H 'content-type: application/json' \
  -H 'Idempotency-Key: <fresh-64-character-random-hex-key>' \
  --data-binary @- <<'JSON'
{
  "metricType": "proportion",
  "alpha": 0.05,
  "alternative": "two-sided",
  "control": {
    "users": 10000,
    "conversions": 1000
  },
  "treatment": {
    "users": 10000,
    "conversions": 1100
  }
}
JSON

An unpaid probe returns HTTP 402 with x402 payment requirements. After paying the exact USDC amount on Base, retry the identical request with a payment-signature header.

FAQ

  • Do I need an API key? No. Pay the listed USDC amount with x402.
  • What network? Base (eip155:8453), USDC.
  • Is this an official determination? No. Results are for human review against the cited sources.

Related APIs

A/B Test Statistics Calculator APIFull docsHow it works