> ## Documentation Index
> Fetch the complete documentation index at: https://www.pagent.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Choose your setup

> Pick a statistics level, match your company's standard and trade speed against certainty

export const StatisticsKit = (() => {
  const COLORS = {
    ink: "#14310D",
    accent: "#07C983",
    win: "#15803D",
    loss: "#C2410C",
    neutral: "#9FA09E",
    timeout: "#DFDFDE",
    grid: "#EEEFEE",
    muted: "#707170",
    surface: "#F5F6F0",
    band: "rgba(7, 201, 131, 0.14)"
  };
  const CHEVRON = "url(\"data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='12' viewBox='0 0 12 12'%3E%3Cpath d='M3 4.5 6 7.5 9 4.5' fill='none' stroke='%23707170' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E\")";
  const CSS = `
.pgw { font-family: inherit; color: #3F403E; background: #FFFFFF; border: 1px solid rgba(10, 11, 10, 0.1); border-radius: 16px; padding: 20px 24px; margin: 24px 0; }
.pgw-eyebrow { font-size: 14px; line-height: 20px; font-weight: 600; color: ${COLORS.ink}; }
.pgw-title { font-size: 16px; line-height: 24px; font-weight: 600; color: #252625; margin-top: 2px; }
.pgw-desc { font-size: 14px; line-height: 20px; color: #707170; margin-top: 2px; }
.pgw-fields { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 14px 16px; margin: 20px 0; }
.pgw-field { display: flex; flex-direction: column; justify-content: space-between; gap: 6px; }
.pgw-label { font-size: 14px; line-height: 20px; font-weight: 600; color: #252625; }
.pgw-label span { font-weight: 400; color: #707170; }
.pgw input, .pgw select { width: 100%; height: 36px; padding: 0 12px; border: none; border-radius: 12px; background-color: ${COLORS.surface}; box-shadow: 0 0 0 1px rgba(159, 160, 158, 0.3); font: inherit; font-size: 14px; color: #171817; transition: box-shadow 0.15s; }
.pgw select { appearance: none; -webkit-appearance: none; background-image: ${CHEVRON}; background-repeat: no-repeat; background-position: right 12px center; padding-right: 32px; cursor: pointer; }
.pgw input:hover, .pgw select:hover { box-shadow: 0 0 0 1px rgba(112, 113, 112, 0.5); }
.pgw input:focus, .pgw select:focus { outline: none; box-shadow: 0 0 0 2px ${COLORS.ink}; }
.pgw-stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); gap: 12px; }
.pgw-stat { background: ${COLORS.surface}; border-radius: 12px; padding: 12px 14px; }
.pgw-stat-label { font-size: 13px; line-height: 18px; color: #707170; }
.pgw-stat-value { font-size: 20px; line-height: 28px; font-weight: 600; color: #171817; margin-top: 4px; font-variant-numeric: tabular-nums; }
.pgw-chart { display: block; width: 100%; height: auto; margin-top: 20px; }
.pgw-chart text { font-family: inherit; font-size: 13px; fill: #707170; }
.pgw-legend { display: flex; flex-wrap: wrap; gap: 4px 16px; font-size: 13px; line-height: 20px; color: #707170; margin-top: 8px; }
.pgw-swatch { display: inline-block; width: 10px; height: 10px; border-radius: 3px; margin-right: 6px; vertical-align: -1px; box-shadow: inset 0 0 0 1px rgba(10, 11, 10, 0.08); }
.pgw-line { display: inline-block; width: 16px; height: 0; margin-right: 6px; vertical-align: 3px; border-top: 2px solid; }
.pgw-verdict { font-size: 16px; line-height: 24px; font-weight: 600; margin-top: 16px; }
.pgw-note { font-size: 14px; line-height: 22px; color: #707170; margin-top: 12px; }
.pgw-note strong { color: #252625; font-weight: 600; }
@media (max-width: 640px) { .pgw { padding: 16px; } }
`;
  const Card = ({title, description, children}) => <div className="not-prose pgw">
      <style>{CSS}</style>
      <div className="pgw-eyebrow">Try it</div>
      <div className="pgw-title">{title}</div>
      {description ? <div className="pgw-desc">{description}</div> : null}
      {children}
    </div>;
  const Legend = ({items}) => <div className="pgw-legend">
      {items.map(([label, color, kind]) => <span key={label}>
          <span className={kind === "line" || kind === "dashed" ? "pgw-line" : "pgw-swatch"} style={kind === "line" || kind === "dashed" ? {
    borderTopColor: color,
    borderTopStyle: kind === "dashed" ? "dashed" : "solid"
  } : {
    background: color
  }} />
          {label}
        </span>)}
    </div>;
  const ERFC = [-1.26551223, 1.00002368, 0.37409196, 0.09678418, -0.18628806, 0.27886807, -1.13520398, 1.48851587, -0.82215223, 0.17087277];
  const erfc = x => {
    const z = Math.abs(x);
    const t = 1 / (1 + 0.5 * z);
    let poly = 0;
    for (let i = ERFC.length - 1; i >= 0; i--) poly = ERFC[i] + t * poly;
    const tail = t * Math.exp(-z * z + poly);
    return x >= 0 ? tail : 2 - tail;
  };
  const normalCdf = z => {
    if (z === Infinity) return 1;
    if (z === -Infinity) return 0;
    return 0.5 * erfc(-z / Math.SQRT2);
  };
  const normalPdf = z => 0.3989422804014327 * Math.exp(-z * z / 2);
  const QA = [-3.969683028665376e1, 2.209460984245205e2, -2.759285104469687e2, 1.38357751867269e2, -3.066479806614716e1, 2.506628277459239];
  const QB = [-5.447609879822406e1, 1.615858368580409e2, -1.556989798598866e2, 6.680131188771972e1, -1.328068155288572e1];
  const QC = [-7.784894002430293e-3, -3.223964580411365e-1, -2.400758277161838, -2.549732539343734, 4.374664141464968, 2.938163982698783];
  const QD = [7.784695709041462e-3, 3.224671290700398e-1, 2.445134137142996, 3.754408661907416];
  const lowerTail = p => {
    const q = Math.sqrt(-2 * Math.log(p));
    return (((((QC[0] * q + QC[1]) * q + QC[2]) * q + QC[3]) * q + QC[4]) * q + QC[5]) / ((((QD[0] * q + QD[1]) * q + QD[2]) * q + QD[3]) * q + 1);
  };
  const normalQuantile = p => {
    if (p <= 0) return -Infinity;
    if (p >= 1) return Infinity;
    if (p < 0.02425) return lowerTail(p);
    if (p > 1 - 0.02425) return -lowerTail(1 - p);
    const q = p - 0.5;
    const r = q * q;
    return (((((QA[0] * r + QA[1]) * r + QA[2]) * r + QA[3]) * r + QA[4]) * r + QA[5]) * q / (((((QB[0] * r + QB[1]) * r + QB[2]) * r + QB[3]) * r + QB[4]) * r + 1);
  };
  const betaRate = (conversions, visitors) => {
    const a = 1 + conversions;
    const b = 1 + Math.max(0, visitors - conversions);
    const total = a + b;
    return {
      mean: a / total,
      variance: a * b / (total * total * (total + 1))
    };
  };
  const bayesianEvidence = (cv, cc, vv, vc, prior, width) => {
    const control = betaRate(cc, cv);
    const variation = betaRate(vc, vv);
    const ratio = variation.mean / control.mean;
    const liftVariance = ratio * ratio * (variation.variance / (variation.mean * variation.mean) + control.variance / (control.mean * control.mean));
    let mean = ratio - 1;
    let sd = Math.sqrt(liftVariance);
    let chance;
    let lossShip;
    if (prior === "skeptical") {
      const dataPrecision = 1 / liftVariance;
      const precision = dataPrecision + 1 / (width * width);
      mean = mean * dataPrecision / precision;
      sd = 1 / Math.sqrt(precision);
      chance = normalCdf(mean / sd);
      lossShip = control.mean * (sd * normalPdf(mean / sd) - mean * normalCdf(-mean / sd));
    } else {
      const diff = variation.mean - control.mean;
      const diffSd = Math.sqrt(variation.variance + control.variance);
      chance = normalCdf(diff / diffSd);
      lossShip = diffSd * normalPdf(diff / diffSd) - diff * normalCdf(-diff / diffSd);
    }
    const inBand = band => normalCdf((band - mean) / sd) - normalCdf((-band - mean) / sd);
    return {
      mean,
      sd,
      chance,
      lossShip,
      controlRate: control.mean,
      inBand
    };
  };
  const pooledZ = (cv, cc, vv, vc) => {
    if (cv <= 0 || vv <= 0) return 0;
    const pooled = (cc + vc) / (cv + vv);
    const se = Math.sqrt(pooled * (1 - pooled) * (1 / cv + 1 / vv));
    return se === 0 ? 0 : (vc / vv - cc / cv) / se;
  };
  const correctedRate = (c, n) => c <= 0 || c >= n ? (c + 0.5) / (n + 1) : c / n;
  const relativeLiftInterval = (cv, cc, vv, vc, level) => {
    const pc = correctedRate(cc, cv);
    const pv = correctedRate(vc, vv);
    const ratio = pv / pc;
    const se = ratio * Math.sqrt(pv * (1 - pv) / vv / (pv * pv) + pc * (1 - pc) / cv / (pc * pc));
    const lift = cc > 0 ? vc / vv / (cc / cv) - 1 : ratio - 1;
    const z = normalQuantile(1 - (1 - level) / 2);
    return {
      lower: lift - z * se,
      upper: lift + z * se
    };
  };
  const INTERVALS = 160;
  const simpsonWeight = i => i === 0 || i === INTERVALS ? 1 : i % 2 === 1 ? 4 : 2;
  const obrienFleming = (alpha, t) => {
    if (!(t > 0)) return 0;
    if (t >= 1) return alpha;
    return Math.min(alpha, 2 - 2 * normalCdf(normalQuantile(1 - alpha / 2) / Math.sqrt(t)));
  };
  const crossing = (prev, c, t) => {
    if (!prev) return 2 * normalCdf(-c / Math.sqrt(t));
    const sd = Math.sqrt(t - prev.t);
    const h = 2 * prev.w / INTERVALS;
    let total = 0;
    for (let i = 0; i <= INTERVALS; i++) {
      const y = -prev.w + i * h;
      total += simpsonWeight(i) * prev.d[i] * (normalCdf((-c - y) / sd) + normalCdf((y - c) / sd));
    }
    return total * h / 3;
  };
  const propagate = (prev, w, t, spent) => {
    const d = new Float64Array(INTERVALS + 1);
    const step = 2 * w / INTERVALS;
    if (!prev) {
      const sd = Math.sqrt(t);
      for (let j = 0; j <= INTERVALS; j++) d[j] = normalPdf((-w + j * step) / sd) / sd;
      return {
        t,
        w,
        d,
        spent
      };
    }
    const sd = Math.sqrt(t - prev.t);
    const h = 2 * prev.w / INTERVALS;
    for (let j = 0; j <= INTERVALS; j++) {
      const x = -w + j * step;
      let total = 0;
      for (let i = 0; i <= INTERVALS; i++) {
        total += simpsonWeight(i) * prev.d[i] * (normalPdf((x - (-prev.w + i * h)) / sd) / sd);
      }
      d[j] = total * h / 3;
    }
    return {
      t,
      w,
      d,
      spent
    };
  };
  const spendingBoundaries = (alpha, fractions) => {
    const result = [];
    let prev = null;
    for (const requested of fractions) {
      const t = Math.min(1, Math.max(prev ? prev.t : 0, requested));
      if (t - (prev ? prev.t : 0) < 1e-9) {
        result.push(Infinity);
        continue;
      }
      const spent = obrienFleming(alpha, t);
      const increment = Math.max(0, spent - (prev ? prev.spent : 0));
      const maxW = 8 * Math.sqrt(t);
      let w;
      if (increment <= 0) w = maxW; else if (!prev) w = Math.min(maxW, Math.sqrt(t) * normalQuantile(1 - increment / 2)); else if (crossing(prev, maxW, t) >= increment) w = maxW; else {
        let lo = 0;
        let hi = maxW;
        for (let s = 0; s < 40; s++) {
          const mid = (lo + hi) / 2;
          if (crossing(prev, mid, t) > increment) lo = mid; else hi = mid;
        }
        w = (lo + hi) / 2;
      }
      result.push(increment <= 0 ? Infinity : w / Math.sqrt(t));
      prev = propagate(prev, w, t, spent);
    }
    return result;
  };
  const pct = (value, digits = 1) => Number.isFinite(value) ? `${(value * 100).toFixed(digits)} %` : "–";
  const signedPct = (value, digits = 1) => Number.isFinite(value) ? `${value >= 0 ? "+" : "−"}${Math.abs(value * 100).toFixed(digits)} %` : "–";
  const toNumber = (value, fallback) => {
    const parsed = Number(value);
    return Number.isFinite(parsed) ? parsed : fallback;
  };
  const NumberField = ({label, value, onChange, min, max, step, suffix}) => <label className="pgw-field">
      <span className="pgw-label">
        {label}
        {suffix ? <span> ({suffix})</span> : null}
      </span>
      <input type="number" inputMode="decimal" value={value} min={min} max={max} step={step} onChange={event => onChange(event.target.value)} />
    </label>;
  const SelectField = ({label, value, onChange, options}) => <label className="pgw-field">
      <span className="pgw-label">{label}</span>
      <select value={value} onChange={event => onChange(event.target.value)}>
        {options.map(option => <option key={option.value} value={option.value}>
            {option.label}
          </option>)}
      </select>
    </label>;
  const Stat = ({label, value, color}) => <div className="pgw-stat">
      <div className="pgw-stat-label">{label}</div>
      <div className="pgw-stat-value" style={color ? {
    color
  } : undefined}>{value}</div>
    </div>;
  const mulberry32 = seed => () => {
    seed |= 0;
    seed = seed + 0x6d2b79f5 | 0;
    let t = Math.imul(seed ^ seed >>> 15, 1 | seed);
    t = t + Math.imul(t ^ t >>> 7, 61 | t) ^ t;
    return ((t ^ t >>> 14) >>> 0) / 4294967296;
  };
  const gaussian = rng => {
    const u = Math.max(rng(), 1e-12);
    return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * rng());
  };
  const drawConversions = (rng, visitors, rate) => {
    const mean = visitors * rate;
    if (mean < 30 && rate < 0.1) {
      const limit = Math.exp(-mean);
      let k = 0;
      let p = rng();
      while (p > limit) {
        k++;
        p *= rng();
      }
      return k;
    }
    return Math.min(Math.floor(visitors), Math.max(0, Math.round(mean + Math.sqrt(mean * (1 - rate)) * gaussian(rng))));
  };
  const LEVELS = [{
    id: "explore",
    label: "Explore",
    odds: "1 in 7",
    rate: 0.15,
    threshold: 0.93,
    significance: 0.3,
    minDays: 7,
    maxDays: 14,
    minConversions: 30,
    band: 0.15
  }, {
    id: "fast",
    label: "Fast",
    odds: "1 in 10",
    rate: 0.1,
    threshold: 0.96,
    significance: 0.2,
    minDays: 7,
    maxDays: 14,
    minConversions: 50,
    band: 0.1
  }, {
    id: "balanced",
    label: "Balanced",
    odds: "1 in 20",
    rate: 0.05,
    threshold: 0.985,
    significance: 0.1,
    minDays: 7,
    maxDays: 21,
    minConversions: 100,
    band: 0.075
  }, {
    id: "careful",
    label: "Careful",
    odds: "1 in 40",
    rate: 0.025,
    threshold: 0.995,
    significance: 0.05,
    minDays: 7,
    maxDays: 28,
    minConversions: 200,
    band: 0.05
  }, {
    id: "strict",
    label: "Strict",
    odds: "1 in 100",
    rate: 0.01,
    threshold: 0.9975,
    significance: 0.02,
    minDays: 14,
    maxDays: 35,
    minConversions: 400,
    band: 0.03
  }];
  const simulate = options => {
    const {engine, runs, dailyVisitors, rate, trueLift, checksPerDay, minDays, maxDays, minConversions, ropeBand, threshold, prior, priorWidth, significance, correction, oneSided = false, stopLosing = 0.05} = options;
    const rng = mulberry32(1455);
    const slots = Math.round(maxDays * checksPerDay);
    const perSlot = dailyVisitors / checksPerDay;
    const fractions = [];
    for (let s = 1; s <= slots; s++) fractions.push(s / slots);
    const boundaries = engine === "frequentist" && correction ? spendingBoundaries(significance, fractions) : null;
    const fixedZ = normalQuantile(1 - significance / 2);
    const equivalenceLevel = 1 - 2 * significance;
    const counts = {
      win: 0,
      loss: 0,
      equivalent: 0,
      timeout: 0
    };
    const stopDays = {
      win: [],
      loss: [],
      equivalent: [],
      timeout: []
    };
    for (let run = 0; run < runs; run++) {
      let cv = 0;
      let cc = 0;
      let vv = 0;
      let vc = 0;
      let outcome = "timeout";
      let stopDay = maxDays;
      for (let s = 1; s <= slots; s++) {
        cv += perSlot;
        vv += perSlot;
        cc += drawConversions(rng, perSlot, rate);
        vc += drawConversions(rng, perSlot, rate * (1 + trueLift));
        const day = s / checksPerDay;
        const atHorizon = s === slots;
        const enoughData = cc + vc >= minConversions && day >= minDays;
        if (!enoughData && !atHorizon) continue;
        let winSide = 0;
        let equivalent = false;
        if (engine === "bayesian") {
          const evidence = bayesianEvidence(cv, cc, vv, vc, prior, priorWidth);
          if (evidence.chance >= threshold) winSide = 1; else if (evidence.chance <= (oneSided ? stopLosing : 1 - threshold)) winSide = -1;
          equivalent = ropeBand > 0 && evidence.inBand(ropeBand) >= 0.95;
        } else {
          const z = pooledZ(cv, cc, vv, vc);
          const critical = boundaries ? boundaries[s - 1] : fixedZ;
          if (Math.abs(z) >= critical) winSide = z > 0 ? 1 : -1;
          if (ropeBand > 0) {
            const interval = relativeLiftInterval(cv, cc, vv, vc, equivalenceLevel);
            equivalent = interval.lower > -ropeBand && interval.upper < ropeBand;
          }
        }
        if (atHorizon) {
          outcome = enoughData && winSide !== 0 ? winSide > 0 ? "win" : "loss" : "timeout";
          stopDay = day;
          break;
        }
        if (equivalent) {
          outcome = "equivalent";
          stopDay = day;
          break;
        }
        if (winSide !== 0) {
          outcome = winSide > 0 ? "win" : "loss";
          stopDay = day;
          break;
        }
      }
      counts[outcome]++;
      stopDays[outcome].push(stopDay);
    }
    const histogram = [];
    for (let day = 1; day <= Math.ceil(maxDays); day++) histogram.push({
      day,
      win: 0,
      loss: 0,
      equivalent: 0,
      timeout: 0
    });
    for (const key of Object.keys(stopDays)) {
      for (const value of stopDays[key]) {
        const index = Math.min(histogram.length - 1, Math.max(0, Math.ceil(value) - 1));
        histogram[index][key]++;
      }
    }
    const all = [].concat(stopDays.win, stopDays.loss, stopDays.equivalent, stopDays.timeout);
    const averageDays = all.reduce((sum, value) => sum + value, 0) / Math.max(1, all.length);
    return {
      counts,
      runs,
      histogram,
      averageDays
    };
  };
  return {
    COLORS,
    Card,
    Legend,
    normalCdf,
    normalPdf,
    normalQuantile,
    bayesianEvidence,
    spendingBoundaries,
    pct,
    signedPct,
    toNumber,
    NumberField,
    SelectField,
    Stat,
    simulate,
    LEVELS
  };
})();

export const FalseWinnerSimulator = ({engine: initialEngine = "bayesian"}) => {
  const {COLORS, Card, Legend, pct, toNumber, NumberField, SelectField, Stat, simulate, LEVELS} = StatisticsKit;
  const balanced = LEVELS.find(level => level.id === "balanced");
  const [engine, setEngine] = useState(initialEngine);
  const [level, setLevel] = useState("balanced");
  const [visitors, setVisitors] = useState("1000");
  const [rate, setRate] = useState("3");
  const [lift, setLift] = useState("0");
  const [threshold, setThreshold] = useState(String(balanced.threshold * 100));
  const [policy, setPolicy] = useState("one");
  const [stopLosing, setStopLosing] = useState("5");
  const [prior, setPrior] = useState("skeptical");
  const [width, setWidth] = useState("30");
  const [significance, setSignificance] = useState(String(balanced.significance * 100));
  const [correction, setCorrection] = useState("on");
  const [checks, setChecks] = useState("1");
  const [minDays, setMinDays] = useState(String(balanced.minDays));
  const [maxDays, setMaxDays] = useState(String(balanced.maxDays));
  const [minConversions, setMinConversions] = useState(String(balanced.minConversions));
  const [band, setBand] = useState(String(balanced.band * 100));
  const chooseLevel = id => {
    setLevel(id);
    const chosen = LEVELS.find(entry => entry.id === id);
    if (!chosen) return;
    setThreshold(String(chosen.threshold * 100));
    setSignificance(String(chosen.significance * 100));
    setMinDays(String(chosen.minDays));
    setMaxDays(String(chosen.maxDays));
    setMinConversions(String(chosen.minConversions));
    setBand(String(chosen.band * 100));
    setPolicy("one");
    setStopLosing("5");
    setPrior("skeptical");
    setWidth("30");
    setCorrection("on");
  };
  const custom = setter => value => {
    setLevel("custom");
    setter(value);
  };
  const oneSided = engine === "bayesian" && policy === "one";
  const options = {
    engine,
    runs: 1000,
    dailyVisitors: Math.max(10, toNumber(visitors, 1000)),
    rate: Math.min(0.9, Math.max(0.001, toNumber(rate, 3) / 100)),
    trueLift: Math.max(-0.9, toNumber(lift, 0) / 100),
    checksPerDay: Number(checks),
    minDays: Math.max(0, toNumber(minDays, 7)),
    maxDays: Math.min(60, Math.max(1, Math.round(toNumber(maxDays, 21)))),
    minConversions: Math.max(0, toNumber(minConversions, 100)),
    ropeBand: Math.max(0, toNumber(band, 7.5) / 100),
    threshold: Math.min(0.9999, Math.max(0.5, toNumber(threshold, 98.5) / 100)),
    oneSided,
    stopLosing: Math.min(0.49, Math.max(0.001, toNumber(stopLosing, 5) / 100)),
    prior,
    priorWidth: Math.max(0.01, toNumber(width, 30) / 100),
    significance: Math.min(0.5, Math.max(0.001, toNumber(significance, 10) / 100)),
    correction: correction === "on"
  };
  const result = useMemo(() => simulate(options), [JSON.stringify(options)]);
  const share = key => result.counts[key] / result.runs;
  const noEffect = options.trueLift === 0;
  const lossLabel = oneSided ? "Stopped: losing" : noEffect ? "Loss (false loser)" : "Loss";
  const W = 520;
  const H = 180;
  const pad = {
    l: 12,
    r: 12,
    t: 12,
    b: 30
  };
  const maxBar = Math.max(1, ...result.histogram.map(b => b.win + b.loss + b.equivalent + b.timeout));
  const barWidth = (W - pad.l - pad.r) / result.histogram.length;
  const order = [["win", COLORS.win], ["loss", COLORS.loss], ["equivalent", COLORS.neutral], ["timeout", COLORS.timeout]];
  return <Card title="Test simulator" description="Runs 1,000 tests with your settings right in your browser.">
      <div className="pgw-fields">
        <SelectField label="Method" value={engine} onChange={setEngine} options={[{
    value: "bayesian",
    label: "Bayesian"
  }, {
    value: "frequentist",
    label: "Frequentist"
  }]} />
        <SelectField label="Level" value={level} onChange={chooseLevel} options={[...LEVELS.map(entry => ({
    value: entry.id,
    label: `${entry.label} (${entry.odds})`
  })), {
    value: "custom",
    label: "Custom settings"
  }]} />
        <NumberField label="Visitors per variation per day" value={visitors} onChange={setVisitors} min={10} step={100} />
        <NumberField label="Control conversion rate" suffix="%" value={rate} onChange={setRate} min={0.1} max={90} step={0.5} />
        <NumberField label="True lift of the variation" suffix="%" value={lift} onChange={setLift} step={1} />
        {engine === "bayesian" ? <>
            <NumberField label="Chance to beat control threshold" suffix="%" value={threshold} onChange={custom(setThreshold)} min={50} max={99.99} step={0.5} />
            <SelectField label="Decision policy" value={policy} onChange={custom(setPolicy)} options={[{
    value: "one",
    label: "One-sided"
  }, {
    value: "two",
    label: "Two-sided"
  }]} />
            {policy === "one" ? <NumberField label="Stop losing variations at" suffix="%" value={stopLosing} onChange={custom(setStopLosing)} min={0.1} max={49} step={1} /> : null}
            <SelectField label="Prior" value={prior} onChange={custom(setPrior)} options={[{
    value: "skeptical",
    label: "Skeptical"
  }, {
    value: "flat",
    label: "Flat"
  }]} />
            {prior === "skeptical" ? <NumberField label="Prior width" suffix="%" value={width} onChange={custom(setWidth)} min={1} max={500} step={5} /> : null}
          </> : <>
            <NumberField label="Significance level (two-sided)" suffix="%" value={significance} onChange={custom(setSignificance)} min={0.1} max={50} step={0.5} />
            <SelectField label="Sequential correction" value={correction} onChange={custom(setCorrection)} options={[{
    value: "on",
    label: "On"
  }, {
    value: "off",
    label: "Off"
  }]} />
          </>}
        <SelectField label="Automatic checks per day" value={checks} onChange={setChecks} options={[{
    value: "1",
    label: "Once a day"
  }, {
    value: "2",
    label: "Twice a day"
  }, {
    value: "4",
    label: "Every 6 hours"
  }, {
    value: "24",
    label: "Every hour"
  }]} />
        <NumberField label="Minimum runtime" suffix="days" value={minDays} onChange={custom(setMinDays)} min={0} step={1} />
        <NumberField label="Maximum runtime" suffix="days" value={maxDays} onChange={custom(setMaxDays)} min={1} max={60} step={1} />
        <NumberField label="Minimum total conversions" value={minConversions} onChange={custom(setMinConversions)} min={0} step={10} />
        <NumberField label="Relative ROPE band" suffix="% · 0 = off" value={band} onChange={custom(setBand)} min={0} step={0.5} />
      </div>
      <div className="pgw-stats">
        <Stat label={noEffect ? "Win (false winner)" : "Win"} value={pct(share("win"))} color={COLORS.win} />
        <Stat label={lossLabel} value={pct(share("loss"))} color={COLORS.loss} />
        <Stat label="Inconclusive: no meaningful difference" value={pct(share("equivalent"))} />
        <Stat label="Inconclusive at maximum runtime" value={pct(share("timeout"))} />
        <Stat label="Average runtime" value={`${result.averageDays.toFixed(1)} days`} />
      </div>
      <div className="pgw-label" style={{
    marginTop: 20
  }}>Day each simulated test ended</div>
      <svg viewBox={`0 0 ${W} ${H}`} className="pgw-chart" style={{
    marginTop: 8
  }} role="img" aria-label="Day on which simulated tests ended">
        <line x1={pad.l} x2={W - pad.r} y1={H - pad.b} y2={H - pad.b} stroke={COLORS.grid} />
        {result.histogram.map((bucket, index) => {
    let offset = 0;
    return <g key={bucket.day}>
              {order.map(([key, color]) => {
      const height = bucket[key] / maxBar * (H - pad.t - pad.b);
      offset += height;
      return height > 0 ? <rect key={key} x={pad.l + index * barWidth + 1} y={H - pad.b - offset} width={Math.max(1, barWidth - 2)} height={height} fill={color} /> : null;
    })}
              {bucket.day === 1 || bucket.day % 7 === 0 ? <text x={pad.l + (index + 0.5) * barWidth} y={H - 12} textAnchor="middle">
                  {bucket.day}
                </text> : null}
            </g>;
  })}
      </svg>
      <Legend items={[["Win", COLORS.win], [oneSided ? "Stopped: losing" : "Loss", COLORS.loss], ["No meaningful difference", COLORS.neutral], ["Inconclusive at maximum runtime", COLORS.timeout]]} />
      <p className="pgw-note">
        {noEffect ? `With a true lift of 0 %, every win is a false winner.${oneSided ? " Stopped tests are not results, so they add no false losers." : " Every loss is a false loser."} Set a true lift to see how often a real effect is found.` : "With a real lift, the win share is how often the test finds it before its maximum runtime."}{" "}
        The simulation assumes the same traffic every day and one variation. Real traffic changes by weekday and season, so treat the numbers as a guide.
      </p>
    </Card>;
};

export const StandardMapper = () => {
  const {COLORS, Card, pct, toNumber, NumberField, SelectField, Stat, LEVELS} = StatisticsKit;
  const [confidence, setConfidence] = useState("95");
  const [sides, setSides] = useState("two");
  const [variations, setVariations] = useState("1");
  const level = Math.min(99.99, Math.max(50, toNumber(confidence, 95))) / 100;
  const count = Math.max(1, Math.round(toNumber(variations, 1)));
  const winSideRisk = sides === "two" ? (1 - level) / 2 : 1 - level;
  const match = LEVELS.find(entry => entry.rate <= winSideRisk + 1e-9) ?? null;
  const significance = Math.min(0.5, 2 * winSideRisk);
  return <Card title="Match your standard" description="Enter your company's testing rule to find the level that keeps it.">
      <div className="pgw-fields">
        <NumberField label="Your confidence level" suffix="%" value={confidence} onChange={setConfidence} min={50} max={99.99} step={0.5} />
        <SelectField label="Your test is" value={sides} onChange={setSides} options={[{
    value: "two",
    label: "Two-sided"
  }, {
    value: "one",
    label: "One-sided"
  }]} />
        <NumberField label="Variations besides control" value={variations} onChange={setVariations} min={1} max={10} step={1} />
      </div>
      <div className="pgw-stats">
        <Stat label="False winners your standard allows" value={pct(winSideRisk, 2)} />
        <Stat label="Level to choose" value={match ? match.label : "Custom"} color={COLORS.ink} />
        <Stat label="Or exactly: frequentist, two-sided" value={`${pct(significance, 2)} significance`} />
      </div>
      <p className="pgw-note">
        {match ? `${match.label} promises at most ${match.odds} false winners (${pct(match.rate, 1)}), within your ${pct(winSideRisk, 2)}. A faster level would allow more.` : `Your standard is stricter than Strict (1 in 100). Choose Strict, or set the frequentist significance level to ${pct(significance, 2)} under Advanced settings.`}{" "}
        {count > 1 ? `With ${count} variations, pagent tightens the bar for each comparison automatically, so the promise holds for the whole test.` : "pagent tightens the bar automatically when a test has several variations."}
      </p>
    </Card>;
};

Every statistics setup trades **speed** against **certainty**. A looser bar ends tests sooner and finds more winners, but more of them are luck. A stricter bar calls fewer false winners, but tests need more visitors.

pagent packs this trade-off into five levels. Pick one under **Settings → Tests → Statistics**.

## Pick a level

| Level | False winners at most | Choose it when |
| - | - | - |
| **Explore** | 1 in 7 | You screen many ideas and only care about big wins. Wrong calls are cheap to undo. |
| **Fast** | 1 in 10 | You test copy and layout changes that are easy to undo. |
| **Balanced** (default) | 1 in 20 | Most product and landing page tests. |
| **Careful** | 1 in 40 | A wrong call costs money: checkout, pricing, forms. Matches a classic "95 % confidence, two-sided" standard. |
| **Strict** | 1 in 100 | Big redesigns, or results you report to leadership. Runs at least two weeks. |

A few rules of thumb:

* **Small lifts need a strict level.** Explore and Fast treat lifts below ±15 % and ±10 % as "no meaningful difference". If a 5 % lift would matter to you, use Balanced or stricter.
* **Low traffic needs patience, not a looser level.** Below a few hundred visitors per variation per day, few tests find a 5 % lift at any level. Test bolder changes, or longer: raise the maximum runtime under Advanced settings.
* **Pick the method by how you read results.** Bayesian shows a chance to win; frequentist shows p-values. Each level keeps the same promise with both.

[Statistics settings](/docs/statistics/settings#statistics-levels) lists the exact values each level sets.

## Match your company's standard

Many teams have a rule like "we test at 95 % confidence". Enter yours to find the level that keeps it:

<StandardMapper />

The common cases:

| Your standard | False winners it allows | Level |
| - | - | - |
| 90 %, two-sided | 5 % | Balanced |
| 95 %, two-sided | 2.5 % | Careful |
| 99 %, two-sided | 0.5 % | Stricter than Strict: use frequentist at 1 % under Advanced settings |
| 90 %, one-sided | 10 % | Fast |
| 95 %, one-sided | 5 % | Balanced |

## One-sided and two-sided standards

A **two-sided** standard checks for wins and losses with the same bar. "95 % confidence, two-sided" allows 2.5 % false winners and 2.5 % false losers.

A **one-sided** standard only checks for wins. "95 % confidence, one-sided" allows 5 % false winners.

pagent's Bayesian levels are one-sided: only a win counts as a result, and a clearly losing variation stops the test without being recorded as a loss. The frequentist levels are two-sided. Either way, the level promises the share of false winners, so you can compare it directly with your standard. See [Decision policy](/docs/statistics/settings#decision-policy).

<Warning>
  A chance to beat control is not the same as "confidence". A Bayesian threshold of 95 % with daily checks calls more false winners than a 95 % confidence standard. The levels take this into account; a custom threshold does not.
</Warning>

## Several variations

You do not need to adjust anything for the number of variations. pagent tightens the bar for each comparison automatically, for both methods, so the level's promise holds for the whole test. See [Variations and goals](/docs/statistics/variations-and-goals#tests-with-several-variations).

## What your traffic can detect

pagent has no power or sample-size setting. To see what your traffic can find, use the simulator with a real lift: the win share is how often a test finds that lift before its maximum runtime. If the share is low, test bolder changes, raise the maximum runtime, or choose a page with more traffic.

<FalseWinnerSimulator engine="bayesian" />

## Changing your setup

1. Go to **Settings → Tests**. Under **Statistics**, choose the method and the level.
2. Open **Advanced settings** only if you need a rule the levels do not cover. Changing a value there shows **Custom settings**.
3. Click **Save changes**.
4. Open a test and choose **View test rules** to see which level it runs on.

Running tests that follow the website's settings switch to the new level from their next check on. Change your setup between tests where you can: changing the rules of a running test, especially after looking at its results, weakens what the result means.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.