A machine-checked 'anytime' risk ledger halved exact-fallbacks in live model serving while preserving per-request guarantees, turning runtime compression risk into an auditable, spendable account; the remaining gap to observed errors is localized to the gate operating point, enabling explicit pricing of quality–capacity trade-offs.

Main Finding

The authors replace union-bound-based, horizon-priced risk accounting for runtime compression with an anytime-valid, cumulative-loss admission ledger that (a) is machine-checked in Lean4, (b) holds on hundreds of thousands of live admission calls, and (c) meaningfully reduces fallbacks to exact computation at matched declared risk in a held-out confirmatory run (fallback 0.30 → 0.14). They derive a machine-checked chain from per-step moment bounds to a request-level served-output total-variation (TV) tail bound, provide a design “law” that turns the served-TV target into an operator knob, and empirically localize the remaining numerical gap in the certified witness to the gate’s operating point (the deployment choice) rather than unknown mathematics. They also replace vacuous conformal-certificates with exchangeable-history order-statistic extrapolation that discriminates calibration risk.

Key Points

• Problem: Runtime compression (e.g., KV-write quantization, MoE serving) adapts precision to load without per-request soundness; prior certified approaches budget risk by a union bound over pre-declared event counts — which is impractical for free-running/adaptive serving.
• Union-bound failure in practice: the union-budget formulation exhausts on every long request in the production stack (100%), so it cannot be used to make operational decisions.
• New object: cumulative realized-loss budget (cumloss admission). Admission decisions are made against an anytime-valid account on cumulative loss rather than per-event union allocation. This object is the thing you can actually “spend.”
• Machine-checked guarantee: Theorem (machine-checked in Lean4) links per-step moment bounds to an expected per-step TV bound and, through Bernoulli domination and Ville/Doob arguments, to an anytime-valid request-level tail: Pr(∃t≤T ∑s≤t TVs > Bt) ≤ δ when the admission invariant (Eq. 6) is enforced.
• Live deployment and measurement:
  • Physical gate deployment: 317,293 and 352,333 admission calls in two arms (loose/tight).
  • No authorization violations (the account never reported exceeding admitted charge).
  • In a pre-registered held-out confirmatory round (same budget/λ frozen), the physical-arm fallback-to-exact rate improved from 0.30 (its union baseline) to 0.14 at matched risk.
  • The admission ledger incurs ~+3.8% judgment cost (wall time).
• Per-read certificates: inherited per-read bounds (ℓ∞-style) become vacuous at modest masking (e.g., vacuous on 41.8% of reads at m=3 masked mantissa bits). Replacing worst-coordinate bounds with mass-weighted bounds (Bhattacharyya or sub-Gaussian) produced non-vacuous per-request behavior.
• Three-layer audit of the step-budget instantiation:
  • Local operator-norm query envelope measured ~1.5× from tight (σmax ≈ 16.0).
  • Measured-ellipsoid replacement for Cauchy–Schwarz ball measured 0.89× (held-out sound).
  • Gate operating point ~700× — this is where the entire 1064× gap between certified witness and served output localizes.
• Quantifier/extrapolation: using exchangeable without-replacement sampling over 80 serving histories gives order-statistic bounds that discriminate calibration risk (0.41 vs 0.51) and replace binary conformal prediction’s vacuous certificates; the remaining important caveat is that the population is the constructed pool (not guaranteed to equal future runtime traffic).
• Formalization scale: 228 exported theorems in Lean4, no sorry-steps; probabilistic kernels are machine-checked.

Data & Methods

• Production-scale setting and models:
  • Models exercised include DeepSeek-V4-Flash-FP8 (256 experts, top-6), DeepSeek-V2-Lite (64/6), GLM-5.2 (256/8), Qwen1.5-MoE-A2.7B, Qwen2.5-7B, and others (Table 1 maps results to models).
  • Experiments include long-running admission calls on production stacks and pre-registered frozen confirmatory rounds; every gate and verdict was pre-registered.
• Core theoretical tools:
  • Per-step moment bounds on rounding noise (per-coordinate MGF-style bounds separated into deterministic bias and fluctuation proxy).
  • A mass-weighted inequality linking per-step moment bounds to expected per-step TV (Theorem 1: Eω[TV(ph, p′h,ω)] ≤ sqrt{log Σv ph(v)Bh(v)}).
  • Bernoulli domination and a martingale (Ville/Doob) argument turning the per-step MGF-driven bound into an anytime-valid cumulative-loss admission invariant (Eq. 6).
  • Practical computable approximations: mass-weighted top-k truncation (probability concentrated on a small S) to make the sum over vocabulary tractable.
• Empirical instantiation:
  • Five attempted routes to estimate step budget constants; four failed because they priced worst-case directions or assumptions; the retained route uses an empirical joint scalar/betting Cauchy–Schwarz functional that matches the quantity the theorem requires.
  • Measurements and diagnostics: sparsity/concentration of served distributions, operator norms, per-token bounded differences (cmax ≈ 3.81), realized served-TV medians vs bounds, and probe falsifications of first-order surrogate propagation constants.
  • Dual-accounted admission experiments (Table 2): compared union-budget baseline to cumulative-loss budgets at different tolerances; showed monotone tradeoff between budget and fallback rate but union budget always exhausts on long requests.
• Pre-conditions & caveats:
  • The chain is machine-checked up to measured propagation constants; some propagation constants are measured (and falsify simplistic surrogates) rather than given analytic tensorized bounds.
  • The exchangeable extrapolation relies on the sample/pool being representative (authors explicitly note the population is a constructed pool, not guaranteed equal to runtime traffic).
  • One frozen confirmatory replication (single replication), not a full hyper-sweep.

Implications for AI Economics

• Risk-priced runtime compression: the paper operationalizes “risk as money” for runtime precision decisions. Rather than a priori union-bounded risk that pays for an entire horizon, the cumulative-loss ledger buys coverage incrementally and transparently — letting operators trade risk for capacity with a readable price per token/request.
• Better utilization of capacity vs quality tradeoffs: by turning served-TV guarantees into a knob (via the machine-checked law and the audit that localizes the numeric gap to the gate operating point), operators can optimize the gate operating point (the real economic lever) with a known price in delivered risk.
• Quantifiable operational decisions: the admission account gives an anytime-valid guarantee that is usable by controllers to admit/delay/force-exact operations; this supports tighter SLAs and more aggressive compression policies when budgets allow, and a principled fall-back rule when budgets exhaust.
• Measurement-driven certification is essential: many theoretically appealing worst-case bounds are too loose for practice (vacuous certificates). Economically useful guarantees need (a) mass-weighting or variance-based functionals, (b) measured propagation constants, and (c) pre-registered, machine-adjudicated evaluation. This raises the bar for certifying production serving tradeoffs: you must pay for measurement and formalization effort.
• Policy and capacity planning:
  • The approach converts unknown model-side propagation slack into an explicit priced item (the gate’s operating point). That enables more accurate cost-benefit decisions when choosing compression levels across components (KV writes, expert weights, etc.).
  • The companion measurements suggest expert-weight precision could buy multi× capacity headroom; the cumloss admission engine is object-agnostic and can be extended to other components once measured.
• Limits and caution for economic adoption:
  • The guarantee depends on a representativeness assumption (exchangeable sample). If traffic distribution shifts in ways not covered by the constructed pool, the priced account may not cover the next request.
  • Some constants remain empirically measured (not analytically bounded); operators must budget for measurement and revalidation. The work shows how to localize the dominant pricing factor (the gate operating point) — but changing that operating point is an operational choice that directly drives costs and user-experience risk.
• Research takeaways:
  • For AI economics modeling and resource allocation, treat per-request risk as a budgeted commodity under an anytime-valid accounting regime rather than a horizon-paid fixed allocation.
  • Invest in machine-checked probabilistic kernels plus production measurement to make guarantees actionable and non-vacuous.
  • Design incentives and SLAs around spendable risk accounts; this can align compression/throughput optimizations with transparent economic tradeoffs.

Short takeaway: the paper provides a practically deployable, anytime-valid accounting mechanism (machine-checked and measured in production) that turns runtime compression risk into a priced, spendable object — enabling principled tradeoffs between capacity and quality — while making explicit the empirical measurements and assumptions operators must sustain for the guarantees to be meaningful.