Operating LRMs at a critical energy-provisioning point avoids systematic baseline or auxiliary energy waste; once balanced, further gains come from routing policies that absorb variability across time and models rather than from changing average supply.

Main Finding

Routing tasks among large reasoning models (LRMs) with attention to both which model to use and how long to run it can substantially reduce non-renewable (auxiliary) energy use in AI data centers. The paper shows there is a critical operating regime—where mean baseline supply matches mean demand—at which costs are not dominated by mean mismatches but by stochastic fluctuations; in that regime reserve (auxiliary) energy scales diffusively (∝√T) rather than linearly. This implies that variance-aware routing and dispatch (not just mean-optimal routing) is a principled and necessary design axis for energy-aware AI factories.

Key Points

• System model

  • Tasks arrive stochastically with latency and error-tolerance constraints; a dispatcher assigns each task a model and a thinking time (compute budget).
  • Each model Mi consumes energy as it runs; more thinking time typically increases success probability (reasoning-quality) and consumes more energy.
  • Energy supply is composed of stochastic baseline (harvested, e.g., renewables) and auxiliary (non-renewable) injections; stored energy cannot go negative.
  • Objective: online policy to meet task deadlines/accuracy while minimizing expected cumulative auxiliary energy.

• Analytical framing

  • A pathwise identity (Theorem 1) shows cumulative auxiliary injections equal the maximal deficit of the unconstrained battery trajectory; thus dispatch decisions enter only through the induced consumption process {Ct}.
  • Define a per-task arrival-feasible energy lower bound ELB(X) (minimum energy to satisfy the task at arrival using the cheapest feasible model).
  • A myopic baseline policy routes each task immediately to the arrival-feasible minimal-energy model; this gives a tractable, conservative reference.

• Drift–fluctuation regimes (Theorem 2)

  • Let µ be mean drift (baseline supply minus mean demand) and σ the volatility. Long-horizon expected auxiliary energy exhibits three regimes:
    • µ < 0 (persistent deficit): E[aux] ≈ |µ| T + σ^2/(2|µ|) — linear growth in T (drift-dominated).
    • µ > 0 (persistent surplus): E[aux] ≈ σ^2/(2µ) — bounded (over-supply, wasteful baseline).
    • µ = 0 (critical/balanced): E[aux] ≈ σ √(2T/π) — scales like √T (fluctuation-dominated).
  • Practical designs are often tuned near µ ≈ 0 to avoid systematic waste or large persistent auxiliary draw, making stochastic fluctuations the leading cost driver.

• Variance-aware routing

  • Dispatch-induced per-task excess energy ∆E (routing error) shifts the drift; large systematic routing errors lead to drift-dominated costs.
  • When routing errors are comparable to fluctuation scale (σ√T), second-order (variance) effects matter: policies should trade small increases in mean energy for reductions in variance (variance-aware routing).
  • Dispatcher overhead (energy and latency to make routing decisions) trades off against routing accuracy: more sophisticated dispatchers can reduce ∆E but incur their own energy/latency that may shrink feasible model choices.

• Compute-scaling grounding

  • The framework is linked to empirical training- and inference-compute scaling laws for transformer-style LRMs: success probability and energy as functions of model size and thinking time are used to estimate ELB and predict energy–quality tradeoffs.
  • Oracle stopping is assumed (stop immediately when told), isolating routing-layer limits.

Data & Methods

• Models and assumptions

  • Unlimited parallel processing capacity (only energy is binding).
  • Task arrivals: Poisson batches; task attributes i.i.d.
  • Baseline harvested energy {Rt}: i.i.d. with mean R and variance σ_R^2 (used as a short-timescale stochastic residual after forecasting predictable components).
  • Dispatcher is online (nonanticipative); can observe current stored energy and queued tasks (may also observe harvest-side covariates).
  • Oracle stopping: chosen thinking time is strictly enforced and stops immediately—no early/heuristic stopping modeled.
  • Abstractions ignore batching mechanics, preemption, engine-level scheduling complexities; these can be incorporated but are left out for tractability.

• Analytical methods

  • Battery dynamics: unconstrained random-walk Bt+1 = Bt + Rt − Ct; constrained trajectory with auxiliary injections equivalently characterized via the running minimum of Bt (Theorem 1).
  • Lower-bound energy per task (ELB) defined using model-specific minimum thinking times and per-step energy rates.
  • Myopic policy analysis: two views (lumped-at-arrival and distributed-in-service); lumped version used as conservative reference.
  • Diffusion approximation: via Donsker’s invariance principle, the rescaled battery process converges to Brownian motion; used to derive closed-form asymptotics for expected auxiliary energy (Theorem 2).
  • Quantitative tradeoffs: compare deterministic drift effects (from mean routing error) vs. stochastic fluctuation effects (variance) to identify when to prioritize mean-optimality vs. variance-reduction.

• Proofs and lemmas

  • Theorems and lemmas provide (i) identity between cumulative auxiliary injections and running minimum of unconstrained battery, (ii) variance and drift expressions for the myopic battery random walk, (iii) diffusion-limit derivations and explicit expected-deficit formulas.

Implications for AI Economics

• Energy provisioning and capacity planning

  • Targeting mean balance (µ ≈ 0) between renewable baseline supply and expected LRM demand minimizes systematic auxiliary consumption, but exposes the system to fluctuation-driven reserve needs that grow as √T.
  • Planners must size battery/reserve capacity and auxiliary budgets based on volatility (σ), not just mean supply/demand differences. Higher volatility (σ) increases reserve requirements even at mean balance.

• Dispatch policy design and operational expenditures

  • Routing that minimizes expected per-task energy (ELB) reduces mean auxiliary draw, but variance-aware routing can further reduce reserve costs in the critical regime by lowering variance of consumption over time.
  • Investing in smarter dispatchers (e.g., learned routers) is only justified if their reduction in routing error (∆E) and induced variance exceeds their own operational costs (energy for routing and extra latency that may force higher-energy executions). This defines a clear ROI tradeoff for dispatcher complexity versus auxiliary energy savings.

• Model mix and productization

  • Maintaining a heterogeneous portfolio of LRMs (smaller, cheaper models and larger, costly ones) enables routing that trades compute and energy against accuracy and latency, allowing finer-grained control of mean and variance of demand.
  • Pricing and SLA design can reflect that tasks with flexible latency or error tolerance are valuable for reducing auxiliary energy needs; economic incentives (discounts, deferred-execution credits) could be used to shift workload slack and reduce costly reserve provisioning.

• Investment in renewables vs. auxiliary capacity

  • The analysis quantifies when adding baseline renewable supply vs. adding auxiliary capacity (or batteries) is economically preferable: increasing baseline supply beyond mean demand shifts to surplus (µ>0) and wastes baseline energy; reducing baseline supply induces persistent auxiliary reliance (µ<0). Operating near the critical point with sufficient variance hedging (battery or flexible scheduling) can be more cost-effective than over-provisioning baseline capacity.

• Policy and carbon accounting

  • Reducing auxiliary (typically fossil-derived) energy via smarter routing contributes to operational carbon reductions. Policymakers and firms can target variance reduction (forecasting, spatial workload shifts, demand-side flexibility) as much as adding renewable capacity to lower non-renewable backup usage.

• Practical deployment guidance

  • Use empirical compute–quality scaling laws to predict ELB and per-task energy/latency tradeoffs, enabling low-overhead dispatch rules that approximate optimal routing without requiring an energy-heavy router model.
  • Implement variance-aware dispatch heuristics: prefer allocations that slightly increase mean energy if they considerably reduce peak or variance in consumption, especially when operating near mean balance.
  • Account for dispatcher overheads (energy/latency) when evaluating investments in smarter routing infrastructure.

Caveats and limitations (relevant for economic decisions) - i.i.d. harvest model: the paper treats baseline supply as i.i.d. residuals after predictable components are removed; real renewable generation has temporal and spatial correlation and seasonality that affect reserve needs. - Unlimited parallelism and no engine-level constraints: ignores limits on concurrency, GPU contention, batching, and preemption overheads that affect practical costs. - Oracle stopping and simplified per-step energy model: assumes exact enforcement of thinking times and known energy curves per token/iteration; empirical estimation errors could change optimal routing and reserves. - Myopic policy as a reference: while analytically tractable, real policies could exploit slack windows and scheduling to further reduce auxiliary energy; the paper sets a conservative baseline.

Overall, the paper provides a quantitative, diffusion-based framework that connects model-level compute–quality scaling to system-level energy provisioning and dispatch economics, identifying variance-aware routing as a key lever for reducing auxiliary energy costs in renewable-powered AI factories.