A Pandora’s-Box routing rule lets AI systems avoid many expensive value-estimation calls while keeping allocation quality, using closed-form reservation prices to decide when extra computation is worth the cost; in decentralized bidding, inspection improves efficiency when estimates are accurate but can advantage strategic agents under noisy competition.

Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation

Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen, Michael Kaisers, Mirella Lapata, Kate Larson, Jacob Eisenstein · August 20, 2026

arxiv theoretical medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: August 21, 2026

Adam Fisch unresolved corpus identity
Shubhendu Trivedi unresolved corpus identity
Fantine Huot unresolved corpus identity
William W. Cohen unresolved corpus identity
Michael Kaisers unresolved corpus identity
Mirella Lapata unresolved corpus identity
Kate Larson unresolved corpus identity
Jacob Eisenstein unresolved corpus identity

Semantic Scholar

Latest observation: August 24, 2026

Adam Fisch provider ID
Shubhendu Trivedi provider ID
Fantine Huot provider ID
William W. Cohen provider ID
Michael Kaisers provider ID
Mirella Lapata provider ID
Kate Larson provider ID
Jacob Eisenstein provider ID

Framing model routing with costly value estimation as a Pandora’s Box problem yields closed-form reservation-price policies that substantially reduce expensive estimator queries while preserving routing quality across multiple LLM and retrieval settings.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

Semantic Scholar · Observed August 24, 2026

View corpus context

Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost. Routing requires estimating each specialist's expected return, but this value estimation has a cost. Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models with access to retrieval results or partial reasoning traces) are expensive. We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection. Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost. We call the centralized policy Pandora's Router. We extend this to a decentralized setting, Pandora's Bidder, where specialists independently decide whether to invest in self-assessment before accepting an offered price to claim a query. Experiments across three domains---a standard multi-LLM benchmark, retrieval-augmented specialists, and LLMs with variable inference-time reasoning---show that Pandora's Router matches the routing quality of exhaustive estimation, while querying the expensive estimator far less often. In the decentralized setting, value-of-information reasoning improves allocative efficiency when competing estimates are accurate; when competing estimates are noisy, however, it can increase the strategic specialist's utility at the expense of others.

Summary

Main Finding

Routing queries among heterogeneous AI specialists should account for the (non‑negligible) cost of value estimation. Casting the problem as a Pandora’s Box instance yields simple, interpretable value‑of‑information (VOI) rules: compute expensive value estimates only when their expected upside exceeds their cost. Under a Gaussian signal model these VOI rules (reservation prices) have closed‑form expressions. A centralized policy (Pandora’s Router) recovers near‑optimal allocation quality while invoking costly estimators far less often than exhaustive inspection. In a decentralized variant (Pandora’s Bidder), specialists deciding whether to pay for self‑assessment at posted prices can improve allocative efficiency when competing estimates are accurate, but strategic inspection can also redistribute surplus (helping the inspecting specialist at others’ expense) when competing signals are noisy.

Key Points

Problem formulation
- Each specialist m has a cheap estimate Fm(x) (always available) and a costly, more accurate estimate Gm(x) obtainable for cost cg,m.
- The router’s objective: maximize expected reward of the assigned specialist minus total estimation costs.
- Estimation itself is an economic decision (tradeoff between cost and information value).
Pandora’s Box mapping
- Specialists ↔ boxes; paying cg to observe Gm is opening a box.
- Reservation price ursv_m (Weitzman) encodes when opening a box is worth the cost: E[(Vm − ursv_m)+] = c_m (Vm is the unknown reward).
- Two variants:
  - Pandora‑OI (obligatory inspection): must open chosen boxes; optimal policy opens boxes in descending reservation‑price order and stops according to remaining reservation prices.
  - Pandora‑NI (non‑obligatory inspection): you may commit sight‑unseen to an unopened box; NP‑hard in general, so the paper uses a tractable “committing” policy with a backup price ubackup_m defined similarly.
Gaussian signal model & closed form
- Under a conditional Gaussian model for the relationship between cheap and costly signals, the VOI expressions and reservation/backup prices have closed forms that are straightforward to compute per input and per specialist.
Decentralized setting (Pandora’s Bidder)
- Specialists face a posted price (current best offer) and decide whether to invest in self‑assessment (pay cost to learn Gm) before claiming the query.
- This models a single stage of an ascending‑price mechanism with costly preference elicitation; it isolates incentives and strategic value of private estimation.
Empirical domains and findings
- Three empirical settings: Math (inference‑time CoT partial traces), RAG (retrieval‑augmented specialists with private corpora), and EmbedLLM (large model pool routing).
- Cheap estimator: KNN on prompt embeddings (Fm). Costly estimator: small fine‑tuned LMs predicting reward from prompt + private auxiliary info (Gm).
- Calibration/training/test splits used to fit estimators and calibrate VOI parameters.
- Table highlights (held‑out calibration MSE; cost ratios):
  - Math: MSE(F)=0.154 vs MSE(G)=0.096; cg/cf ≈ 5.8
  - RAG: MSE(F)=0.175 vs MSE(G)=0.109; cg/cf ≫ (very large; retrieval costs dominate)
  - EmbedLLM: MSE(F)=0.266 vs MSE(G)=0.198; cg/cf ≈ 1.6
- Results:
  - Pandora’s Router matches the routing quality of exhaustive estimation but runs costly estimators far less often.
  - In decentralized auctions, VOI reasoning improves allocative efficiency when competing estimates are reliable; if competitor estimates are noisy, strategic inspection can increase one specialist’s utility while harming others.

Data & Methods

Formal model
- Inputs X; for each specialist m the true (cost‑adjusted) reward Rm has conditional expectation ρm(x) = E[Rm | X = x].
- Two estimators per specialist: cheap Fm(x) (computed free in their setup) and costly Gm(x, Zm) revealed after paying cost cg,m (Zm is auxiliary info, e.g., partial CoT or retrieval).
- Router chooses which Gm to compute (sequentially) and then selects a specialist; utility = realized reward − sum of estimator costs.
Analytical approach
- Reduction to Pandora’s Box (Weitzman 1979; non‑obligatory variant Doval 2018).
- Define reservation price ursv_m by the VOI equation E[(Vm − ursv_m)+] = c_m; backup price ubackup_m used for committing policies in non‑obligatory inspection.
- Under a Gaussian conditional model linking F and G (and the true reward), closed‑form expressions for VOI and reservation/backup prices are derived, enabling per‑input decisions.
Decentralized mechanism modeling
- Pandora’s Bidder: posted price auction stage; each specialist decides whether to pay cg to refine its private estimate before submitting a claim.
- Analysis focuses on allocative efficiency and specialist payoff under variations in competitor signal quality.
Empirical methodology
- Value estimators trained on training split; calibration split used to fit Gaussian VOI parameters and estimate MSE/cost ratios; test split for final evaluation.
- Domains:
  - Math: partial chain‑of‑thought tokens constitute private info for Gm; target models differ in cost/accuracy.
  - RAG: Gm uses retrieved documents (private corpora like PubMed/Wikipedia); retrieval cost large.
  - EmbedLLM: large candidate pool, costs proportional to parameter counts; Gm is a fine‑tuned regressor on prompt.
- Baselines: exhaustive estimation (always run G for all specialists), cheap‑only routing, and other heuristic routing strategies.
- Metrics: expected reward minus estimation costs; fraction of queries invoking costly estimators; allocation quality compared to oracle.

Implications for AI Economics

Explicitly pricing information acquisition
- Value estimation is an economic action; system designers should price/charge for expensive estimation steps (retrieval, partial reasoning, fine‑tuned scoring) and incorporate VOI reasoning to avoid wasteful checks.
Cost‑sensitive routing design
- Providers can implement lightweight cheap screens (embeddings/KNN) and selectively trigger costly inference only when reservation prices indicate positive VOI—improves cost‑effectiveness without substantial quality loss.
Market design and incentives
- Decentralized specialists with private information (e.g., specialized corpora, internal benchmarks, partial reasoning access) can strategically invest in self‑assessment. Mechanism designers must consider:
  - When accurate competing signals exist, allowing costly self‑assessment increases allocative efficiency.
  - When signals are noisy, strategic inspection can produce winner‑takes‑more outcomes, redistributing surplus and potentially harming fairness.
- Posted‑price or auction formats should account for the costs of preference elicitation and the possibility of strategic withholding or over‑investing in self‑assessment.
Value of private information and access asymmetries
- Specialists with private retrieval corpora or internal tooling hold market power: the VOI framework quantifies when private information justifies higher prices or reserved allocation.
- Transparency about what information the router can access vs what specialists can privately compute will affect market outcomes.
Practical operational guidance
- Use calibrated, low‑cost estimators as default screens and compute reservation prices (or their approximations) to decide when to run expensive checks.
- For systems with many candidate models, sequential inspection in reservation‑price order is an efficient control flow.
Directions for policy and future research
- Extend results beyond Gaussian signal assumptions and to multi‑stage/continuous inspection costs.
- Study equilibrium behavior in repeated markets with learning specialists, dynamic pricing, and richer auction formats.
- Analyze fairness and distributional effects when some specialists can afford more self‑assessment or have privileged data access.
- Consider regulatory implications where private data access creates persistent allocation advantages.

Short summary: The paper formalizes routing with costly value estimation as Pandora’s Box, provides closed‑form VOI/reservation rules under a Gaussian model, implements centralized (Pandora’s Router) and decentralized (Pandora’s Bidder) policies, and empirically demonstrates that selective costly estimation achieves near‑optimal allocation while significantly reducing estimation cost—highlighting important economic and mechanism‑design consequences for multi‑model AI systems.

Assessment

Paper Typetheoretical Evidence Strengthmedium — The paper offers rigorous analytical results (closed-form VOI/reservation-price policies) and systematic experiments across three distinct routing domains showing consistent gains, but empirical evidence is limited to benchmarks/simulations, relies on a parametric (Gaussian) signal assumption and on cost estimates from a single pricing source, and lacks field/deployment validation. Methods Rigormedium — High theoretical rigor in mapping routing to Pandora's Box and deriving closed-form policies under clear assumptions; empirical evaluation uses appropriate calibration/test splits, multiple domains, and estimator MSE reporting. However, key assumptions (Gaussian signal model, specific estimator families), approximations for the NP-hard non-obligatory variant, and cost derivation from a single API/pricing table limit methodological generality. SampleThree experimental domains: (1) Math: 16,512 math problems pooled from MATH, Omni-Math, AIME, and HMMT; routing between Gemma3-4B and Gemini-3.1-Flash-Lite with cheap KNN embedding estimate and costly fine-tuned SFT-CoT on prompt+first 20 CoT tokens; (2) RAG: mixture of general factoid and biomedical QA (following Eisenstein et al. 2025) with low-cost no-retrieval model, Wikipedia RAG, and PubMed RAG specialists; cheap KNN embedding baseline and costly fine-tuned model using retrieval results; (3) EmbedLLM: >100 open-weight models as candidates with queries from benchmarks like MMLU and GSM8K; cheap KNN and costly fine-tuned prompt models. For all domains, data split into training (fit estimators), calibration (estimate VOI params), and test; MSEs of estimators reported on calibration sets. Monetary costs for estimators derived from a single provider's API pricing (Gemini API) to compute cost ratios. Themesproductivity org_design IdentificationNot a causal paper; identification consists of analytical derivations under a Gaussian signal model to obtain closed-form reservation prices/value-of-information, plus empirical evaluation via controlled experiments on benchmark datasets (calibration/train/test splits) to compare routing/bidding policies against baselines. GeneralizabilityThe Gaussian signal model and closed-form solutions may not hold when estimator errors/signals are non-Gaussian or heteroskedastic., Empirical results are on benchmarks and simulated cost models, not on live production traffic or heterogeneous real-world deployments., Cost estimates come from one vendor/pricing table and may not generalize across providers, regions, or different pricing structures (e.g., per-token vs. per-request, licensing fees)., Value estimators evaluated are KNN embeddings and small SFT regressors; different estimator architectures or richer private signals could change performance., Decentralized (Pandora’s Bidder) analysis is limited to a simplified single-stage posted-price/auction setting and a committing-policy approximation for the non-obligatory case, so strategic dynamics in richer markets may differ.

Claims (8)

Claim	Direction	Outcome	Confidence & Evidence	Details
Under a Gaussian signal model, Pandora’s Router has closed-form value-of-information expressions that determine whether refining a specialist’s value estimate is worth its cost. Organizational Efficiency	positive	Efficiency of deciding when to purchase more accurate specialist-value estimates	Reading fidelity high Study strength high	not reported 0.2
Pandora’s Router matches the routing quality of exhaustive value estimation while querying the expensive estimator substantially less often. Task Allocation	positive	Routing quality and frequency of expensive-estimator queries	Reading fidelity high Study strength medium	not reported 0.12
In decentralized allocation, value-of-information reasoning improves allocative efficiency when competing estimates are accurate. Task Allocation	positive	Allocative efficiency in decentralized specialist competition	Reading fidelity high Study strength medium	not reported 0.12
When competing estimates are noisy, value-of-information reasoning can increase the strategic specialist’s utility at the expense of other specialists. Market Structure	mixed	Strategic specialist utility and distribution of gains across competing specialists	Reading fidelity high Study strength medium	not reported 0.12
The cheap and costly value estimators used in the experiments differ in predictive accuracy: the costly estimator has lower held-out calibration MSE than the cheap KNN estimator in Math, RAG, and EmbedLLM. Decision Quality	positive	Value-estimation mean-squared error	Reading fidelity high Study strength medium	Math: MSE 0.154 (cheap) vs 0.096 (costly); RAG: 0.175 vs 0.109; EmbedLLM: 0.266 vs 0.198 0.12
The Math experiment evaluates routing over 16,512 mathematical problems spanning MATH, Omni-Math, AIME, and HMMT. Task Completion Time	positive	Routing performance for mathematical reasoning tasks	Reading fidelity high Study strength medium	n=16512 0.12
In the Math setting, the costly estimator uses the first 20 tokens of a model’s chain-of-thought reasoning trace to predict reward, and its reported calibration MSE is lower than that of the cheap KNN estimator. Decision Quality	positive	Prediction accuracy for mathematical-specialist reward	Reading fidelity high Study strength medium	n=16512 MSE 0.096 vs 0.154 0.12
Pandora’s Router can inspect specialists sequentially in reservation-price order and stop when the current best observed value exceeds the highest reservation price among the remaining specialists. Task Allocation	positive	Cost-aware specialist-selection and stopping efficiency	Reading fidelity high Study strength high	not reported 0.2