6 cumulative citations
View corpus contextA new routing algorithm lets agentic systems match high-cost model performance at much lower expense by dynamically choosing cheap or expensive models during multi-step workflows; Boundary-Guided Policy Optimization learns when to spend for success and when to conserve budget, improving efficiency under strict per-task limits.
Budget-Aware Agentic Routing via Boundary-Guided Training
Boundary-Guided Policy Optimization (BoPO) and boundary-guided SFT data synthesis enable dynamic per-step routing between cheap and expensive models, improving the cost--success frontier and matching strong baselines at substantially lower per-task cost while generalizing to strict budget constraints.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
As large language models (LLMs) evolve into autonomous agents that execute long-horizon workflows, invoking a high-capability model at every step becomes economically unsustainable. While model routing is effective for single-turn queries, agentic routing is a sequential, path-dependent problem: early mistakes compound, feedback is often at the end of the episode, and deployments often demand strict per-task spending limits. We propose Budget-Aware Agentic Routing, which selects between a cheap and an expensive model at each step to optimize the cost--success frontier and to operate under strict per-task budgets. We propose Boundary-Guided Training, which leverages two boundary policies (always-small vs.\ always-large) to build a difficulty taxonomy and to anchor learning under sparse rewards. Our approach warms start with boundary-guided SFT data synthesis via stratified sampling of cost-efficient trajectories, then applies Boundary-Guided Policy Optimization (BoPO), combining boundary-relative rewards with a reference-guided advantage to avoid degenerate cheap-failure solutions. Experiment results show that our method improves the efficiency frontier, matching strong routing baselines at substantially lower cost while demonstrating generalization to strict inference-time budget constraints. Overall, our work establishes a foundational framework for agentic routing, shifting the paradigm from static model selection to dynamic, budget-aware sequential decision-making.
Summary
Main Finding
Budget-Aware Agentic Routing (BAAR) reframes model routing in long-horizon agentic workflows as a sequential decision problem and shows that a boundary-guided two-stage training pipeline (Boundary-Guided SFT + Boundary-Guided Policy Optimization, BoPO) can substantially improve the cost–success efficiency frontier. The approach matches the success of always-using a high-capability model at a fraction of cost, and it generalizes to strict hard-budget inference via a simple decoding constraint.
Key Points
- Problem framing
- Agentic routing is sequential and path-dependent: early routing errors compound and feedback is sparse (terminal rewards).
- Two deployment paradigms: soft-budget (optimize trade-off via λ) and hard-budget (strict per-task spending cap).
- Boundary-Guided Training (two-stage)
- Task difficulty taxonomy: probe each task with two extremes (always-small, always-large) and split tasks into Easy, Hard, Intractable.
- BoSFT (Boundary-Guided Supervised Fine-Tuning): synthesize SFT labels
- For Easy and Intractable tasks, label with always-small trajectories (solve cheaply or fail cheaply).
- For Hard tasks, use stratified sampling across different probabilities of calling the large model; among successful runs pick the lowest-cost successful trajectory as the SFT “critical path” proxy.
- BoPO (Boundary-Guided Policy Optimization): online RL built on Group Relative Policy Optimization with two anchors
- Boundary-relative reward: calibrates cost penalties by task difficulty and normalizes cost relative to static baselines; adds a bonus term for Hard tasks so success is prioritized when large-model reasoning is needed.
- Reference-guided advantage: compare sampled trajectories to group mean and to the SFT critical-path reference to avoid degrading below the known boundary and to prevent collapse to always-small failure modes.
- Token-level KL penalty to keep semantics aligned with SFT reference.
- Hard-budget handling
- Budget-Constrained Decoding: enforce strict budget at inference by pruning any action (choosing the large model) that would exceed remaining budget; budget state is embedded in the prompt.
- Empirical results (high level)
- Evaluated on three long-horizon interactive benchmarks (scientific discovery SciWorld, embodied ALFWorld, and complex app-interaction AppWorld).
- BoPO consistently improves the cost–success frontier, matching large-model performance at substantially lower cost.
- Policies trained on soft-budget objectives transfer reasonably under hard-budget inference via BCD, though fixed soft-budget training does not fully adapt to different hard caps.
- Limitations & future directions
- Policies trained with a fixed soft-budget trade-off are not fully adaptive to varying hard caps—motivating budget-conditioned policies.
- Current work focuses on binary small/large choice; naturally extensible to multiple-model pools.
- The stratified sampling gives an upper-bound proxy for optimal cost paths; further RL is needed to push below that.
Data & Methods
- Formalism
- POMDP-style router πθ(at | st) over T-step agentic trajectories; actions choose between Msmall and Mlarge, each with costs c(·).
- Soft-budget objective: maximize E[I(success) − λ · total_cost]; varying λ traces the efficiency frontier.
- Hard-budget objective: maximize E[I(success)] subject to total_cost ≤ Bmax (CMDP).
- Task difficulty taxonomy
- Run πsmall and πlarge K times (K=5) per task; partition into Deasy (πsmall succeeds ≥4/5), Dhard (πsmall unreliable but πlarge can), Dintractable (πlarge fails).
- BoSFT (offline warm start)
- Easy & Intractable: label with πsmall trajectories.
- Hard: stratified sampling across N probabilities P(at=Mlarge)=k/N; collect successful trajectories and choose minimal-cost successful trajectory τ* as the reference.
- BoPO (online optimization)
- Base optimizer: Group Relative Policy Optimization (GRPO) to avoid value-function instability with long histories.
- Reward R(τ,x) = (rsuccess + rhard) − λ·Cnorm(τ,x), where rhard>0 only for Dhard and Cnorm normalizes trajectory cost relative to baseline min/max costs and clips to [0,1].
- Advantage A(τg) = [R(τg,x) − max(µgroup, R(τ*,x))] / (σgroup + ϵ) to require improvement beyond the SFT reference.
- Final objective includes importance weights and KL regularization toward πref (SFT model).
- Hard-budget inference
- Budget-Constrained Decoding (BCD): at each step, if remaining budget bt < cost(Mlarge), force Msmall; otherwise use learned policy with budget embedded in prompt.
- Evaluation
- Long-horizon benchmarks: SciWorld, ALFWorld, AppWorld (AppWorld is the primary complex testbed with many API interactions).
- Metrics: success rate vs cost (efficiency frontier), performance under strict per-task budget caps.
- Key empirical claims: BoPO pushes the efficiency frontier (match-large at lower cost), transfers to hard-budget via BCD but does not fully adapt to multiple diverse hard caps when trained only under a fixed soft-budget.
Implications for AI Economics
- Operational cost reduction
- Dynamic, per-step routing yields substantial compute cost savings relative to always-using high-capability models; this shifts the marginal cost of deploying agentic systems downward while preserving performance.
- Providers/teams can maintain user-facing quality while reducing spending on expensive model-invocations by allocating expensive compute only to critical steps.
- Pricing and product design
- Fine-grained metering and tiered model APIs (cheap vs. expensive calls) become more valuable; BAAR-style routers increase the utility of offering distinct capability tiers.
- Pricing models could be redesigned to reflect step-wise marginal value of capability (e.g., discounted bulk small-model calls + priced critical large-model credits).
- Demand and capacity implications
- If adopted widely, dynamic routing can reduce aggregate demand for continuous large-model usage, smoothing load and potentially lowering the need to scale large-model serving capacity.
- At the same time, concentration of large-model calls on critical steps may induce bursty demand patterns; capacity planning should account for temporal clustering of expensive calls.
- Business strategy and product SLAs
- Hard-budget routing and BCD enable vendors to offer budget-capped SLAs (bounded per-task spending guarantees) with provable routing strategies, making advanced agentic features accessible to cost-sensitive customers.
- Sellers could offer “budget-conditioned” tiers: customers set per-task budgets and the router optimizes success subject to that cap.
- Economic value of model capability
- The method quantifies the marginal value of expensive calls: improvements in success rate per additional expensive call can be inferred from the efficiency frontier, informing investment decisions in model R&D and deployment trade-offs.
- Market opportunities
- Growth in specialized small models and efficient routers: tools and marketplaces for router tuning (profiling task difficulty, training BoSFT/BoPO) can become products.
- Opportunity for third-party router-as-a-service that mediates between customers and model providers, optimizing spending across provider APIs.
- Risks & considerations
- Incentives: providers may adjust pricing or capability mixes if dynamic routing materially reduces usage of expensive models.
- Behavioral responses: developers may design tasks to circumvent routers (e.g., splitting tasks into more steps) or exploit budget rules—requiring monitoring and robust policy design.
- Welfare & equity: lower costs per task can expand access to advanced agentic tools, but careful attention to distributional effects and possible quality variability is needed.
- Recommendations for practitioners and economists
- Instrument tasks to obtain small/large baseline profiles to build difficulty taxonomies before deployment.
- Train routers with boundary-guided methods to obtain stable, cost-aware policies; embed remaining-budget state to improve hard-budget adaptability.
- Evaluate and publish efficiency frontiers so stakeholders can understand marginal returns to expensive compute and design pricing/SLAs accordingly.
- Consider multi-model extensions and budget-conditioned policies to better handle heterogeneous budget constraints and pricing regimes.
Assessment
Paper Typeother
Evidence Strengthn/a — The paper proposes and empirically evaluates a machine-learning algorithm for model routing and cost-aware agents rather than making causal claims; standard causal identification criteria are not applicable.
Methods Rigormedium — The paper develops a clear algorithmic approach (Boundary-Guided Training and BoPO) and reports experiments showing improved cost--success frontiers and generalization to tight budgets, but the description lacks details here about datasets, scale, real-world deployments, robustness checks, and statistical uncertainty; without those details the evaluation appears solid for an ML methods paper but not exhaustive.
SampleExperiments on agentic long-horizon workflow tasks using two-tier models (a cheaper model and a higher-capability expensive model); training data is synthesized via boundary-guided supervised fine-tuning (stratified sampling of cost-efficient trajectories) and policy optimization is evaluated on simulated episodic environments with sparse, end-of-episode rewards; exact benchmarks, task domains, and data sizes are not specified in the summary.
Themesproductivity adoption
GeneralizabilityEvaluation appears limited to simulated or benchmark long-horizon workflows rather than diverse real-world production tasks, Assumes a two-tier (cheap vs expensive) model architecture; results may not extend to many-model or continuous-cost settings, Performance depends on the assumed cost model and budget constraints, which may differ across deployments and cloud pricing, Sparse end-of-episode reward environments used for training may not reflect tasks with richer intermediate feedback, Scalability and robustness to adversarial or highly nonstationary task distributions not demonstrated
Claims (8)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Invoking a high-capability model at every step of long-horizon agentic workflows becomes economically unsustainable. Organizational Efficiency | negative | economic sustainability / cost of invoking high-capability models per-step |
Reading fidelity
high
Study strength
speculative
|
not reported
|
| Model routing is effective for single-turn queries. Organizational Efficiency | positive | effectiveness of model routing for single-turn queries |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Agentic routing is a sequential, path-dependent problem where early mistakes compound, feedback is often only at the episode end, and deployments often impose strict per-task spending limits. Task Allocation | mixed | sequential path-dependence and feedback sparsity in agentic routing; constraints on per-task spending |
Reading fidelity
high
Study strength
medium
|
not reported
|
| We propose Budget-Aware Agentic Routing: selecting between a cheap and an expensive model at each step to optimize the cost--success frontier and operate under strict per-task budgets. Organizational Efficiency | positive | cost--success frontier (tradeoff between monetary cost and task success under budget constraints) |
Reading fidelity
high
Study strength
speculative
|
not reported
|
| We propose Boundary-Guided Training which leverages two boundary policies (always-small vs. always-large) to build a difficulty taxonomy and to anchor learning under sparse rewards. Other | positive | ability to build difficulty taxonomy and stabilize learning under sparse rewards |
Reading fidelity
high
Study strength
low
|
not reported
|
| The approach warm-starts with boundary-guided SFT data synthesis via stratified sampling of cost-efficient trajectories, then applies Boundary-Guided Policy Optimization (BoPO) combining boundary-relative rewards with a reference-guided advantage to avoid degenerate cheap-failure solutions. Other | positive | prevention of degenerate cheap-failure policies and improved policy learning |
Reading fidelity
high
Study strength
low
|
not reported
|
| Experiment results show that our method improves the efficiency frontier, matching strong routing baselines at substantially lower cost while generalizing to strict inference-time budget constraints. Organizational Efficiency | positive | efficiency frontier (cost vs. success), cost reduction, generalization to inference-time budget constraints |
Reading fidelity
high
Study strength
medium
|
not reported
|
| This work establishes a foundational framework for agentic routing, shifting the paradigm from static model selection to dynamic, budget-aware sequential decision-making. Innovation Output | positive | conceptual framing / adoption of dynamic budget-aware routing |
Reading fidelity
high
Study strength
speculative
|
not reported
|