Small scout models can discover the order in which larger models should resolve skill bottlenecks; replaying these phase-ordered trajectories cuts large-model training tokens by roughly 30–40% in the authors' experiments, outperforming static proxy mixtures.

Main Finding

Small “scout” models can reveal a transferable curriculum structure not captured by a static data mixture: the phase-ordered sequence in which skills become bottlenecks. A simple closed-loop controller (LogFloor) that prioritizes the current bottleneck produces ordered allocation trajectories that (1) greatly reduce tokens-to-target on the same scale and (2) transfer across scales (e.g., a 70M scout → 12B target) after a very short target probe, still yielding large token savings even when the scouting cost is accounted for.

Key Points

• First-passage skill training objective: stop when every monitored skill k reaches a prespecified floor τk; goal is to minimize training tokens until the first round T s.t. sT,k ≥ τk for all k (s measures fold reduction in monitored log loss).
• LogFloor controller: compute deficits dr,k = max{τk − sr,k, 0}; soft allocation ˜wr,k ∝ qk · exp(dr,k / η); then wr ← (1−α)wr−1 + α·Bound(˜wr). Parameters η (focus on bottlenecks) and α (smoothing) control behavior. Greedy LogFloor is a hard-bottleneck variant.
• On controlled bAbI skill slices (5 skills, Qwen2.5-1.5B):
  • LogFloor pooled token saving = 56.2% vs matched uniform training (across 30 route targets, 100% route-hit).
  • Greedy LogFloor pooled = 51.1%.
  • Target-free baselines under the same objective perform poorly: Skill-It pooled = −7.8%, ODM pooled = −0.9%.
  • Heldout audits corroborated route hits in 28/30 runs.
• Cross-scale transfer (trajectory replay):
  • Replay schemes: raw round-indexed, target-probe replay (short uniform probe H to localize phase), and oracle phase-aligned replay (uses target trajectory length).
  • A 70M LogFloor trajectory replayed to 12B using a 3-round target probe:
    • Pair-mean saving = 30.9%
    • Pooled saving = 39.4%
    • Scout-cost-inclusive saving (70M tokens scaled proportionally) = 37.6%
    • Replayed trajectories hit all 8/8 12B targets and won 7/8 matched comparisons vs uniform.
  • 12B online LogFloor (fully adaptive at target scale) saved 34.4% pair-mean, 39.4% pooled — replay is nearly as effective.
  • Oracle phase-aligned replay also effective (≈35.1% pooled), showing phase alignment matters.
• Order matters:
  • Collapsing the source path to a static marginal mixture or reversing phase order removes most benefits (example diagnostics showed static collapse and order reversal preserved only ~10% and ~6% of benefit in a diagnostic).
  • On MMLU-control, a frozen scout path saved ~60.3% pooled rounds; destroying order removes most gains.
• Static proxy-mixture search can improve the small-model proxy score but transfer to the large model can be non-monotonic — the best mixture for the small model is not guaranteed best for the large one.
• Short probe H = 3 was sufficient and robust in the reported experiments (sensitivity checks done for H ∈ {1,3,5}).

Data & Methods

• Monitored objective: first-passage over K monitored skill slices; score sr,k = ℓ0,k / ℓr,k (fold reduction in log loss); deficits dr,k = [τk − sr,k]+; stop at first round where all floors met.
• Controller (LogFloor) implements floor-pressure exponential weighting with caps and smoothing; outputs an ordered allocation trajectory γ = {wr}.
• Datasets / domains and models:
  • bAbI skill slices (five controlled skills) evaluated on Qwen2.5-1.5B and on multiple Pythia models (70M, 160M, 410M, 1B, 2.8B, 12B depending on experiments).
  • MMLU-control (academic-domain route targets) used to test generalization of phase order signals.
• Experiment design highlights:
  • Qwen2.5-1.5B sweep: floors 2.8–3.3, five random seeds per floor → 30 routes.
  • 70M→12B transfer: eight seed-matched source–target pairs; target-probe H=3 used for replay phase localization.
  • Baselines: matched uniform training, Skill-It, Online Data Mixing (ODM), RegMix (static proxy mixtures), AC-ODM, PROXYMIX; static summaries: cumulative static, final static mixtures.
• Evaluation metrics:
  • Pooled saving: savings after summing token costs across runs.
  • Pair-mean saving: mean of savings across matched runs.
  • Hit rate: fraction of runs that reach all monitored floors within horizon.
  • Heldout audit: heldout evaluation at the route first-hit round to verify generalization.
• Accounting for scout cost: scale 70M source tokens by parameter ratio (70M/12B) and add to target cost; replay savings remain substantial under this accounting.

Implications for AI Economics

• Cost-effective curriculum discovery: Running small, cheap scout models to discover bottleneck-resolving phase order and replaying that ordered path with a short target probe can substantially lower the tokens (and therefore compute cost) needed to bring large models to required per-skill thresholds. Example: ~30–40% token savings in 70M→12B experiments after reasonable scout-cost accounting.
• Scout ROI and budget trade-offs: Because scout costs can be amortized and savings remain after including scout cost, organizations can formalize an ROI calculation (expected token savings × cost per token − scouting cost) to decide whether to run scouts and how many. The paper shows scouting is frequently worthwhile under the studied settings.
• Beyond static mixtures: Static proxy mixtures or expensive static-search over mixtures can mislead (non-monotonic transfer). Investing in trajectory (order) information is often more valuable than searching for a single static mixture on a proxy model.
• Operational workflow recommendation:
  • Use small-model LogFloor runs to produce an ordered trajectory.
  • During target training, do a very short uniform probe (empirically small H, e.g., 3 rounds) to localize the target’s phase, then replay the source trajectory monotonically.
  • Optionally, run the full online LogFloor at target scale when practical, but replay offers near-online performance at lower complexity.
• Limits and cautions for economic deployment:
  • Applicability depends on having meaningful monitored skill slices and well-specified floors (the first-passage objective specifically targets “reach every floor” problems).
  • Transfer success depends on representativeness of scout domains and the assumption that bottleneck geometry (which skills become bottlenecks in which phase) is similar across scales and data regimes. This held in reported experiments but may fail for some tasks/domains.
  • Short probe length and matching procedure are design choices that may require calibration per domain; mis-localization can reduce benefits.
  • The approach optimizes time-to-threshold (tokens until worst skill clears floor), not aggregate final performance — appropriate when minimum per-skill performance guarantees are required.
• Research and policy angles:
  • From an organizational economics perspective, this method suggests a structured, low-cost exploration stage (scout + probe) that improves allocation of expensive training budgets and reduces waste from overtraining non-bottleneck skills.
  • For cost forecasting and procurement, models of scout amortization and risk (probability of transfer success) should be integrated into model-development investment decisions.
  • Future work could formalize optimal scouting budgets, multi-scout portfolio strategies (diverse scouts for robustness), and market-style competition between scout-informed versus target-only schedule optimization.

Limitations noted by the authors: results are for monitored first-passage objectives and controlled skill slices; not claimed to be universally applicable. Order-destruction tests show most benefits come from phase ordering rather than static mixtures, but generalization to broader pretraining or unmonitored objectives remains to be studied.