Frontier LLMs can already write competitive operations-research algorithms: with one untuned prompt and modest sandboxed compute, the top model matches or beats the best-known methods on most benchmark instances — though results are limited to well-specified problems and may partly reflect the models' training data.

Main Finding

A single untuned query to a frontier LLM (gpt-5.6-sol) can produce executable algorithms that match or outperform the best existing methods on canonical, well-specified operations-research (OR) benchmarks. This holds both when the LLM solves individual instances (level 1) and when it is asked to return a reusable algorithm for an entire problem class (level 2). The LLM’s outputs are often inspectable algorithms that reproduce known OR ideas and, in many cases, improve on tuned heuristics or learned policies.

Key Points

• Two query granularities:
  • Level 1: LLM receives a fully specified instance and returns a solution for that instance (one-query per instance).
  • Level 2: LLM receives a problem-class description and parameter ranges, and returns an algorithm mapping instance parameters → solutions; that algorithm is fixed before evaluation and must produce instance-specific outputs within 30s.
• Domains tested: inventory control, queueing network control, assortment optimization.
• Models evaluated: gpt-5.1, gpt-5.4, gpt-5.6-sol (OpenAI) and claude-fable-5 (Anthropic). Performance improved substantially across these releases (released within an 8-month window), indicating rapid capability gains.
• gpt-5.6-sol results (high level):
  • Matches or beats the best existing method on nearly every evaluated instance across the ten problem classes studied.
  • Mean performance no worse than the best existing method in 8 of 10 classes at both levels.
  • Level-2 (reusable algorithm) often nearly as good as per-instance level-1 outputs.
  • Main exception: nested-logit assortment where mean revenue was ~1.2% lower than the best comparator on a tail of hard instances.
• Generated algorithms are interpretable and use familiar OR constructs (e.g., capped base-stock variants, projected inventory statistics, dynamic programming on small state spaces, pressure-based queueing rules, greedy + relaxation + local improvement for assortments).
• All prompts, code, and run records are publicly available for reproducibility.

Data & Methods

• Benchmarks and instance counts:
  • Inventory: 34 instances from Gijsbrechts et al. (2022) covering lost-sales (deterministic & stochastic lead time), dual sourcing, and multi-echelon distribution.
  • Queueing: 13 multiclass queueing-network instances from Dai & Gluzman (2022) (criss-cross, N-model, extended six-class reentrant-line).
  • Assortment: 3,393 instances from Guo et al. (2025) covering mixed-MNL (MMNL), nested-logit (hard regime), and constrained mixed-MNL.
  • Note: Level-1 evaluation used stratified subsets for large assortment classes (e.g., 72 MMNL, 48 nested-logit, 36 constrained MMNL).
• LLM usage and protocol:
  • Single untuned prompt per task class (no prompt tuning, minimal human intervention).
  • LLMs had access to a Python sandbox with a fixed compute budget; level-2 algorithms were allowed up to 30s per instance to produce a solution.
  • Comparison baselines: best-performing methods reported in the source benchmark papers, including exact dynamic programming (where tractable), tuned heuristics, conic MIP, and per-instance trained RL controllers.
• Evaluation metrics:
  • Inventory & queueing: relative cost reduction (percent) vs best existing method.
  • Assortment: relative revenue gain (percent) vs best existing method.
  • "No worse" metric: % of instances where LLM is within 0.1% of or better than the comparator.
• Representative quantitative highlights from the strongest model (gpt-5.6-sol):
  • Multi-echelon inventory: large mean cost improvement (e.g., +22.78% on a tiny class of 2 instances reported in table).
  • Deterministic lost-sales: median costs close to or better than benchmarks; other inventory classes also showed consistent improvement or parity.
  • Queueing: matches DP optimum on criss-cross, within ~0.2% of optimal on N-model, beats per-instance PPO on many reentrant-line instances.
  • Assortment: matches optimum on MMNL and constrained MMNL classes; nested-logit class shows ~1.2% mean revenue deficit on hard tail.
• Robustness and checks:
  • Authors acknowledge potential dataset contamination (model may have seen benchmark material). They partially address this by testing level-2 algorithms on out-of-benchmark parameter regimes and generated instances.
  • No theoretical approximation guarantees provided; results are empirical and limited to the three OR domains tested.

Implications for AI Economics

• Productivity and task specialization:
  • Algorithm-design costs for well-specified OR problems can drop sharply if LLMs reliably generate competitive algorithms with minimal human tuning. This could shift labor from manual algorithm engineering toward specification, validation, and deployment.
  • Lower marginal cost of algorithm prototyping may accelerate empirical experimentation and make custom algorithm development accessible to smaller firms or teams without deep OR expertise.
• Research and baseline dynamics:
  • Frontier LLMs may become standard empirical baselines in OR and operations economics, forcing researchers to benchmark against LLM-generated algorithms and to report robustness beyond benchmark families.
  • Rapid model improvement implies moving targets for baselines; reproducible benchmarking and archived model outputs will be important.
• Markets and comparative advantage:
  • Commercial services (consulting, software) that provide tailored algorithms might face competition from LLM-assisted automated algorithm design, altering pricing and labor demand in OR-intensive industries.
  • Conversely, firms that integrate LLMs with domain expertise (for validation, safety, regulatory compliance, and deployment) may capture new value-added roles.
• Risks and policy considerations:
  • Overreliance on LLM-generated algorithms raises robustness, safety, and auditability concerns—especially if models have seen proprietary training data (contamination) or fail out-of-distribution.
  • No formal guarantees: economic applications with high-stakes outcomes will still need expert validation, stress-testing, and possibly regulatory oversight.
  • Need for standards: shared benchmarks, transparency about model training data, and protocols for robustness testing should accompany adoption in economically significant settings.
• Research directions for AI economics:
  • Quantify economic gains from reduced algorithm-design costs across sectors (sensitivity to domain specificity and data availability).
  • Study labor-market impacts on OR researchers, algorithm engineers, and consulting markets.
  • Evaluate LLM generalization to problem classes without rich prior literatures (tests the limits of implicit literature-derived performance).
  • Develop frameworks for combining LLMs with domain constraints, formal verification, and human-in-the-loop validation to manage risk while capturing productivity benefits.