A lightweight, OR-informed AI system cut inventory turnover by 5.27 days and slashed holding costs by nearly 30% in a JD.com rollout, while raising in-stock rates by 2.29%; the gains were achieved by pretraining on OR-generated optimal policies and aligning them via reinforcement learning.

Main Finding

The paper introduces an OR-Guided "Pretrain-then-Reinforce" framework for large-scale inventory management that combines a simulation-augmented operations-research (OR) model with a domain-informed deep learning foundation model and reinforcement-learning (RL) fine-tuning. Using OR-derived reference decisions for supervised pretraining and RL as a deep alignment mechanism, the approach outperforms incumbent industrial policies in offline tests and a field deployment at JD.com: ~5.27 fewer inventory-days (reduced turnover), +2.29 percentage points in in-stock rate, and a 29.95% reduction in holding cost across the treated SKU-DC pairs.

Key Points

• Conceptual innovation: Treat OR as structured guidance (providing feasible, economically rational reference solutions) and AI as perception/adaptation. Use RL to align the learned policy with OR optimality while allowing exploration and scenario-specific adaptation.
• Two-stage pipeline:
• Pretraining: a domain-informed deep neural network is trained using high-quality OR-generated reference decisions (labels).
• Reinforcement learning fine-tuning: RL refines and aligns the pretrained policy with OR objectives and business preferences, enabling exploration and adaptation (e.g., promotions).
• Simulation-augmented OR model: The OR solver is embedded in a simulator that evaluates replenishment decisions over realistic horizons using historical demand realizations and empirical parameters (holding costs, stockout penalties, margins, lead times). This produces reference decisions closer to operational optima than closed-form analytic approximations.
• Practical design choice: A relatively lightweight model (a few million parameters) suffices when guided by structured OR logic—avoiding brute-force model scaling.
• Empirical validation: Offline numerical experiments and a live field experiment at JD.com (Sept 1–Oct 1, 2025) on 3,899 SKU-DC pairs (331 SKUs across three categories) demonstrate substantial operational improvements versus JD’s incumbent algorithm. Difference-in-differences (DiD) analysis was used to isolate time effects.
• Role of RL: Framed as an alignment mechanism (akin to RL fine-tuning in LLMs), not merely as policy-learning-from-scratch. RL promotes internalization of OR optimality patterns and permits exploration to find improved policies under realistic constraints.

Data & Methods

• Data:
  • Real sales and inventory logs from JD.com for 331 SKUs across three product categories ("meat snacks," "cakes & pastries," "egg snacks").
  • Field experiment dataset: 3,899 SKU–DC pairs, treatment vs. control groups, Sept 1–Oct 1, 2025.
• OR and simulation:
  • A multi-category joint replenishment OR model encodes business constraints, costs (holding, stockout), vendor lead times, margins, and managerial preferences.
  • A simulator runs policies over time using historical demand realizations to compute realized costs and end-of-period inventory — these outputs inform and calibrate the OR reference decisions.
• Learning pipeline:
  • Pretraining: supervised learning using OR-generated decisions as labels to learn strong feature representations and a baseline replenishment policy.
  • Fine-tuning: reinforcement learning (variants inspired by RL-from-feedback methods) used to align and further improve the policy; RL leverages exploration and allows further injection of expert guidance for special scenarios (e.g., promotions).
• Evaluation:
  • Offline benchmarks against multiple baselines (including JD’s production algorithm).
  • Live A/B style field deployment with DiD analysis to account for time trends and isolate treatment effects.
• Implementation notes:
  • Emphasis on computational efficiency and scalability (small model size, simulation-augmented labels rather than brute-force RL over enormous state-action spaces).

Implications for AI Economics

• Cost-effective AI deployment: Demonstrates that integrating structured domain knowledge (OR) can substantially reduce the need for very large models, lowering compute and data requirements for effective industrial AI—important for firm-level investment and cost-benefit calculations.
• Value of principled hybrid systems: Embedding economic structure and managerial constraints via OR produces policies that better align with business objectives (reduced holding costs, improved in-stock), illustrating higher economic returns than black-box end-to-end approaches or PTO pipelines that decouple prediction and decision.
• RL as alignment vs. pure optimization: Viewing RL as a post-training alignment tool (to internalize economic optimality and preferences) can improve robustness and adaptability while controlling the exploration costs—this reframes how firms should allocate resources between simulation/OR engineering and ML training.
• Transferability and scalability: A lightweight, OR-guided model that generalizes across SKUs and categories suggests a scalable path for wider industrial adoption, implying that investments in accurate simulation and OR modeling may yield more leverage than scaling model size alone.
• Policy and managerial guidance:
  • Firms should prioritize constructing high-fidelity, simulation-augmented OR models to generate training labels and define feasible decision spaces.
  • Use pretraining on OR references followed by RL fine-tuning to balance adherence to economic constraints with the ability to adapt to out-of-distribution events (promotions, seasonality shifts).
• Risks and limitations (practical economics considerations):
  • Quality of gains depends on the fidelity of the OR model and simulator; misspecification can bias learning.
  • RL exploration in live systems must be constrained to avoid costly inventories or stockouts; safe exploration and human-in-the-loop controls are necessary.
  • External validity: results are from specific categories and a one-month field trial—scaling across broader assortments and longer horizons needs further testing.
• Research agenda: The paper motivates further economic research on hybrid AI+OR system design, cost–benefit analysis of label-simulation investments, and rigorous methods for safe RL deployment in revenue- and cost-sensitive operational settings.