0 cumulative citations
View corpus contextA transformer-based DRL planner cut total truck-routing costs by 18–23% in three industrial milk‑run case studies versus an LTL-price baseline; however, results rely on small synthetic training sets and narrow baseline comparisons.
Vehicle routing problem using deep reinforcement learning - A case study about truck planning in the industry
A transformer-based deep reinforcement learning policy applied to three industrial external milk-run routing problems produced reported transportation cost reductions of ~18–23% compared with an LTL-price baseline.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
As an important component of the supply chain industry, transportation has experienced rapid development in the past decade with the assistance of digital platforms and intelligent algorithms. Within the field of transportation research, Vehicle Routing Problem (VRP) has remained a persistent and enduring challenge. In the realm of management science, experts, and scholars from both the industrial and academic sectors have continuously explored optimization models and algorithms to effectively address routing problems, from the classical Traveling Salesman Problem to the more general Vehicle Routing Problem. These models and algorithms are applied in real-world industrial scenarios to achieve cost optimization and reduce carbon footprints. However, due to the complexity of real-world problems, numerous specific constraints are often added, and challenges such as information opacity, uncertainty, and irrational human behavior may arise. Therefore, deploying and optimizing mathematical models for VRP in practical scenarios while maintaining optimal results poses numerous challenges. This paper discusses and provides solutions for three different logistic use cases involving external truck network design. Through these industrial case study, the paper introduces how deep reinforcement learning-based vehicle routing optimization has been implemented. As a result, it can be observed that the routes optimized by reinforcement learning agent have over 10% total cost compared to baseline results. Furthermore, the paper proposes that in future research, DRL algorithms for vehicle routing problems could be generalized into more variations of VRP.
Summary
Main Finding
A Transformer-based deep reinforcement learning (DRL) policy can produce deployable routing plans for heterogeneous capacitated VRP with hybrid modes (external milk-run (EMR) consolidation plus option for less-than-truck-load (LTL) shipments) and yields substantial cost savings in industrial truck-planning case studies: the paper reports total-transportation-cost reductions of 23.48%, 18.10%, and 18.87% on three real-inspired use cases versus a baseline built from summed LTL prices. The DRL approach also achieves faster inference than classical optimization and reportedly improves load factors (empty load ≤ ~25%).
Key Points
- Problem & contributions
- Targets heterogeneous capacitated VRP (HCVRP) in industry, adding a hybrid decision between EMR consolidation and LTL delivery.
- Proposes a deployable, model-free Transformer policy network for routing decisions.
- Claims >10% cost reduction generally and specific reductions of 23.48%, 18.10%, 18.87% in three case studies.
- Modeling highlights
- Objective: minimize total cost = sum over vehicles (fixed cost per vehicle + distance × unit cost × adjacency indicator).
- Decisions: at each timestep select a vehicle and next location; LTL chosen when single-order route is cheaper than EMR alternative.
- State includes per-vehicle remaining capacity, accumulated time, and visited-coordinate history; point states include coordinates and transported volume.
- Rewards = negative of delivery costs (per-step negative cost).
- Constraint handling via real-time masks (e.g., to prevent capacity violations); time constraints (e.g., ≤24 h) and service windows can be included.
- Algorithm & implementation
- Uses Transformer-based policy network (cites Li et al. 2021 for related DRL solution to HCVRP); training algorithm not fully specified (authors mention PPO as an example but do not clearly state the exact optimizer used).
- During sampling the action space is restricted via masks to respect feasibility.
- Inference requires padding inputs to the graph size used in training when test graph sizes differ.
- Empirical setup & results
- Three industrial "external milk-run" use cases (plants near Shanghai) with different capacity metrics (quantity, weight, volume) and several vehicle types; vehicle speeds set to 35 km/h; service times and windows modeled.
- Synthetic training/validation data were generated using real-feature ranges; training sets described as 10 training samples and 8 validation samples (each sample with 200 orders) — small-scale synthesis used to mimic real cases.
- Two models trained (one for the first use case, another for the 2nd and 3rd); training ran ~5 hours per model for 100 epochs on an NVIDIA Quadro RTX 8000; graph size during training = 200 nodes.
- Training/validation metrics showed decreasing cost and decreasing (more negative) reward across epochs, indicating learning progress.
- Compared to baseline defined as sum of LTL prices (per-order LTL costs from domain pricing), DRL produced the reported cost reductions and visually sensible consolidated routes.
- Limitations (noted or evident)
- Small reported training set (10 samples) raises concerns about generalization.
- Baseline is LTL-sum; no direct comparison to strong optimization baselines (MIP, classic heuristics) in the paper’s reported numeric comparisons.
- Key algorithmic details (exact RL algorithm, hyperparameters, ablations) are not fully specified.
- Padding requirement and training graph-size dependence limits immediate generalization to arbitrary instance sizes without retraining or careful handling.
Data & Methods
- Data
- Real industrial use cases (three external milk-run scenarios near Shanghai) informed parameter ranges.
- Synthetic training/validation datasets generated from real-value ranges:
- Coordinates: longitude 28–32, latitude 118–122.
- Demand ranges: QTY 1–44 for case 1; weight 0.007–17.5 tons and volume 0.5–41.4 m3 for cases 2–3.
- Service time 0.75–1.5 hours; vehicle speed 35 km/h.
- Vehicle types and capacities detailed (e.g., multiple capacity tiers in quantity, weight, volume).
- Baseline LTL prices computed using domain-expert formulas for per-order LTL cost.
- Methods
- Formulated VRP as an MDP: states = vehicles × their remaining capacities/time/history plus point demands; actions = (vehicle, next location); rewards = negative incremental transport cost (including fixed vehicle usage cost when relevant).
- Transformer architecture for policy network (graph size 200 during training).
- Constraint masking to enforce feasibility when sampling actions.
- Training: two models, 100 epochs ≈ 5 hours training each on Quadro RTX 8000; monitored average total cost (training) and average reward (validation).
- Inference: pad graphs to the trained graph size; output route sequences, vehicle assignments, and EMR vs LTL decisions.
Implications for AI Economics
- Operational cost and asset utilization
- Reported cost reductions of ~18–23% imply substantial short-run savings for logistics planners when DRL consolidation replaces per-order LTL pricing; this translates directly to lower variable transport costs and better truck utilization.
- Improved load factors (reduced empty miles) increase labor and capital productivity (trucks and drivers), potentially lowering per-unit transport cost and influencing pricing/contracting between shippers and carriers.
- Market and incentive effects
- If carriers or shippers widely adopt DRL consolidation, demand for LTL services may decline on routable lanes, pressuring LTL pricing and potentially prompting carriers to change service offerings (e.g., more focus on dynamic consolidation).
- DRL-enabled faster inference supports near-real-time routing and dynamic re-planning, enabling more responsive supply chains and potentially creating market preference for partners who can provide such optimization.
- Deployment and adoption costs
- Model training requires realistic, representative data; the paper indicates training on synthetically generated examples shaped by domain ranges — practitioners will need investment in data engineering to ensure models generalize to their demand distributions.
- The need to pad to a fixed graph size and limited training sample diversity suggests nontrivial engineering costs to scale models to varying instance sizes and regions.
- Policy and externalities
- Route consolidation reduces total vehicle kilometers traveled, implying potential emissions reductions and congestion externality mitigation; quantified environmental benefits would strengthen the social-case for adoption.
- However, increased efficiency could reduce labor hours for drivers in the long run if fleet consolidation materially reduces required runs—policy implications for local labor markets deserve attention.
- Research & industry gaps relevant to economics
- Benchmarking: economic assessment requires comparisons to strong operational baselines (MIP/heuristics) under realistic cost structures; the paper’s baseline (sum of LTL) may overstate relative gains in regions where good heuristics are already used.
- Generalization risk: models trained on limited or locally biased data may underperform out of sample; economic value depends on robust transferability across demand patterns and geographic networks.
- Pricing dynamics: carriers may adapt pricing if DRL consolidation changes their utilization patterns; studies of equilibrium effects between optimization adoption and carrier pricing are needed.
Overall, the paper demonstrates promising operational benefits from Transformer-based DRL for industrial truck routing and consolidation, but for economic decision-making and broader deployment, further work is needed on benchmarking against stronger baselines, testing out-of-sample robustness, scaling to varied instance sizes, and quantifying wider market and labor impacts.
Assessment
Paper Typedescriptive
Evidence Strengthlow — The paper reports cost reductions (18–23%) on three industrial case studies versus an LTL-price baseline, but uses a very small synthetic training/validation set (10 training / 8 validation samples), unclear or weak baselines (sum of LTL price only), no statistical tests, and limited out-of-sample evaluation, so empirical claims are not robustly established.
Methods Rigorlow — Methodological description of the DRL model and MDP is reasonable, but experimental design is weak: tiny number of generated training/validation samples, unclear train/test split and potential leakage, limited baseline comparisons (no modern heuristics or exact solvers), no statistical significance or robustness checks, and simplified assumptions (fixed speeds, padding strategy).
SampleTraining/validation data were synthetically generated using real feature ranges from three industrial external milk-run datasets near Shanghai. The authors report 10 training samples and 8 validation samples (each sample containing demand for 200 orders). Two models were trained (one for use case 1, another for use cases 2 and 3) on a Quadro RTX 8000 for ~5 hours (100 epochs); graph size during training was 200. Test (case-study) instances correspond to three real use cases with 171, 360, and 504 order demands. Baseline used for comparison was the sum of per-order LTL prices; LTL price generation relied on domain-expert calculations.
Themesproductivity innovation
GeneralizabilityTraining data are synthetic and limited (10 training samples); risk of overfitting to generated distributions., Geographically limited to plants near Shanghai; results may not transfer to different geographies or logistics networks., Simplifying assumptions (fixed vehicle speed, fixed service times, worst-case unloading assumption) reduce realism., Baseline comparisons are narrow (sum of LTL prices); no comparison to state-of-the-art heuristics or exact solvers., No exploration of dynamic or stochastic demand arrivals, driver hours, traffic variation, or real-time replanning., Model architecture and padding choices may not scale or generalize to much larger graphs or different order distributions.
Claims (5)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| The deep reinforcement learning algorithm reduced total transportation costs by 23.48%, 18.10%, and 18.87% in the three industrial external milk-run use cases, respectively, compared with the baseline. Organizational Efficiency | positive | Total transportation cost relative to the LTL baseline |
Reading fidelity
high
Study strength
medium
|
n=3
23.48%, 18.10%, and 18.87% reductions
|
| The reinforcement-learning agent produced lower-cost external milk-run routes than the baseline results. Organizational Efficiency | positive | Cost of optimized external milk-run routes |
Reading fidelity
high
Study strength
medium
|
n=3
|
| The proposed deep reinforcement learning model has faster inference than classical mixed-integer programming and heuristic optimization methods. Task Completion Time | positive | Inference speed for solving vehicle-routing instances |
Reading fidelity
high
Study strength
low
|
not reported
|
| The Transformer-based deep reinforcement learning algorithm achieved improved vehicle load utilization, described by the authors as no more than 25% empty load. Organizational Efficiency | positive | Empty vehicle-load proportion |
Reading fidelity
high
Study strength
speculative
|
no more than 25% empty load
|
| The deep reinforcement learning agent can generate new routes by connecting multiple nodes in a single trip to reduce transportation costs. Task Allocation | positive | Number of nodes served per route and associated transportation cost |
Reading fidelity
high
Study strength
low
|
n=3
5 nodes in one trip; 4 nodes in one trip in each of the other two cases
|