An LLM-driven pricing agent lifted gross merchandise value by 13.2% and improved ROI by 7.6% in a two-week randomized trial on Tao Factory, delivering interpretable, long-term-value-aligned price recommendations.

Main Finding

AIGP is a production-deployed hybrid framework that combines chain-of-thought LLM decision-making with an offline-RL–trained Long-Term Value Estimator (LTVE) and preference alignment (DPO) to produce interpretable, business-aligned dynamic pricing in large-scale e‑commerce. In large online A/B tests on Alibaba’s Tao Factory, AIGP improved 14‑day outcomes versus the production baseline by +13.21% GMV, +7.59% ROI, and +8.20% milestone achievement while returning human-auditable pricing rationales.

Key Points

• Architecture: LLM policy that outputs both CoT reasoning and a structured pricing action (JSON). The action is the daily discount change (dt − dt−1) constrained to a safe action set Asafe to meet business rules and ethics (identity-blind).
• Two-stage training:
  • Supervised fine-tuning (SFT) with teacher–student distillation: a large teacher (Qwen3‑235B) generates high-quality CoT + actions; a judge model filters outputs; a 30B student (Qwen3‑30B) is fine-tuned via LoRA for efficient production.
  • Preference alignment using Direct Preference Optimization (DPO), where preferences are created automatically by ranking candidate actions with the LTVE.
• Long-Term Value Estimator (LTVE): a critic-only offline RL value model trained on ~5 million historical transitions (6 months of logs) to score candidate pricing actions by estimated long-horizon reward.
• Reward design: category-normalized relative reward balancing milestone progress (14‑day cumulative GMV milestones) and ROI: r_t = λ1(prog_t − prog_ref_t) + λ2(log(1+ROI_t) − log(1+ROI_ref_t)). Normalization isolates strategy quality from intrinsic product demand.
• LTVE training details to reduce extrapolation and instability:
  • Decoupled value learning separating Vψ(s) (trained by expectile regression to upper quantiles) from Qϕ1, Qϕ2 critics.
  • Clipped double-critic ensemble q_min = min(Q̄ϕ1, Q̄ϕ2), multi-step TD bootstrapped from Vψ, target clipping and soft updates (see Algorithm 1).
• Preference construction & DPO:
  • Candidate actions sampled in a decision-only mode (with a [ACT_ONLY] control token) so LTVE ranks pure actions and the model’s DPO optimization focuses only on action tokens.
  • This avoids confounding long-form reasoning quality when creating preferences and concentrates alignment on actions.
• Safety/quality pipeline:
  • LLM-as-Judge evaluates teacher responses on Data Accuracy, Content Completeness, Internal Coherence, and Reasoning‑Decision Consistency; only perfect-scored examples retained.
  • Judge validated with 96.4% agreement vs experts on held-out samples.
• Empirical outcomes: significant offline and online gains in GMV, ROI, and milestone rates; produced interpretable pricing rationales for ops review and auditing.
• Practical deployment: tradeoff of reasoning quality and inference cost resolved via LoRA student fine-tuning (30B model) for large-scale serving.

Data & Methods

• Data
  • Historical production logs: ~5 million transitions over 6 months.
  • Sampling: 60% expert trajectories (top 30% cumulative GMV within comparable product groups) + 40% diverse non-expert for distribution coverage.
  • Inputs to prompts: structured features (GMV, ROI, PV, CTR, CVR, inventory, temporal flags, historical prices), unstructured text (titles, descriptions, reviews), and domain knowledge/diagnostics.
• Models & sizes
  • Teacher / Judge: Qwen3‑235B variants for demonstration generation and judging.
  • Deployed student: Qwen3‑30B (Qwen3‑30B‑A3B) fine-tuned with LoRA.
• Supervised fine-tuning (SFT)
  • Teacher generates CoT + JSON action responses; judge filters to high-quality exemplars across four dimensions.
  • Student optimized with cross-entropy loss on token sequences; outputs must end with valid JSON actions satisfying Asafe constraints.
• Long-Term Value Estimator (LTVE)
  • Critic-only architecture with two Q critics and a separate V network.
  • Training uses multi-step TD targets y^(n)t = sum{k=0}^{n-1} γ^k r_{t+k} + γ^n Vψ(s_{t+n}), clipped to numerical bounds.
  • Vψ trained by expectile regression (τ > 0.5) to bias toward better actions; double critics and target clipping mitigate overestimation/extrapolation.
• Preference alignment
  • For each prompt/state, sample M candidate actions (stochastic decoding with temperature + top-k/top-p); validate Asafe; score by LTVE; pick chosen/rejected pairs for DPO.
  • Use [ACT_ONLY] control during preference generation/training; during runtime, generate full CoT + action for transparency.
• Evaluation
  • Extensive offline evaluations and large-scale online A/B tests on real production traffic (Tao Factory).
  • Key reported metrics: % changes in 14-day cumulative GMV, ROI, milestone achievement rate versus baseline.

Implications for AI Economics

• Demonstrates a practical pathway to align LLM-driven decisions with long-horizon firm objectives:
  • Hybrid approach (LLM reasoning + value-model scoring) bridges interpretability and long-term economic optimization, overcoming typical LLM short-sightedness.
• Better use of unstructured data in economic decision-making:
  • Product textual signals (titles, reviews, descriptions) can be integrated effectively by LLMs to inform pricing beyond what structured elasticity models capture—useful for cold-start / long-tail items.
• Scalable preference generation:
  • Using an offline RL value estimator to automatically create preference pairs enables DPO-style alignment at scale without requiring costly human labeling of long-term outcomes.
• Production trade-offs:
  • Distillation (teacher→student) and LoRA enable deploying capable LLMs at lower inference cost while preserving CoT interpretability—relevant to firms weighing model size, latency, and economics.
• Market and welfare considerations:
  • Automated, interpretable pricing that optimizes long-term firm metrics may shift market dynamics—possible effects include intensified competition on lifecycle pricing strategies and altered consumer surplus profiles.
  • Authors mitigate direct price discrimination by being identity-blind, but broader concerns remain (e.g., opacity of automated pricing strategies, strategic firm interactions).
• Methodological caveats for economic researchers and practitioners:
  • Offline RL–based value estimators depend heavily on logged-policy coverage; extrapolation risk remains for actions outside historical support. The paper addresses this with decoupled value learning and clipping, but residual risk exists.
  • Reward specification (choice of reference baselines, λ weights, milestone framing) materially affects learned behavior; economic analysis must inspect these institutional choices.
  • Generalizability: results are from one large platform (Tao Factory); cross-platform external validity and longer-horizon consumer welfare consequences need further study.
• Research opportunities:
  • Study second-order market effects from widespread adoption of LLM-based long-term pricing (price dynamics, consumer expectations, entry/exit).
  • Evaluate welfare and distributional impacts when automated pricing targets long-run firm objectives.
  • Compare alternative alignment strategies (human preference labels, counterfactual estimators, online safe RL) and their economic costs/benefits.
  • Investigate robust evaluation and audit frameworks to certify alignment and non-discrimination in automated pricing systems.

If you want, I can extract key equations (reward, LTVE loss terms, DPO objective), produce a simplified diagram of the training pipeline, or outline follow-up experiments to test external validity and welfare impacts.