A Q-value–regularized Decision Transformer for auto-bidding outperforms prior RL and generative baselines and delivers a 3.27% lift in ad GMV and 2.49% higher ad ROI in large-scale A/B testing, suggesting algorithmic refinements can materially boost platform and advertiser outcomes.

Q-Regularized Generative Auto-Bidding: From Suboptimal Trajectories to Optimal Policies

Mingming Zhang, Na Li, Zhuang Feiqing, Hongyang Zheng, Jiangbing Zhou, Wang Wuyin, Sheng-jie Sun, XiaoWei Chen, Junxiong Zhu, Lixin Zou, Chenliang Li · January 06, 2026

arxiv other medium evidence 8/10 relevance Summary only summary available; pdf_status=not_found Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: July 29, 2026

Mingming Zhang unresolved corpus identity
Na Li unresolved corpus identity
Zhuang Feiqing unresolved corpus identity
Hongyang Zheng unresolved corpus identity
Jiangbing Zhou unresolved corpus identity
Wang Wuyin unresolved corpus identity
Sheng-jie Sun unresolved corpus identity
XiaoWei Chen unresolved corpus identity
Junxiong Zhu unresolved corpus identity
Lixin Zou unresolved corpus identity
Chenliang Li unresolved corpus identity

Semantic Scholar

Latest observation: August 25, 2026

Mingming Zhang provider ID
Na Li provider ID
Feiqing Zhuang provider ID
Hong-Kai Zheng provider ID
Jiangbin Zhou provider ID
Wuyin Wang provider ID
Shengjie Sun provider ID
Xiaowei Chen provider ID
Junxiong Zhu provider ID
Lixin Zou provider ID
Chenliang Li provider ID

QGA — a Decision Transformer augmented with Q-value regularization and Q-guided dual exploration — outperforms prior auto-bidding methods on benchmarks and raises Ad GMV by 3.27% and Ad ROI by 2.49% in a large-scale production A/B test.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

Semantic Scholar · Observed August 25, 2026

View corpus context

With the rapid development of e-commerce, auto-bidding has become a key asset in optimizing advertising performance under diverse advertiser environments. The current approaches focus on reinforcement learning (RL) and generative models. These efforts imitate offline historical behaviors by utilizing a complex structure with expensive hyperparameter tuning. The suboptimal trajectories further exacerbate the difficulty of policy learning. To address these challenges, we proposes QGA, a novel Q-value regularized Generative Auto-bidding method. In QGA, we propose to plug a Q-value regularization with double Q-learning strategy into the Decision Transformer backbone. This design enables joint optimization of policy imitation and action-value maximization, allowing the learned bidding policy to both leverage experience from the dataset and alleviate the adverse impact of the suboptimal trajectories. Furthermore, to safely explore the policy space beyond the data distribution, we propose a Q-value guided dual-exploration mechanism, in which the DT model is conditioned on multiple return-to-go targets and locally perturbed actions. This entire exploration process is dynamically guided by the aforementioned Q-value module, which provides principled evaluation for each candidate action. Experiments on public benchmarks and simulation environments demonstrate that QGA consistently achieves superior or highly competitive results compared to existing alternatives. Notably, in large-scale real-world A/B testing, QGA achieves a 3.27% increase in Ad GMV and a 2.49% improvement in Ad ROI.

Summary

Main Finding

QGA (Q-value regularized Generative Auto-bidding) is a novel approach that combines a Decision Transformer (DT) backbone with a Q-value regularization using double Q-learning. It jointly optimizes policy imitation and action-value maximization, and adds a Q-value–guided dual-exploration mechanism (multi return-to-go conditioning + locally perturbed actions). QGA outperforms or matches existing auto-bidding methods on public benchmarks and simulations, and in large-scale production A/B testing yields a 3.27% increase in Ad GMV and a 2.49% improvement in Ad ROI.

Key Points

Problem: Existing RL and generative-model auto-bidding methods tend to imitate historical (often suboptimal) behavior, require complex models and heavy hyperparameter tuning, and struggle with suboptimal trajectories in offline data.
Core idea: Integrate an action-value (Q) regularizer into a Decision Transformer to combine behavioral cloning (imitation) with explicit action-value maximization.
Double Q-learning: Used to stabilize and reduce overestimation bias in the Q-value module.
Dual exploration:
- Condition DT on multiple return-to-go targets to explore different performance levels.
- Apply local perturbations to actions to generate candidate alternatives.
- Use the Q-value module to evaluate and guide selection among candidates (safe, principled off-distribution exploration).
Benefits: Leverages offline dataset experience while mitigating the negative influence of suboptimal trajectories; enables controlled exploration beyond data distribution.
Empirical performance: Superior or competitive on public benchmarks and simulations; significant real-world gains in large-scale A/B tests (Ad GMV +3.27%, Ad ROI +2.49%).

Data & Methods

Model architecture:
- Backbone: Decision Transformer (sequence-modeling approach that conditions on return-to-go).
- Q-module: Learned action-value estimator trained with double Q-learning to provide reliable action evaluation.
- Regularization: Q-value term added to the generative (DT) loss to push the policy toward actions with higher estimated value while preserving imitation of good behavior.
Exploration mechanism:
- Multi-target conditioning: DT is conditioned on several different return-to-go targets to produce diverse candidate trajectories.
- Local action perturbation: Small modifications to DT outputs create nearby candidate actions for evaluation.
- Q-guided selection: The Q-module scores candidates; selection favors candidates with higher estimated value, enabling safe policy improvement outside the support of offline data.
Training and evaluation:
- Offline datasets (historical bidding logs) for imitation and Q learning.
- Benchmarks: public offline RL/auto-bidding benchmarks (paper reports consistent gains).
- Simulation environments to test robustness and exploration behavior.
- Production: large-scale A/B testing in a real e-commerce advertising system (reported GMV and ROI improvements).
Implementation notes (implied):
- Balances imitation and value objectives via a regularization coefficient (trades off conservatism vs. improvement).
- Double Q-learning reduces overestimation and stabilizes the Q-guidance.

Implications for AI Economics

Platform revenue and advertiser return: Measurable improvements in Ad GMV and Ad ROI suggest platforms can increase transaction volume and advertiser profitability simultaneously, affecting bid pricing dynamics and budget allocation.
Market efficiency: Better bidding policies reduce wasted spend on low-value placements and improve allocative efficiency of ad inventory, potentially increasing consumer-relevant matches.
Reduced tuning and engineering cost: By combining generative imitation with value-guided improvement, QGA may lower the practical burden of hyperparameter tuning compared to complex RL pipelines, speeding deployment and iteration.
Safer offline policy improvement: The Q-guided safe exploration mechanism provides a principled way to evaluate off-distribution actions, reducing risk when optimizing policies from historical (and possibly suboptimal) data — important for high-stakes marketplace environments.
Strategic behavior and competition: More effective auto-bidding can change competitive dynamics (e.g., bid levels, advertiser strategies); platforms should monitor implications for fairness and potential concentration effects.
Policy and regulation considerations: Improved automated bidding efficacy may increase ad spend efficiency but also amplify automated strategic bidding; regulators and platforms might need to reassess transparency, disclosure, and market-power concerns.
Research directions: Opportunities to analyze broader welfare impacts (consumer surplus vs. platform/advertiser surplus), generalization across market segments, robustness under nonstationary environments, and costs of maintaining accurate Q-estimators under distributional shifts.

Assessment

Paper Typeother Evidence Strengthmedium — The paper reports consistent wins on public benchmarks and simulations and — importantly — a large-scale A/B test with positive percent gains in platform outcomes (3.27% Ad GMV, 2.49% Ad ROI), which provides credible causal evidence for the deployed setting; however, the summary lacks key experimental details (sample size, randomization procedure, statistical significance, duration, heterogeneity checks, and pre-registration) needed to judge robustness and rule out operational confounders. Methods Rigormedium — The technical approach is novel (Decision Transformer with Q-value regularization and Q-guided exploration) and evaluated across offline benchmarks, simulators, and production A/B, which is a strong mixed-methods strategy; but the reported description omits important methodological details (hyperparameter search procedures, ablation diagnostics, confidence intervals and hypothesis tests for A/B results, treatment of interference, and advertiser-level/subgroup analyses) that would be expected for high rigor in an applied-economics claim. SampleExperiments use (a) public offline auto-bidding benchmarks and custom simulation environments for algorithmic comparison, (b) offline historical advertiser logs for imitation learning components, and (c) a large-scale production A/B test on a real ad platform comparing QGA versus existing auto-bidding policies (summary provides percent gains but does not report number of advertisers/campaigns, traffic split, test duration, or geographic/sector composition). Themesproductivity innovation IdentificationMethodological identification for algorithmic performance comes from offline benchmark comparisons and simulation experiments; causal inference for real-world impact is claimed via a large-scale randomized A/B test that assigns traffic between QGA and baseline bidding policies (randomized assignment provides causal identification of differences in Ad GMV and Ad ROI). GeneralizabilityResults are platform- and auction-mechanism-specific (may not transfer to platforms with different auction rules, pricing, or inventory)., Advertiser composition and vertical (e-commerce) likely drive gains; may not generalize to non-e-commerce advertisers or small-budget campaigns., Operational deployment details (budget pacing, floor prices, bidder constraints) can affect performance and are not fully described., Lack of reported subgroup/heterogeneity analysis limits understanding of which campaigns benefit most., Temporal/contextual factors (seasonality, competitive responses) may affect durability of reported gains.

Claims (9)

Claim	Direction	Outcome	Confidence & Evidence	Details
We propose QGA, a novel Q-value regularized Generative Auto-bidding method that plugs a Q-value regularization with double Q-learning strategy into the Decision Transformer backbone. Other	null_result	method architecture / algorithmic design	Reading fidelity high Study strength speculative	not reported 0.02
This design enables joint optimization of policy imitation and action-value maximization, allowing the learned bidding policy to both leverage experience from the dataset and alleviate the adverse impact of the suboptimal trajectories. Decision Quality	positive	policy performance / policy quality (improvement by combining imitation and value maximization)	Reading fidelity high Study strength medium	not reported 0.12
We propose a Q-value guided dual-exploration mechanism in which the Decision Transformer model is conditioned on multiple return-to-go targets and locally perturbed actions to safely explore policy space beyond the data distribution. Other	null_result	exploration strategy for policy learning	Reading fidelity high Study strength speculative	not reported 0.02
The exploration process is dynamically guided by the Q-value module, which provides principled evaluation for each candidate action. Decision Quality	positive	action evaluation during exploration (quality of candidate actions)	Reading fidelity high Study strength speculative	not reported 0.02
Experiments on public benchmarks and simulation environments demonstrate that QGA consistently achieves superior or highly competitive results compared to existing alternatives. Decision Quality	positive	performance on benchmarks / simulation (relative performance vs baselines)	Reading fidelity high Study strength medium	not reported 0.12
In large-scale real-world A/B testing, QGA achieves a 3.27% increase in Ad GMV. Firm Revenue	positive	Ad GMV (Gross Merchandise Volume)	Reading fidelity high Study strength medium	3.27% increase in Ad GMV 0.12
In large-scale real-world A/B testing, QGA achieves a 2.49% improvement in Ad ROI. Firm Revenue	positive	Ad ROI (return on investment for advertising)	Reading fidelity high Study strength medium	2.49% improvement in Ad ROI 0.12
Current approaches focus on reinforcement learning (RL) and generative models and imitate offline historical behaviors by utilizing a complex structure with expensive hyperparameter tuning. Other	null_result	characteristics of prior methods (model complexity and tuning cost)	Reading fidelity high Study strength low	not reported 0.06
Suboptimal trajectories in offline datasets exacerbate the difficulty of policy learning for auto-bidding. Decision Quality	negative	impact of suboptimal trajectories on policy learning difficulty	Reading fidelity high Study strength medium	not reported 0.12