A dynamic auction that picks not only which ad to show but when to show it inside multi-turn LLM conversations extracts timing option value and—under the paper's assumptions—yields truthfulness in expectation and an 11% simulated net-revenue lift over fixed-timing baselines, though the gains rest on synthetic dialog simulations and a bid-independent click estimator.

Main Finding

LLM-OSDA is a dynamic cost-per-click auction that jointly decides when to insert a native ad in a multi-turn LLM conversation, which advertiser wins, how to render the sponsored response, and how much to charge. By separating bid-independent contextual click-quality estimation (an LLM layer) from a bid-aware committed mechanism (Bellman optimal stopping + envelope pricing) the mechanism (i) preserves a single-parameter structure that yields monotone expected-click allocations and thus truthful bidding in expectation under an exact Bellman oracle, (ii) prices the option value created by bid-dependent stopping via an envelope CPC integrated over expected discounted clicks, and (iii) can be implemented with a learned StopNet approximating Bellman values with provable bounds on incentive loss. On a simulated conversational-ad corpus, LLM-OSDA raises net revenue by about 11% over the strongest fixed-timing baseline while maintaining comparable user retention.

Key Points

• Problem novelty: native LLM ads convert selling from a fixed slot to a single insertion moment per session. Timing (stopping) is endogenous and bid-dependent, coupling allocation and timing; single-turn incentive arguments no longer suffice.
• Layered design:
  • Bid-independent LLM layer (quality estimator Gη and renderer πR) predicts turn-level click probabilities bqη,i(ht) and generates the native ad content. Crucially, these components do not observe bids.
  • Bid-aware committed mechanism uses the predicted quality plus bids to decide stopping (when to insert), winner allocation at that turn, and envelope payments.
• Optimal stopping (ideal): the Bellman recursion uses immediate exercise value Wt(ht; b) = max{0, max_i ψi(bi) qi(ht)}, continuation CVt = E[Vt+1 | Ht], and Vt = max{Wt, γ CVt}. Stop at the first turn where exercise value ≥ continuation value and positive.
• Envelope pricing: payments are computed from the expected discounted-click allocation xi(bi) (an expectation of γ^{τ−1} q_i(hτ) when advertiser i wins). For bidder i,
  • mi(bi) = bi xi(bi) − ∫_{θi}^{bi} xi(z) dz,
  • P_ENV = mi/xi (if xi>0). This integrates over bid levels and thus prices the timing option (global changes in stopping/allocation), not just a local continuation threshold.
• Theoretical guarantees:
  • Lemma: Under the ideal Bellman oracle, xi(bi; b−i) is weakly increasing in bi (monotonicity).
  • Theorem: Monotone expected-click allocation + envelope payment ⇒ DSIC in expectation and click-contingent individual rationality.
  • For learned StopNet: stopping decisions deviate from optimal only near the stopping boundary; the paper bounds the value/incentive loss due to approximation errors. Proposition gives a uniform bound: if the implemented allocation/payment deviate by ≤δx, δm from ideal, worst-case gain from misreporting ≤ 2θi δx + 2 δm.
• Implementation:
  • StopNet: shared trunk + separate stop/wait heads regress the two action values directly from a scalar summary of current-turn auction state. Deployed stopping rule mimics the ideal test on regressed values.
  • Iterative mechanism-aware refinement: alternate regenerating responses with the renderer, re-scoring with a user-simulation judge, refitting Gη and StopNet, and refining the renderer (best-of-N sampling), freezing committed tuples each round.
• Empirical findings:
  • Simulated dataset (no real users): ~14.9k three-turn shopping dialogues, 3k users, 100 advertisers. Click labels via LLM-as-judge user simulator.
  • Models: Qwen3-based encoder + MLP for quality estimator; MLP StopNet; Qwen3-4B fine-tuned renderer. Bids withheld from LLM layers.
  • LLM-OSDA increases net revenue by ~11% over best fixed-timing baseline (Always-Round-1) while keeping comparable retention/CTR. Diagnostics indicate revenue gains come from timing choices aligned with intent.

Data & Methods

• Formal setting:
  • Up to T turns per session, at most one native insertion, survival probability γ for continuation (so discounted click weight γ^{τ−1}).
  • n advertisers with private per-click values θi drawn i.i.d. from known Fi; advertisers submit CPC bids bi.
  • Mechanism is a (b, ω)-dependent mapping to (τ (stopping time), I (winner), P (payment)). Click probabilities qi(ht) depend on history and renderer but are bid-independent.
  • Allocation object: xi(b) = E[γ^{τ−1} qi(hτ) 1{i wins}]: expected discounted number of clicks for advertiser i.
• Ideal mechanism:
  • Uses exact qi and environment transitions to compute Bellman action values QS and QW and stop/wait decisions; winner chosen as argmax_i ψi(bi) qi(hτ).
  • Monotonicity proof leverages bid-independent transitions to show optimal expected-click allocation affine in yi = ψi(bi).
• Learned mechanism:
  • Gη (quality estimator) and πR (renderer) do not take bids; StopNet bQϕ regresses QS and QW from state summary st.
  • Deployed stopping rule: stop when regressed QS ≥ regressed QW > 0.
  • Envelope payment estimated by rollouts: compute bx_i(z) (estimated expected clicks using predicted qualities) over a bid grid and numerically integrate to get mi.
  • Theoretical bounds: stopping stability (errors only matter near boundary) and transfer of approximate IC with explicit bounds on incentive gain from misreporting in terms of approximation errors.
• Experimental details:
  • Dataset: 14,918 simulated three-turn shopping dialogs, 2,983 held-out test.
  • Model architectures: Qwen3-Embedding-0.6B + MLP for Gη; MLP StopNet; Qwen3-4B fine-tuned for renderer.
  • Baselines: Always-Round-1 (insert at turn 1), Always-Last-Round (insert at last turn), Random-Round, all sharing same Gη and envelope accounting; only stopping rule differs.
  • Metrics: Net revenue (expected-click-weighted revenue), reward, CTR, trigger distribution, envelope CPC, information rent, gross revenue. Iterative refinement rounds improve renderer and models.

Implications for AI Economics

• New dimension of monetization: converting a single-turn slot into a timing option inside multi-turn LLM interactions materially changes auction design. Platforms can extract additional revenue by optimally timing ad insertions as user intent evolves.
• Option-value pricing: bid-dependent stopping creates an option (when to insert). Proper pricing must integrate over bid-induced changes in expected discounted clicks rather than applying single-turn critical prices. This affects how platforms set CPCs and compute reserves.
• Design principle: maintain a bid-independent signal/estimation layer (LLM quality estimator + renderer) to preserve single-parameter structure and enable envelope-based incentive guarantees even when timing is endogenous. This separation can guide industrial architectures where generative models and auction engines coexist.
• Practical deployability: learned approximations (StopNet) can deliver most value with bounded incentive loss, but the mechanism relies on accurate, bid-independent intent/click estimators and realistic user models. Performance depends on the fidelity of the click model and the renderer’s effect on clicks.
• Policy and market effects:
  • Advertisers face a new strategic dimension (bids affect not just allocation but insertion timing). Envelope pricing mitigates strategic manipulation but relies on accurate estimation/rollouts.
  • Platforms gain a lever to trade off revenue vs. user experience (timing affects retention). The framework allows optimization of platform objectives beyond pure revenue (e.g., retention, long-run value).
  • Reliance on simulated judges and LLM-based click predictors raises robustness and fairness concerns: mis-calibration, distribution shift, or adversarial behavior could distort allocation/pricing.
• Limitations and open directions relevant to AI economics:
  • Current evaluation is on simulated conversations (three turns). Real-world user behavior, longer dialogues, multi-insertion settings, budget constraints, or correlated bidder types could change theoretical and empirical outcomes.
  • Latency/cost: integrating a StopNet and multiple LLM forward passes (for renderer or rollouts) introduces computational cost; industrial deployment needs engineering trade-offs.
  • Extensions: analyze budget-constrained advertisers, multi-slot/multi-insertion sessions, richer bidder type dynamics, and robustness to model mis-specification; explore revenue-optimal virtual-value transformations in the dynamic setting.
• Ethical/regulatory considerations: explicit native ads embedded in model responses alter user transparency; platforms should ensure labeling, consent, and avoid manipulative timing strategies that exploit vulnerable users.

Source: "LLM-OSDA: An Optimal-Stopping Dynamic Auction for Native Advertising in Multi-Turn LLM Conversations" (Fang et al., 2026). Code and experimental assets published by the authors (repo link in paper).