Reading an LLM's token probabilities, not just its sampled answers, extracts far more information: 'model belief' matches the mean choice asymptotically but cuts sampling variance and — in a demand simulation — reduced compute needed for accurate estimates by about 20-fold.

Main Finding

The paper introduces "model belief"—the LLM’s normalized token-level probability vector over a predefined choice set at the pivot step—as a richer, single-run measure of an LLM’s internal decision distribution. The authors prove that model belief is asymptotically equivalent to the long-run average of model choices but is strictly more statistically efficient: it has lower variance and faster convergence. These efficiency gains extend to smooth downstream quantities (e.g., market shares, price sensitivity). In a demand-estimation application (diaper-brand choice by an LLM), model belief (vs. repeated sampling of model choices) predicts and explains choice better in finite samples and reduces required computation by about a factor of 20.

Key Points

• Definitions
  • Model choice (c): the discrete token chosen on a generation run that resolves the decision among alternatives set M.
  • Model belief (b): the normalized probability mass that the LLM assigns (via softmax of logits) to each alternative in M at the pivot token position τ of that run: b(j) = Pτ(j) / ∑j′∈M Pτ(j′).
• Theoretical results
  • Proposition 1 (Asymptotic equivalence): Under unbiased sampling, the expected model belief equals the true choice distribution: E[b] = Pr(c = ·). Thus model belief and mean model choice converge to the same limit.
  • Proposition 2 (Efficiency): The empirical model-belief estimator has weakly smaller sampling variance than the empirical model-choice estimator; it requires no more runs (and typically fewer) than resampling to attain a target estimation accuracy. Equality holds only when model belief is almost surely deterministic.
  • Propositions 3–4: For any smooth (C^1) mapping g from the choice simplex to downstream quantities (e.g., price sensitivity), the plug-in estimator using model belief is consistent and has weakly smaller asymptotic covariance / faster convergence than the plug-in using model choice.
• Intuition: model belief uses the model’s internal probability information at the time of choice and therefore retains continuous, informative signals that are lost when observing only the sampled, discrete choice.
• Empirical evidence
  • Context: LLM-based discrete-choice demand estimation (diaper-brand choice across prices).
  • Finding: In limited-run regimes typical of applied research, model belief delivers more accurate and precise estimates of price sensitivity and choice probabilities; it outperforms model-choice resampling in in-sample fit and out-of-sample prediction.
  • Computation: To reach comparable accuracy, model choice resampling required roughly 20× more runs than collecting model belief (i.e., model belief reduces compute roughly twenty-fold in the reported setting).

Data & Methods

• Formal setup
  • Input prompt x1:T0; auto-regressive generation produces logits zt and probabilities Pt = softmax(zt) at each step t.
  • Identify pivot token τ: the first generated token that unambiguously maps to an alternative in choice set M.
  • For run r, record choice c(r) = yτ and belief b(r) constructed from P(r)τ restricted/normalized to M.
• Estimators
  • Empirical choice share: ĉ(N)(j) = (1/N) ∑r 1{c(r)=j}.
  • Empirical belief share: b̂(N)(j) = (1/N) ∑r b(r)(j).
  • Plug-in for downstream g(·): θ̂b = g(b̂(N)); θ̂c = g(ĉ(N)).
• Key assumptions
  • Unbiased sampling: sampling tokens proportional to their underlying probabilities (realized in practice by temperature = 1 and full sampling dictionary).
  • Independent runs (each generation run treated as an independent draw conditional on the same prompt).
  • Pivot identification: ability to determine the pivot token for each run (including multi-token alternatives via shortest unique prefix).
• Theoretical techniques
  • Law of total expectation to relate conditional pivot-step probabilities to unconditional choice probabilities.
  • Variance comparisons and convergence-rate arguments for empirical averages; Loewner dominance for plug-in covariance comparisons.
• Empirical design (illustrative)
  • LLM simulated diaper choices under varying prices; both the sampled choice and token-level probabilities at pivot captured per run.
  • Evaluated in-sample fit and out-of-sample predictive power, and computed run counts needed to achieve a given accuracy.

Implications for AI Economics

• Efficiency gains in synthetic-data research
  • Researchers using LLMs as synthetic agents can extract substantially more information per run by recording token-level probabilities (model belief) rather than treating each generation as a single binary/multiway observation—cutting compute and API costs sharply.
• Better inference from fewer runs
  • Econometric estimation (demand, elasticities, discrete-choice models) benefits from lower variance estimators; using model belief improves finite-sample precision and predictive validity for counterfactual and policy analyses that rely on LLM-generated choices.
• Improved experimental design with LLM agents
  • Model belief enables more powerful tests and tighter confidence sets for treatment effects, allowing more ambitious designs or more scenarios per budget.
• Broader methodological change
  • Encourages a default practice change in applied AI-economics work: log and use token-level logits/probabilities (when available) and adopt unbiased sampling protocols (temperature≈1) to realize theoretical gains.
• Caveats, limitations, and open issues
  • Access to internals: Many deployed APIs either do not expose logits or add nontransparent sampling layers; practical adoption requires access to token probabilities or model configurations that return them.
  • Unbiased-sampling requirement: The theoretical efficiency claims require sampling proportional to Pt (temperature 1 and full support). Deterministic decoding (greedy) or high-temperature distortions may invalidate assumptions and change pivot behavior.
  • Pivot identification & multi-token choices: Needs careful implementation when alternatives are multi-token, or when pivot position varies; misidentification can bias belief estimates.
  • Calibration and external validity: Model belief uses the model's internal probabilities, which reflect the model’s training/data and may be miscalibrated relative to human probabilities. Efficiency gains help estimate the model's own choice distribution more precisely, but do not by themselves guarantee better alignment with human behavior.
  • Dependence on model architecture/training: Results are derived under general probabilistic auto-regressive generation; empirical magnitude of gains likely depends on model size, calibration, and prompt design.
• Practical recommendation
  • When using LLMs to simulate decision behavior, collect and use model belief (normalized pivot-token probabilities) as the primary per-run outcome whenever token-level probabilities are available and unbiased sampling is feasible. This yields more efficient estimators for choice distributions and downstream structural parameters, reducing compute and improving statistical power.

If you want, I can: - Draft a short code template (pseudo-code) showing how to extract pivot-token probabilities and construct model belief from common LLM outputs. - Summarize potential checks (e.g., calibration diagnostics, sensitivity to temperature) and implementation heuristics for robust adoption.