Runtime steering can covertly shape what language-model assistants say and cannot be distinguished from model bias by output-only audits; regulators and auditors therefore need tools that attest to the serving stack and inference policies, not just the model checkpoint.

Main Finding

The paper identifies and formalizes an inference-layer attribution problem: deployed language-model outputs can be systematically steered at inference time (via logit manipulation, activation steering, hidden prompts, retrieval, sampling configuration, etc.), and this runtime steering can produce served behavior that is observationally indistinguishable from behavior that would arise from different model weights. As a result, black‑box behavioral audits cannot reliably attribute observed biases or commercial influence to the model’s training/weights versus undisclosed runtime policies. The paper also defines "Probability Placement" — undisclosed, commercial reallocation of token-probability mass inside an ostensibly neutral assistant — and explores governance, auditing, and regulatory consequences.

Key Points

• Distinction emphasized: Model (weights, fine-tuning, RLHF) ≠ Deployed system (model + serving/inference stack). Many runtime interventions can change served outputs without changing θ.
• Mechanisms of inference-time steering covered: logit-level policies, activation engineering, decoding-time steering (PPLM, GeDi, DExperts, FUDGE), retrieval-augmentation, hidden system prompts, statistical watermarking, sampling changes.
• Formal logit model: given base logits zt, a runtime policy I can add z′t(w) = zt(w) + λ s_t(w | context, user, objective). The served distribution becomes proportional to Pθ(w) · exp(λ s(w)).
• Proposition (Observational Non-Identifiability): For any target served distribution Q with support ⊆ support(Pθ), there exists a logit-level policy I mapping Pθ to Q. Thus the same observed Q could result from (1) base model Pθ plus nontrivial inference policy I, or (2) a different model checkpoint θ′ that directly implements Q with identity inference. Output-only observation cannot distinguish these implementations.
• Probability Placement: a deployment pattern where advertisers or commercial principals influence the token-distribution of a general-purpose assistant without visually labeling the result as advertising. It shifts entity selection, attribute association, comparative salience, or persistence of recommendations while leaving alternatives possible (no hard suppression).
• Detection vs Attribution: detecting a distributional/semantic shift (via KL, total variation, embedding-difference metrics, repeated sampling experiments) shows a change in served behavior but not whether that change stems from model weights, hidden prompts, retrieval, activation, or logit policies.
• Practical threat models: state-enforced framing mandates, personalized persuasive steering conditioned on user receptivity, and covert commercial framing (Probability Placement).
• Operational notes: logit-level policies need not leave prompt trace tokens and are relatively low-overhead to run, making them plausible and stealthy in production.
• Governance proposals: require moving beyond purely behavioral audits to runtime transparency/attestation — e.g., confidential-computing/TEE-based measured execution, cryptographic attestations of model identity and active inference-policy hashes, and logging/exposure of inference-policy metadata.

Data & Methods

• Methods are primarily formal/conceptual and constructive, not empirical:
  • Formalization: autoregressive generation model Pθ(wt | x, w<t) via logits zt. Define an inference policy I as an additive transform on logits: z′t = zt + λ s_t(...). Served distribution Pθ,I ∝ Pθ · exp(λ s).
  • Constructive proof: For any target Q with Q(w)>0 ⇒ Pθ(w)>0, set s(w) = (1/λ) log [Q(w)/Pθ(w)]. This s produces Pθ,I = Q, proving observational non-identifiability.
• Metrics discussed for measuring divergence when probabilities are available: Kullback–Leibler divergence and total variation distance. When only samples are available, propose embedding-based semantic shift statistics (ΔS) computed from repeated generations across randomized prompts and interaction histories.
• Comparative qualitative taxonomy (Table 1): contrasts steering layers (weight mutation, hidden prompts, RAG, activation interventions, logit policies) by token cost, textual trace, and serving overhead.
• Threat models and deployment paradigms are analyzed conceptually with illustrative diagrams and examples rather than experimental deployment data.
• Related technical literature cited: controlled generation techniques (PPLM, GeDi, DExperts, FUDGE), activation steering/ITI, watermarking (Kirchenbauer et al., SynthID-Text), token-auctions for generative advertising, and empirical studies of LLM persuasion.

Implications for AI Economics

• New monetizable asset: probability mass in an assistant’s output distribution can be priced. Probability Placement suggests a market for covert influence where advertisers buy shifts in selection/attribute probabilities rather than discrete ad slots. This creates incentives for platforms to monetize steering without explicit ad units.
• Market design & mechanism differences: token-auction models (explicit) vs. covert runtime steering (implicit). Covert steering avoids ad-labeling/friction and can command different prices; it also raises moral hazard and information asymmetry problems.
• Externalities & consumer welfare: undisclosed steering reduces informed decision-making, can distort competition (favor incumbents or paying firms), and undermines user trust — negative externalities not internalized by platforms unless regulated.
• Competition policy & disclosure: covert steering complicates enforcement of advertising/competition law. Regulators may need to treat "served inference pipeline" as a productized object subject to transparency and nondiscrimination rules analogous to ad disclosures or search ranking transparency.
• Auditing and verification markets: need for attestation/certification services (trusted execution attestation, third-party verifiers) — creates supply for technical attestation, auditing firms, and certification bodies. These raise compliance costs that alter incumbent advantages (smaller firms might be burdened; large providers may internalize costs).
• Pricing and contracting implications: if platforms must attest their inference policy state, buyers/sellers of influence will be exposed; pricing models may shift from opaque probability placements to explicit auctions or subscription disclosures. Alternatively, suppliers may segment offerings into attestable (auditable) vs. non-attestable channels with different price points.
• Incentives for detection and signalling: Auditable labels (e.g., "advertising," "sponsored") become valuable trust signals; platforms that credibly signal non-covert behavior could capture demand premium. Conversely, covert steering incentives encourage arms races in detection tech (statistical tests, side-channel probes) and in defensive measures (confidential-computing to hide policies).
• Welfare and regulatory trade-offs: requiring runtime attestations improves transparency but increases operational costs and may reduce flexibility for safety steering done in good faith; policy must balance safety/consumer-protection against compliance burden.
• Research agenda for AI economics:
  • Empirically quantify the market value of probability mass shifts and user welfare impacts of covert vs. disclosed steering.
  • Model platform incentives under incomplete observability and regulation: do disclosure requirements reduce or shift socially harmful steering?
  • Design mechanism/contracting frameworks that internalize externalities (e.g., mandatory disclosure, certified attestation marketplaces, fines for non-disclosure).
  • Study how attestations and confidential-computing costs change market concentration and barriers to entry.

Takeaway for economists and policy-makers: behavioral audits alone are insufficient to identify whether observed biases or endorsements come from model training or from deployed inference policies. Economic analysis, regulation, and market design must therefore treat the deployed inference pipeline (not just model weights) as the relevant object for transparency, disclosure, and potential monetization.