Digests

This weekly digest tracks what is NEW or CHANGED in AI-economics research. For the cumulative state of evidence on any topic, see the /syntheses pages. A single study rarely overturns a body of evidence.

From Alex

A few project notes from me this week. I updated foundryR, the R package I use for working with Microsoft Foundry from R; made another pass on the OES Dashboard, which turns Occupational Employment and Wage Statistics into a browsable workforce-data tool; and added more to onet2r, my R package for pulling and working with O*NET data.

We also published a new NFW Reader piece on using Copilot in Word for causal-impact work.

The Delta

Coming in, Employment Level leaned mixed (154 papers); this week, the signal is mixed. - Strengthened: simple governance constraints on human–AI supervision, via a large retail randomized controlled trial (RCT) and engineering controls for coding agents, help preserve sales while trimming inventory in retail and increase backdoor detection in coding agents in tests compared with unconstrained interaction. - Newly observed: in open-source software, adopting AI coding agents is associated with slightly higher code complexity but no detectable reduction in newcomer inflows or retention, tempering crowd-out concerns in that setting. - Challenged: symbolic, bot-delivered social nudges fail to boost and can suppress downstream engagement on Reddit, pushing against assumptions that cheap automated incentives reliably lift activity.

What Moved & What Held

Coming in, the standing view was that AI can raise productivity and resilience when paired with simple governance and complementary human capital, and that poorly designed human–AI interaction can blunt gains or backfire. Evidence was strongest for targeted operational rules and developer tools tied to higher throughput, with heterogeneous effects depending on users, incentives, and domain reliability.

This week adds two field experiments and several quasi-experimental panels that push on design details: a two-override cap preserved sales while trimming inventory, while engineering controls raised backdoor detection in coding agents in controlled tests; open-source projects adopting AI agents showed no detectable reduction in newcomer participation despite slightly higher code complexity; and bot-driven symbolic awards did not raise and sometimes reduced engagement. Still holds this week: the need for deployment design over blanket delegation, the importance of user traits and training for realizing gains, and wide heterogeneity by setting.

Top Papers

• Confirms · established Two-override cap preserves sales while trimming inventory in large retail field RCT (Minda Zhao, Brian Rongqing Han, Xin Chen, Tao Zhu; randomized controlled trial, high evidence) - In a major Chinese smart-vending retailer (553 workers across 59k+ machines), a randomized cap of two downward overrides per machine reduced inventory by about 1.28% without reducing sales, whereas unlimited free overrides reduced inventory but also reduced sales. This aligns with the standing view that constrained human oversight improves AI-supported operations relative to unconstrained overrides. - So what: If this holds, unmanaged override policies can quietly erode margins through sales leakage even when inventory looks leaner, making governance design a first-order operational risk. - Full numbers

• Extends · established Symbolic awards from users and bots generally fail to boost downstream activity on Reddit; bot lottery awards can suppress it (Hiroki Oda, Kinga Makovi, Taha Yasseri, Milena Tsvetkova; randomized field experiment, high evidence) - A randomized field experiment on the Reddit platform finds symbolic awards do not raise recipients' subsequent activity or downstream impact, and awards delivered by apparent bot accounts using a lottery rationale can reduce it. This extends evidence on governance-by-nudge, showing simple automated incentives can backfire in large social systems. - So what: If this generalizes, product and policy teams face the risk that cheap automated nudges deliver no lift and can depress engagement, distorting key performance indicator (KPI) assumptions. - Full numbers

• New · suggestive AI coding agents raise code complexity slightly but do not crowd out OSS newcomers (Weiwei Xu, Xuanning Cui, Hengzhi Ye, Minghui Zhou; difference-in-differences (DiD), quasi-experiment, medium evidence) - Using GitHub data for 1,888 adopter projects (603 with pre-period) in a difference-in-differences design, adoption is associated with modestly higher per-function code complexity but no detectable reduction in newcomer inflow, onboarding, or retention relative to matched controls. This provides a setting-specific check against broad crowd-out fears. - So what: In this sample, newcomer pipelines did not thin after agent adoption, but slightly higher complexity raises the risk that maintainability costs reappear later if onboarding quality is overstated. - Full numbers

Also Notable

• Extends · suggestive Generative AI adoption is associated with higher firm supply‑chain resilience and greater upstream supplier concentration (Shi Jun, Yijun Chen, Wenli Hu) - Chinese A-share panel with fixed effects and an instrumental variables (IV) strategy links AI adoption to higher resilience via innovation and investment, with an apparent trade-off of greater supplier concentration.

• Extends · suggestive Enterprise mandate and voluntary adoption are associated with about double merged PR throughput tied to AI tool use (H. He, Shyam Agarwal, Yegor Denisov-Blanch, Pavel Azaletskiy, Sanmi Koyejo, Bogdan Vasilescu) - Longitudinal case (802 developers, 196k pull requests) associates an AI-use mandate and cumulative individual usage with ~2.09x merged pull-request (PR) throughput.

• New · suggestive Firms with high exposure to a high‑frequency AI consumption factor exhibit a persistent stock return premium (Nicola Borri, Yukun Liu, Aleh Tsyvinski) - Asset-pricing analysis using 380 trillion tokens constructs an AI factor, reporting an AI premium concentrated in intensive, paid, closed-source usage.

• Extends · suggestive Steerability via constraints: a substrate for scalable oversight of coding agents (Thomas Winninger) - Applying access controls, network policies, and a small documentation command-line interface (CLI) raised recall of injected backdoors from roughly 55% to 91% in controlled agent tests.

• Extends · suggestive Model capability and reasoning effort, more than auxiliary tools, correlate with first‑try success in agentic code runs (Achint Mehta) - Across 90 agent runs, higher-capability models and greater internal reasoning effort are associated with better first-try reliability, while extra testing tools came with higher cost without improving first-pass outcomes in those runs.

• Extends · suggestive AI application associates with higher ESG scores in Chinese firms, especially among lower-tech and cleaner industries (Haixia Feng, Renbo Shi, Qingjin Wang) - Firm panel fixed-effects links AI application to better environmental, social, and governance (ESG) metrics, mediated by human capital and green innovation, with heterogeneous effects across firms and regions.

• Extends · suggestive China's Low-Carbon City Pilot Policy is associated with higher local AI industry development (Luyuan Tang, Shiyao Xie, Yuan Xu, Ziwen Sun) - Staggered difference-in-differences (DiD) across 285 cities (2007–2022) finds pilots correlate with ~16.9% higher local AI enterprise levels, consistent with environmental and AI growth coexisting.

• New · descriptive Top LLMs score ~82.7% on aviation operations benchmark, below informal expert reference of ~95% (Alex Brooker, Tim Hughes) - The 300-question Pre-Flight benchmark suggests a domain reliability gap relative to practitioner performance for non-safety-critical tasks among large language models (LLMs).

• New · framework A contextual-bandit oversight game with two-sided informational asymmetry (Yunjin Tong) - Formal model of an online decision process where actions are chosen based on context delineates when myopic humans decline oversight under private information and how signaling or repetition could mitigate avoidable harm.

• New · descriptive Personas can lead agentic AIs to reproduce human analytical variation and produce divergent conclusions (Jiacheng Miao, Jonathan K. Pritchard, James Zou) - Introduces m-value (a metric for variation) and an Agentic Bootstrap (a way to sample plausible analysis paths) to quantify the range and extremity of agentic analytical choices, showing personas can lead agentic AIs to reproduce human-like analytical variation.

What Moved

• Governance constraints in operations: A large retail RCT moves evidence from suggestive to established that simple caps on human overrides can keep sales intact while reducing inventory, contrasting with sales losses under free overrides; paired with engineering controls that raised backdoor detection in coding agents in controlled tests, this strengthens the case for small, rule-based guardrails in production.

• Developer pipelines and adoption externalities: A GitHub quasi-experiment newly narrows concerns that AI coding agents crowd out newcomers, showing no detectable decline in inflow or retention in adopters while complexity edges up; this tempers displacement narratives for OSS communities but flags maintainability as a slower-moving margin.

• Automated social influence: A Reddit field experiment challenges the expectation that low-cost, symbolic bot incentives boost engagement, documenting null or negative effects; this adds a caveat to product strategies that assume easy gains from automated awards, and it diverges from settings where badges or reputation sometimes help.

• Investor pricing of AI exposure: An asset-pricing study using token consumption data adds a measurable AI premium concentrated in intensive, paid usage; while correlational, it sharpens where markets appear to be pricing AI demand and suggests heterogeneity that prior broad AI baskets masked.

Contested & Watch

• Who actually gains from hybrid AI assistance - Finding: A small real-money forecasting pilot reports that only a minority with collaborative cognitive traits improve with AI assistance, while most defer or rubber-stamp (pilot sample). - Standing evidence: Several reviews and panels (medium strength) find productivity and innovation gains with AI, but emphasize heterogeneity and complementarities rather than universal uplift. - Watch: Larger, preregistered field trials linking measured traits to task-level gains and retention over quarters.

• Are simple oversight caps sufficient across domains - Finding: In Chinese retail (553 workers; 59k+ machines), a two-override cap cuts inventory without sales loss, while free overrides reduce sales. - Standing evidence: Prior causal and engineering studies (medium strength) favor constrained oversight, but theory with two-sided information shows regions where myopic humans still under-oversight. - Watch: Replications in logistics, healthcare, and finance with varied thresholds and asymmetric costs.

• Do AI coding agents erode OSS newcomer pipelines over time - Finding: GitHub DiD on 1,888 adopters (603 with pre) finds no detectable reduction in newcomer inflow, onboarding, or retention, despite modest complexity increases. - Standing evidence: Commentaries and surveys (low-to-medium strength) raise crowd-out concerns, with limited causal tests. - Watch: Multi-year composition, mentorship intensity, and defect rates for adopter vs control projects.

• Can cheap automated nudges reliably lift platform engagement - Finding: A Reddit randomized field experiment shows symbolic awards do not raise and bot-lottery awards can reduce downstream activity. - Standing evidence: Mixed platform experiments (medium strength) on badges and reputation show context-dependent effects, sometimes positive and often small. - Watch: Heterogeneity by community type, transparency of automation, and tie-in to moderation or social capital.

• Is there an economically meaningful AI return premium - Finding: An AI factor built from 380 trillion token-consumption events is associated with a persistent premium, concentrated in paid, closed-source use. - Standing evidence: Tech factor and AI basket returns (medium strength) point to valuation effects, but causality and persistence are unsettled. - Watch: Out-of-sample performance once the factor is publicly replicable, and event studies around exogenous usage shocks.

Methods Spotlight

• Large-scale field RCT of governance rules in operations: A Simple Solution to Improving Human Supervision of Algorithms: Evidence from Smart Vending. Directly tests a simple override-cap policy across 553 workers and 59k+ machines, strengthening causal claims about human–AI supervision design.

• Token-level AI consumption factor construction: AI Premium. Uses 380 trillion tokens of usage data to build an asset-pricing factor, opening a quantifiable path to measure firm exposure to AI demand.

• Agentic Bootstrap for analytical variability: The Agentic Garden of Forking Paths. Formalizes a way to enumerate and score plausible agentic analysis paths, making hidden degrees of freedom measurable.