Capping human overrides improves AI-supervised inventory decisions: a two-per-machine limit cuts stock by 1.28% with no sales loss, whereas unrestricted overrides also reduce sales by 1.19%; benefits concentrate among experienced workers and high-incentive SKUs.

Main Finding

A simple cardinality constraint on override rights — limiting downward overrides to two SKUs per machine per visit — materially improves human supervision of an autonomous replenishment algorithm. In a randomized field experiment (553 workers; >59,000 machines; ~4,000 SKUs), constrained overrides reduced inventory without hurting sales, whereas unrestricted (free) overrides reduced inventory but also significantly reduced sales and sales probability. The mechanism is "selective filtering": scarcity of override opportunities forces workers to prioritize higher-value, less noisy overrides.

Key Points

• Experimental arms: (1) No override (prohibit reductions to algorithmic suggestions); (2) Free override (unrestricted reductions); (3) Constrained override (limit of two downward overrides per machine).
• Aggregate effects (difference-in-differences):
  • Free overrides: inventory ↓ 1.95% (p<0.01); daily sales ↓ 1.19% and sales probability ↓ 1.38 percentage points (p<0.01).
  • Constrained overrides: inventory ↓ 1.28% (p<0.01); no significant change in sales quantity or sales probability.
• Local Average Treatment Effect (LATE) (policy assignment instrumenting SKU-level override status):
  • Constrained vs. no-override: inventory −3.45 items per affected SKU (p<0.01), no significant sales change.
  • Free vs. no-override: inventory −1.26 items, but sales probability fell by 10.55 percentage points (p<0.01).
• Behavioral changes under constrained policy:
  • Fewer SKUs overridden per visit, but larger reductions on overridden SKUs — consistent with prioritization of high-confidence overrides.
• Heterogeneous effects:
  • Largest gains for experienced (senior) workers, SKUs with stronger incentive alignment (top sellers), and growth-stage SKUs (where human local knowledge is more valuable).
  • Free overrides harmed performance even for senior workers.
• Personalized simulation:
  • A logistic model trained on constrained-group overrides used to accept/reject free-group overrides produced a simulated personalized policy that increased sales probability by 9.1 percentage points (p<0.01) over the original constrained group.
• Practical advantages: low cost, easy to implement, requires no algorithm redesign or added training; generalizes to many operational settings where routine autonomous decisions are occasionally overridden.

Data & Methods

• Setting: Fengyi Technology smart vending network (large-scale daily algorithmic replenishment).
• Sample: 553 frontline replenishment workers, >59,000 vending machines, ≈4,000 SKUs.
• Randomization: at worker level into three policy arms (no, free, constrained downward overrides). Upward overrides remained unconstrained.
• Primary data: worker-machine-day panel; SKU-level replenishment quantities, sales, and inventory measured between consecutive replenishments.
• Estimation:
  • Difference-in-differences for intent-to-treat worker-level effects on machine outcomes (inventory %, sales quantity, sales probability).
  • SKU-level LATE using randomized assignment as an instrument for whether a SKU was actually overridden to capture causal effect for compliers (the SKUs whose override status was shifted by policy).
  • Robustness checks: alternative override definitions, scaling, controlling for SKU features (price, discount), and intent-to-treat vs. per-protocol comparisons.
• Theory: simple Bayesian signal model (workers receive noisy, biased private signals) and a rational inattention variant; both imply that a cardinality constraint induces endogenous selection of stronger signals (selective filtering).

Implications for AI Economics

• Discretion design matters economically: structuring human override rights (not just algorithm quality) is a lever to improve system-level outcomes (inventory efficiency and sales).
• Trade-off between flexibility and noise: unrestricted human discretion can reduce inventory but at the cost of sales (welfare loss from noisy downward overrides). Constraining discretion can preserve human private information while reducing harmful noise.
• Mechanism-based policy: a low-complexity organizational rule (cardinality constraint) exploits human comparative judgment (ranking/prioritization) without requiring redesign of AI models or costly human retraining; this is a scalable governance tool for hybrid human-AI systems.
• Heterogeneity and targeting: benefits are concentrated where human private signals are stronger (experienced workers, high-incentive SKUs, growth-stage products). Optimal delegation policies should therefore allow differential discretion or personalized acceptance rules rather than one-size-fits-all.
• Role for data-driven acceptance systems: the successful simulation of a personalized accept/reject classifier suggests that firms can combine constrained human inputs and ML acceptance models to further improve performance — bridging organizational rules and predictive filtering.
• Broader regulatory/organizational insights: policy-makers and managers should consider not only algorithm transparency or accuracy but also the institutional design of human oversight (who can override, how often, and under what constraints) as a component of algorithmic governance.
• Open questions for AI economics research:
  • Generalizability: How do these results extend to higher-stakes domains (healthcare, finance) where overrides have asymmetric consequences?
  • Optimal constraint level: What is the welfare-optimal cap on overrides and how should it vary by task, worker quality, and product characteristics?
  • Strategic responses and dynamics: Do workers shift effort, gaming, or information acquisition in response to long-run constraints? How do feedback loops between overrides and algorithm updates affect equilibrium performance?
  • Integration with algorithmic design: How best to combine constrained human inputs with algorithmic retraining or active learning to exploit human signals without amplifying noise?

Limitations to note: single-domain field experiment (smart vending); constraint applied only to downward overrides; the exact cap (two per machine) was chosen operationally and may not be optimal elsewhere; potential for worker adaptation over longer horizons was not fully explored.