HCI experiments reveal frequent over-reliance on AI advice, but most studies use unrealistic decision tasks; designing interventions to foster appropriate reliance will boost engagement yet can come at a cost to efficiency or scalability.

Do People Appropriately Rely on AI-Advice? An Analytical Review of HCI Research on Human-AI Decision-Making

Muhammad Raees, Vassilis-Javed Khan, Ioanna Lykourentzou, Konstantinos Papangelis · Fetched July 13, 2026 · International Conference on Human Factors in Computing Systems

semantic_scholar review_meta n/a evidence 7/10 relevance Summary only summary available; pdf_status=paywall DOI Source PDF

The review finds that HCI studies commonly show people over-rely on AI advice while often failing to analytically engage with it, notes that many experiments lack realistic decision contexts, and argues that improving 'appropriate reliance' requires interventions that increase user engagement but impose trade-offs.

AI systems are increasingly being positioned to assist people in decision-making. However, recent empirical studies show critical concerns that people over-rely on AI advice without analytically engaging with it. While HCI research explores how people rely on AI advice, we argue that it largely overlooks an important aspect: replicating realistic decision-making scenarios. Human-AI interaction factors influence people’s reliance on AI advice. To understand human-AI interaction factors and their interplay, we conducted an analytical review of recent studies in human-AI reliance literature. We analyzed the decision-making tasks in research and their validity in application-grounded contexts. Our findings show that user engagement is a precious commodity for relying on AI advice; however, it comes at a cost. We also discuss factors contributing to “appropriate reliance”, existing research gaps, and recommendations for intervention design for human-AI reliance. Our work contributes to the critical body of research on building appropriate reliance on AI advice.

Summary

Main Finding

People frequently over-rely on AI advice and often fail to analytically engage with it. Human-AI interaction research has identified many factors that shape reliance, but it has not sufficiently replicated realistic, application-grounded decision contexts. The review shows that user engagement is a scarce, valuable resource for achieving appropriate reliance on AI advice — and increasing engagement typically imposes costs or trade-offs.

Key Points

Empirical studies document a persistent tendency for users to accept AI recommendations without deep analytic scrutiny.
HCI literature has investigated many interaction factors (e.g., explanations, feedback, interface cues, task framing) that influence reliance but often in simplified or artificial tasks.
A central critique: experiments frequently lack application-grounded realism, limiting external validity for high-stakes, real-world settings.
User engagement (attention, cognitive effort, time) is crucial for appropriate reliance, but obtaining it competes with other constraints (time pressure, mental workload, incentives).
“Appropriate reliance” requires balancing trust and skepticism so users accept correct advice and reject incorrect advice; multiple interacting factors determine whether that balance is achieved.
The review identifies gaps in (a) realistic task design, (b) measurement of engagement costs, and (c) the interplay among interaction factors across contexts.
The authors offer recommendations for intervention design to build appropriate reliance (e.g., context-rich tasks, interventions that allocate engagement efficiently).

Data & Methods

Method: analytical review of recent empirical studies in the human-AI reliance literature.
Scope: examined decision-making tasks used in experiments and evaluated their validity relative to application-grounded (realistic, consequential) contexts.
Analysis focused on (a) cataloguing interaction factors studied, (b) assessing how tasks map to real-world decision contexts, and (c) identifying how engagement and costs were handled or neglected across studies.

Implications for AI Economics

Attention and engagement are economic inputs: designing AI systems that demand more user engagement has opportunity costs (time, productivity) that affect user adoption and welfare.
Misaligned reliance leads to economic externalities: over-reliance can cause systematic losses (errors accepted from AI), while under-reliance can reduce the value capture from effective AI advice.
Product design and pricing: providers must trade off interface/information investments (explainability, feedback) against user time costs; optimal design depends on task stakes and user incentives.
Market dynamics and regulation: sectors with high-stakes decisions require stronger standards for evaluative realism, transparency, and accountability to mitigate costly over-reliance.
Measurement and evaluation: economic assessments of AI interventions should include engagement costs, error externalities, and context-specific validity — not just algorithmic accuracy.
Policy levers: subsidies, liability rules, certification, or disclosure requirements can shift incentives toward designs that promote appropriate reliance without imposing undue engagement burdens.
Research-practice alignment: economists studying AI impacts should prioritize experiments and field studies that mimic application-grounded contexts so estimates of benefits, costs, and adoption are externally valid.

Actionable recommendations (for researchers, designers, policymakers) - Use application-grounded tasks or field experiments when estimating user reliance and value from AI advice. - Explicitly measure engagement costs (time, effort) and incorporate them into welfare and cost–benefit analyses. - Design interventions that allocate scarce user engagement where it yields the largest reduction in costly errors (e.g., triage, selective explanations). - Evaluate market and regulatory approaches that internalize externalities from inappropriate reliance (liability, standards, disclosure). - Report context features (stakes, task complexity, user expertise) so economic models can generalize appropriately across applications.

Assessment

Paper Typereview_meta Evidence Strengthn/a — This is an analytical review synthesizing existing HCI experiments rather than generating new causal estimates; it summarizes heterogeneous empirical findings rather than providing primary causal identification. Methods Rigormedium — The paper conducts a targeted analytical review of recent human–AI reliance studies and assesses task validity and interaction factors, but it does not report systematic search procedures, preregistered protocols, or quantitative meta-analytic methods, leaving room for selection and synthesis bias. SampleAn analytical synthesis of recent HCI literature on human reliance on AI advice, drawing on experimental studies (laboratory and online decision-making tasks) that examine user engagement, over-reliance, and interventions; exact inclusion criteria, number of studies, and search strategy are not reported in the abstract. Themeshuman_ai_collab adoption GeneralizabilityMost evidence comes from lab or online experimental tasks that may not mirror high-stakes, real-world decision contexts, Participant pools in HCI experiments (students, crowdworkers) limit population representativeness, Study tasks appear concentrated in particular domains (e.g., simple classification or recommendation) and may not generalize across industries or complex managerial decisions, Lack of longitudinal or field-based studies reduces confidence about long-term behavioral change, adoption, and productivity impacts, Potential publication and selection bias in the reviewed literature could skew the synthesized conclusions

Claims (8)

Claim	Direction	Outcome	Confidence & Evidence	Details
AI systems are increasingly being positioned to assist people in decision-making. Other	positive	use of AI systems to assist decision-making (trend)	Reading fidelity high Study strength low	not reported 0.12
Recent empirical studies show critical concerns that people over-rely on AI advice without analytically engaging with it. Task Allocation	negative	people's reliance on AI advice and level of analytical engagement	Reading fidelity high Study strength medium	not reported 0.24
HCI research explores how people rely on AI advice, but it largely overlooks replicating realistic decision-making scenarios. Other	negative	ecological validity / realism of decision-making tasks used in HCI studies	Reading fidelity high Study strength medium	not reported 0.24
Human-AI interaction factors influence people’s reliance on AI advice. Task Allocation	mixed	degree of reliance on AI advice as a function of human-AI interaction factors	Reading fidelity high Study strength medium	not reported 0.24
We conducted an analytical review of recent studies in human-AI reliance literature. Other	positive	n/a (method description)	Reading fidelity high Study strength high	not reported 0.4
We analyzed the decision-making tasks in research and their validity in application-grounded contexts. Other	positive	validity of decision-making tasks used in prior studies	Reading fidelity high Study strength high	not reported 0.4
Our findings show that user engagement is a precious commodity for relying on AI advice; however, it comes at a cost. Task Allocation	mixed	user engagement and associated costs related to relying on AI advice	Reading fidelity medium Study strength medium	not reported 0.14
We discuss factors contributing to 'appropriate reliance', existing research gaps, and recommendations for intervention design for human-AI reliance. Governance And Regulation	positive	identification of factors, gaps, and recommended interventions to achieve appropriate reliance	Reading fidelity high Study strength high	not reported 0.4