Explainable AI helps teachers perform better in the moment under concurrent assistance but erodes longer-term trust; an RCT of 120 pre-service teachers finds explanations boost immediate execution only in concurrent setups and do not transfer to independent tasks, while paradoxically suppressing accumulated trust.

How AI-Assisted Decision-Making Paradigms and Explainability Shape Human-AI Collaboration

Yingying Wang, Qin Ni, Tingjiang Wei, Haoxin Xu, Lu Liu, Liang He · Fetched April 11, 2026 · Sustainability

semantic_scholar rct medium evidence 7/10 relevance DOI Source PDF

In a randomized experiment with 120 pre-service teachers, explanatory interfaces improved immediate task performance under concurrent AI assistance but not under sequential assistance or on later independent tasks, while explanations reduced learned trust and slowed the development of cognitive and emotional trust; concurrent assistance, despite lower immediate performance than sequential, better supported emotional trust.

The increasing integration of artificial intelligence (AI) in educational decision-making raises a critical question: how to design AI systems that can effectively support teachers while maintaining an appropriate level of trust. Addressing this question requires not only continuous improvements in the technical capabilities of AI systems but also an examination from a human-AI interaction perspective of how different system designs influence users’ cognitive performance and affective responses, thereby providing guidance for system optimization and design. Therefore, this study conducted a randomized controlled experiment with 120 pre-service teachers to investigate how AI-assisted decision-making paradigms and AI explainability jointly influence teachers’ task performance and trust in AI, and whether these effects transfer to subsequent independent tasks. The results indicate that the effect of explanatory interface on task performance is context dependent and yields an immediate positive impact. Under the concurrent paradigm, the explanatory interface of the AI system significantly improves immediate task performance, whereas no significant effect is observed under the sequential paradigm. Moreover, this improvement is confined to the task execution stage and does not transfer to subsequent independent tasks. In contrast, the effect of explanatory interface on trust exhibits a delayed and negative pattern. The explanatory interface has no significant impact on situational trust, while it exerts a negative effect on learned trust and suppresses the natural development of both cognitive trust and emotional trust. In addition, different AI-assisted decision-making paradigms exhibit distinct patterns of influence on task performance and trust. Although the concurrent paradigm performs worse than the sequential paradigm in terms of immediate task performance, it is more effective in promoting users’ emotional trust. Overall, these findings extend the theoretical understanding of the mechanisms of explainability in human-AI interaction and provide empirical evidence for the joint design of explainable AI systems and human-AI collaboration paradigms.

Summary

Main Finding

Explanatory interfaces (explainability) and the AI-assisted decision-making paradigm (concurrent vs. sequential) interact in complex, time-dependent ways: explainability produces an immediate, context-dependent boost to teachers’ task performance only under the concurrent paradigm, but this performance benefit does not transfer to subsequent independent tasks. In contrast, explainability yields delayed negative effects on various forms of learned trust (it does not change situational trust but suppresses the natural development of cognitive and emotional trust). Concurrent assistance yields lower immediate task performance than sequential assistance but better supports users’ emotional trust.

Key Points

Experiment: randomized controlled study with 120 pre-service teachers manipulating (1) decision-making paradigm — concurrent (AI and user act together) vs. sequential (user acts after AI), and (2) AI explainability — presence vs. absence of an explanatory interface.
Performance effects:
- Explanatory interface improves immediate task performance only under the concurrent paradigm.
- No significant explanatory-interface effect on immediate performance under the sequential paradigm.
- The performance gain with explainability is limited to the task-execution stage and does not transfer to later independent tasks.
Trust effects:
- Explanatory interface has no detectable effect on situational trust (immediate, task-specific).
- Explanatory interface exerts a negative effect on learned trust and suppresses the natural development of both cognitive (competence-based) and emotional (affect-based) trust over time.
- The negative trust effects are delayed rather than immediate.
Paradigm differences:
- Sequential paradigm > concurrent paradigm for immediate task performance.
- Concurrent paradigm > sequential paradigm for fostering emotional trust.
Overall implication: explainability and collaboration paradigm jointly affect utility and adoption-relevant attitudes; gains in short-term task performance do not guarantee positive trust outcomes or skill transfer.

Data & Methods

Design: randomized controlled experiment (2 × 2) with 120 pre-service (teacher) participants.
Manipulations:
- AI-assisted decision-making paradigm: concurrent vs. sequential.
- Explainability: AI system with an explanatory interface vs. without.
Outcomes measured:
- Task performance (immediate execution and subsequent independent tasks to assess transfer).
- Multiple trust constructs (situational trust, learned trust, cognitive trust, emotional trust) measured across time to detect immediate and delayed effects.
Analysis: tested main effects and interactions of paradigm × explainability on performance and trust; examined temporal dynamics (immediate vs. later tasks/trust measures).
Note: summary reports the study’s reported patterns of significance and temporal effects; precise operationalizations (task content, trust instruments) follow standard human-AI interaction measures used in the paper.

Implications for AI Economics

Short-term productivity vs. long-term adoption trade-off:
- Explainability can raise immediate productivity in some collaborative modes (concurrent), but it may reduce learned trust over time, potentially lowering longer-run adoption and reliance on the AI—impacting the realized productivity gains at scale.
Design choices alter complementarities between labor and AI:
- Sequential paradigms improve immediate independent performance (skill retention/transfer), suggesting such designs better foster human capital development and worker autonomy; concurrent paradigms may create stronger emotional ties to AI but weaker skill transfer.
Policy and regulation:
- Mandating explainability without attention to interaction paradigm may produce unintended effects (short-run performance gains but weaker long-term trust), implying regulators should consider behavioral outcomes, not just transparency.
Deployment and incentive design:
- Firms and school systems should choose AI interaction paradigms aligned with their goals: maximize short-run task accuracy (concurrent + explainability), strengthen teacher autonomy and skill transfer (sequential, possibly with minimal or differently structured explainability), or prioritize affective acceptance (concurrent).
- Economic evaluations (cost-benefit, adoption models) should incorporate dynamic trust trajectories and transfer effects when estimating returns to implementing explainable systems.
Research/evaluation recommended:
- Quantify how trust dynamics affect adoption rates, usage intensity, and productivity over time in field settings.
- Examine pricing, procurement, and training investments in light of trade-offs between immediate gains and long-term human capital outcomes.

Assessment

Paper Typerct Evidence Strengthmedium — Random assignment gives strong internal validity for the measured immediate effects, but external validity is limited by a small, homogeneous sample (pre-service teachers), lab-style tasks, short follow-up, and reliance on probably self-reported trust measures, so causal claims are credible for the experimental context but not broadly generalizable. Methods Rigormedium — The RCT design and factorial manipulation are rigorous, but likely limitations include modest sample size (N=120), potential demand characteristics, limited detail on manipulation checks and blinding, short-term measurement windows (no long-term follow-up), and trust outcomes that appear to rely on subjective measures rather than behavioral adoption metrics. Sample120 pre-service teachers participated in a randomized laboratory-style experiment; they performed AI-assisted educational decision-making tasks under a 2x2 design (concurrent vs sequential assistance × explanatory vs non-explanatory interface), with immediate task performance measured during assistance and transfer measured on subsequent independent tasks; trust was assessed via situational, learned, cognitive, and emotional trust measures. Themeshuman_ai_collab skills_training adoption IdentificationRandomized controlled 2x2 experiment: participants (N=120 pre-service teachers) were randomly assigned to AI-assisted decision-making paradigm (concurrent vs sequential) and to explanatory-interface presence (explainable vs non-explainable), with causal effects identified through randomized assignment and comparison of immediate and transfer outcomes across conditions. GeneralizabilitySample limited to pre-service (not experienced) teachers, Laboratory/experimental tasks may not reflect real classroom complexity, Short-term measurements; no long-term follow-up on sustained performance or adoption, Specific AI system and explanation design used may not generalize across interfaces or domains, Cultural/geographic context and sampling frame not specified (limits external validity), Relatively small sample size limits detection of heterogeneous effects

Claims (8)

Claim	Direction	Confidence	Outcome	Details
The study was a randomized controlled experiment with 120 pre-service teachers. Other	null_result	high	study_design/sample	n=120 1.0
Under the concurrent AI-assisted decision-making paradigm, the explanatory interface of the AI system significantly improves immediate task performance. Decision Quality	positive	high	immediate task performance (task execution stage)	n=120 0.6
Under the sequential AI-assisted decision-making paradigm, the explanatory interface has no significant effect on immediate task performance. Decision Quality	null_result	high	immediate task performance (task execution stage)	n=120 0.6
The improvement in task performance due to the explanatory interface is confined to the task execution stage and does not transfer to subsequent independent tasks. Decision Quality	negative	high	performance transfer to subsequent independent tasks	n=120 0.6
The explanatory interface has no significant impact on situational trust. Worker Satisfaction	null_result	high	situational trust	n=120 0.6
The explanatory interface exerts a negative effect on learned trust. Worker Satisfaction	negative	high	learned trust	n=120 0.6
The explanatory interface suppresses the natural development of both cognitive trust and emotional trust. Worker Satisfaction	negative	high	cognitive trust and emotional trust development	n=120 0.6
Although the concurrent paradigm performs worse than the sequential paradigm in terms of immediate task performance, it is more effective in promoting users' emotional trust. Decision Quality	mixed	high	immediate task performance (negative) and emotional trust (positive)	n=120 0.6