Marrying formal argumentation with large language models could make AI decisions inspectable and contestable, creating demand for verifiable AI services in regulated sectors; at the same time it will shift experts toward adjudication and oversight and create new governance needs.

Main Finding

The paper argues that integrating computational argumentation with large language models (LLMs) creates a new paradigm—Argumentative Human-AI Decision‑Making—where AI agents do more than justify outputs: they participate in dialectical, contestable, and revisable decision processes with humans. This synergy promises human-aware, verifiable, and trustable AI for high‑stakes domains by combining formal argument structures with LLMs’ ability to mine and generate rich, contextual arguments from unstructured text.

Key Points

• Motivation
  • Computational argumentation: offers formal, verifiable reasoning representations (argumentation frameworks, attack/support relations), but has required heavy feature engineering and domain-specific knowledge.
  • LLMs: excel at extracting and generating arguments from unstructured text but are opaque and hard to evaluate or trust.
• Proposed convergence
  • Argumentation Framework Mining: use LLMs and NLP pipelines to extract claims, premises, relations (attack/support), and provenance from text corpora.
  • Argumentation Framework Synthesis: combine mined fragments into coherent formal argumentation frameworks (AFs) with explicit semantics, enabling verification and automated inference.
  • Argumentative Reasoning & Interaction: run formal dialectical/acceptability semantics and dialogue protocols (contest, rebuttal, revision) enabling agents that reason with humans through structured debates and revisions.
• Value-add
  • Transparency and verifiability: structured AFs make chains of inference inspectable and machine-checkable.
  • Contestability and revision: decisions are not final outputs but subject to dialectical challenge and update, increasing robustness and trust.
  • Human‑AI collaboration: supports collaborative reasoning (“with” humans) rather than opaque automation “for” humans, improving uptake in high‑stakes settings.
• Challenges
  • Faithful extraction: aligning LLM-extracted arguments with formal AF primitives and ensuring fidelity to source evidence.
  • Robustness & adversarial manipulation: AFs and LLMs may be gamed or misled; incentives may drive strategic argumentation.
  • Evaluation: developing metrics and benchmarks for argument quality, fidelity, contestability, and human trust.
  • Integration costs: domain modeling, human-in-the-loop protocols, and regulatory/ liability frameworks required.

Data & Methods

(Conceptual / design-oriented; empirical work would instantiate these components) - Data sources - Domain corpora with argumentative content: legal opinions, clinical notes and guidelines, policy reports, regulatory filings, parliamentary debates, curated debate corpora (e.g., UKP, Persuasive Essays, IBM Debater-type collections). - Expert annotations: labelled claims, premises, stances, attack/support relations and provenance for supervised or weakly supervised training. - Extraction & representation methods - Fine-tuning or prompting LLMs for argument mining (claim detection, premise identification, relation classification). - Information‑extraction pipelines to attach provenance, uncertainty, and source metadata. - Structured representations: convert extracted elements into formal AF primitives (nodes = arguments/claims; edges = attack/support; weights or probabilities for strength). - Synthesis & verification - Algorithms for merging fragmentary arguments into coherent AFs (graph synthesis, resolution of conflicts, canonicalization). - Formal semantics: Dung-style acceptability, bipolar AFs, weighted/probabilistic AFs; model checking to verify logical consistency and provenance constraints. - Hybrid systems: symbolic reasoning layers over LLM-derived content for constraints, counterfactual checks, and rule compliance. - Interaction & evaluation protocols - Dialectical dialogue protocols: structured challenge/response cycles, revision operators, and adjudication mechanisms (human adjudicators or automated semantics). - Evaluation metrics: task performance, fidelity (how accurately AF reflects source), argument quality (coherence, relevance, completeness), contestability (ability to find justified counterarguments), revision responsiveness, human trust and reliance, calibration of uncertainty. - Experimental designs: benchmark tasks, human-subject studies in domain-specific simulations, adversarial stress tests, longitudinal deployment case studies.

Implications for AI Economics

• Adoption & market structure
  • Demand shift toward AI systems that provide verifiable, contestable reasoning in regulated/high‑stakes sectors (healthcare, law, finance, public policy).
  • Competitive advantage for firms offering argumentatively transparent AI—premium pricing for verifiability and auditability.
  • Emergence of new service layers: argumentation-as-a-service, audit firms, explanation certification, and human-in-the-loop orchestration platforms.
• Labor, productivity & complementarities
  • Human experts shift from sole decision-makers to adjudicators, challengers, and validators of AI-generated arguments—changing skill demands toward critical evaluation and dialectical oversight.
  • Potential productivity gains from faster evidence synthesis and argument exploration, but complementary tasks (validation, stakeholder engagement) create new labor demand.
• Information asymmetry & transaction costs
  • Structured AFs reduce information asymmetry by making reasoning traceable, lowering search and verification costs in transactions and contracting.
  • Better contestability can reduce costly litigations and regulatory frictions if decisions are transparently defensible.
• Incentives, strategic behavior & regulation
  • New incentives for strategic argument construction (gaming, persuasion without fidelity) suggest need for governance: standards for provenance, certification, and liability rules.
  • Regulators may prefer systems that support contestability and audit trails—potentially mandating argumentation-style explainability in certain sectors.
• Welfare and risk
  • Welfare gains from improved decision quality and trust in automation, particularly where human oversight is required.
  • Risk of manipulation and misinformation if argument mining/synthesis is unregulated or misaligned with social incentives; externalities may justify public intervention (standards, audits, liability frameworks).
• Research & policy agenda for economists
  • Quantify value of contestable explanations: willingness-to-pay for verifiable reasoning vs. opaque predictive performance.
  • Study labor market impacts: reallocation of tasks, wage effects for validators/adjudicators.
  • Design mechanisms and contracts that align incentives for truthful argument provision and penalize strategic misrepresentation.
  • Evaluate regulatory interventions: certification regimes, liability assignment, mandatory audit trails in high-stakes domains.

Overall, integrating computational argumentation with LLM capabilities creates economically significant opportunities (trustworthy, auditable AI services) and risks (strategic manipulation, new regulatory needs). For AI economics, the key questions are how these systems affect incentives, market structure, labor complementarity, and the design of policies that extract social value while containing potential harms.