A single compromised LLM can hijack group decisions in multi-agent systems via a persuasion cascade; dynamically reducing trust in suspected adversaries (while tuning agent stubbornness and scale) curbs influence at the cost of some coordination or compute overhead.

Main Finding

A single malicious or compromised LLM agent with high stubbornness and persuasive power can trigger a persuasion cascade that steers the collective opinion of a multi-agent LLM system (MAS). Modeling LLM-MAS with the Friedkin–Johnsen opinion-formation framework both theoretically and empirically explains this vulnerability. Practical defenses include increasing benign agent count, increasing agent stubbornness, or reducing trust in adversaries; the authors propose a trust-adaptive mechanism that dynamically adjusts inter-agent trust to limit adversarial influence while preserving cooperative performance, and show it is effective in experiments.

Key Points

• Threat: The interactive communication channels in LLM-based MASs create a new attack surface where agents can propagate misinformation and manipulate group outcomes.
• Modeling: The Friedkin–Johnsen opinion-dynamics model (innate opinions + interpersonal influence weights + stubbornness) closely captures LLM-MAS behavior across settings.
• Vulnerability: A single highly stubborn and persuasive agent can dominate network dynamics, producing a persuasion cascade that reshapes collective opinion.
• Security levers:
  • Increase the number of benign agents (dilutes adversarial influence).
  • Increase agents' innate stubbornness or peer-resistance (reduces susceptibility).
  • Reduce trust in suspected adversaries (limits their weight on others’ updates).
• Tradeoffs:
  • Adding agents is computationally costly.
  • High stubbornness makes the system more robust to manipulation but impairs the network’s ability to reach consensus or coordinate.
  • Naïvely lowering trust may hinder cooperation/performance.
• Proposed defense: A trust-adaptive strategy that dynamically adjusts trust weights to limit the influence of adversaries while maintaining cooperative function.
• Empirical support: Extensive experiments across different network topologies and attack/defense scenarios validate the model and the effectiveness of the trust-adaptive defense.

Data & Methods

• Theoretical framework:
  • Adopt the Friedkin–Johnsen model: each agent has an innate opinion, updates its expressed opinion as a weighted combination of its innate stance and neighbors’ opinions; stubbornness is the weight on the innate opinion; interpersonal trust forms a (time-varying) influence matrix.
  • Analyze fixed points and influence propagation to identify conditions under which a single adversary can dominate.
• Empirical evaluation:
  • Simulation experiments of LLM-based MASs mapped to the Friedkin–Johnsen dynamics.
  • Varied network topologies (e.g., dense vs. sparse, different trust matrices), attacker profiles (stubbornness, persuasiveness), and defensive strategies.
  • Metrics: collective opinion trajectories, final consensus/opinion, extent of adversarial sway, and cooperative task performance under defenses.
• Defense implementation:
  • Trust-adaptive mechanism that monitors influence patterns and dynamically reduces trust weights assigned to agents suspected of adversarial behavior while adjusting others to preserve coordination.
  • Evaluated tradeoffs between security (reduced adversarial influence) and utility (task performance, convergence).

Implications for AI Economics

• Security as an economic externality: MAS security failures impose social costs (misinformation, poor collective decisions). Providers and deployers face incentives to invest in defenses; underinvestment risks negative externalities.
• Cost–benefit tradeoffs for robustness:
  • Scaling by adding benign agents improves resistance but has clear computational and operational costs; economic decisions must weigh marginal security benefits vs. compute expense.
  • Increasing agent stubbornness reduces manipulation risk but impairs consensus and collaborative efficiency — a tradeoff between robustness and joint performance that affects product design and pricing.
  • Trust-lowering measures can reduce adversarial impact but may degrade cooperation and value delivered; adaptive trust schemes can better align incentives by preserving utility while reducing attack surface.
• Product and market design:
  • Market differentiation for “robust” MAS offerings: buyers may pay premiums for systems with built-in trust-adaptive defenses or provable resistance guarantees.
  • Contracting and liability: principals may demand technical specifications (e.g., adaptive-trust mechanisms) in service-level agreements to mitigate systemic manipulation risk.
• Regulation and standards:
  • Regulatory frameworks could require minimum defenses or auditing of inter-agent influence dynamics for high-stakes MAS deployments (finance, critical infrastructure, information dissemination).
  • Standardized metrics (e.g., susceptibility to persuasion cascades) would help buyers compare security-performance tradeoffs.
• Incentive alignment and governance:
  • Robustness investments may be underprovided absent clear liability or reputational pressures; insurance markets and certification bodies can create incentives for better defenses.
  • Governance of agent trust assignment (who controls trust weights, how adjustments are verified) becomes an economic governance question—design choices affect competition and coordination costs.
• Research and investment priorities:
  • Funding and R&D should target efficient defenses (like trust-adaptive mechanisms) that obtain security gains with minimal performance loss and compute overhead.
  • Empirical economic studies are needed to quantify deployment costs, pricing of robust MAS, and social welfare impacts from persuasion cascades in deployed systems.

Limitations and next steps (brief): the paper’s validation appears simulation-based; real-world deployments may introduce more complex behaviors (strategic adversaries, noisy signals, heterogeneous task payoffs). For economic modeling, mapping technical parameters (stubbornness, trust matrices) to monetary costs and user utility is a useful next step.