Autonomous LLM agents are lowering the bar to offensive cyber operations by making actions, impacts and users indeterminate, tipping short-term advantage to attackers and exposing gaps in current dual‑use and AI governance.

Main Finding

LLM-driven autonomous agents transform offensive security by producing a joint shift in three independent forms of indeterminacy — non-deterministic model behavior, open-ended impact, and a much-lowered operating skill floor — that together enable the industrialization and democratization of offensive capability. Short-term economic effects favor attackers (costs of attack fall, asymmetry with defense widens), existing dual-use and cybersecurity ethics frameworks are poorly equipped to allocate responsibility, and governance must move from actor-level rules to supply-chain, access-control, and institution-level interventions.

Key Points

• Three orthogonal indeterminacy dimensions that distinguish agentic offensive tools from classical tooling:
  • Model indeterminacy: LLMs are probabilistic, opaque, and offer poor ex-ante/ex-post explainability; reasoning traces can be hallucinated; tool-makers lose partial control via LLM supply chains.
  • Open-ended impact: non-deterministic, multi-step behavior plus model updates and opaque training chains make impact unpredictable and potentially escalating.
  • User-population indeterminacy: the skill floor to deploy offensive capability drops sharply, enlarging the potential attacker base and blurring moral responsibility.
• Historical context & timeline:
  • Stage 1 (2023): early interactive LLM uses for OSINT and guidance.
  • Stage 2 (2024–25): tool-calling, function-calling, and reasoning techniques produced many autonomous agents; capability jump.
  • Stage 3 (2025–26): frontier models (Gemini-3, Claude-4.6-Opus, GPT-5.5, etc.) greatly increased efficacy; vendor claims report large multipliers in success rates.
• Ethical analysis approach:
  • Uses Formosa et al.’s cybersecurity ethics principles (beneficence, non‑maleficence, autonomy, justice, explicability) and applies multiple moral frameworks (deontology, consequentialism, virtue ethics, Value Sensitive Design, moral agency theory).
  • Finds that moral attribution becomes diffuse across users, scaffolds, model-makers, and downstream deployers.
• Traditional mitigations are insufficient alone:
  • Fail-safe defaults, HITL, and VSD help but cannot fully counteract the combined indeterminacy and model-supply-chain effects.
• Practical caveats:
  • Many capability claims come from vendor benchmarks and preprints that are not independently reproducible; empirical assessment is still emergent.

Data & Methods

• Method: conceptual and normative analysis combining literature review, framework mapping, and stakeholder analysis.
  • Built on Formosa, Wilson, and Richards’ cybersecurity ethics framework to structure principles.
  • Employed multi-framework ethical reasoning (deontology, consequentialism, virtue ethics, VSD, moral agency theory).
• Evidence sources:
  • Public provider abuse reports (OpenAI, Anthropic, Google).
  • Vendor and project disclosures (e.g., cochise, incalmo, XBOW) and public model release announcements.
  • Historical policy texts (Wassenaar Arrangement, EU Dual-Use Regulation) and prior dual-use scholarship (DURC analogies).
• Methods characteristics and limits:
  • Predominantly qualitative, normative, and descriptive rather than experimental.
  • Uses vendor-published capability claims as landscape indicators; acknowledges non-reproducibility and rapid technological change as limits to firm empirical claims.

Implications for AI Economics

• Attack–defense cost asymmetry and market externalities:
  • Lowered cost of mounting cyberattacks (automation + lower skill requirement) increases negative externalities, while defensive effort remains costly, specialized, and harder to scale — aggravating a classic market failure.
  • Socially optimal investment in defensive public goods (shared detection, patching, hardened infrastructure) will likely be under-provided by private markets.
• Industrialization and commoditization:
  • Offensive capabilities can be packaged and scaled like a product/service (agents + APIs + scaffolds), creating new markets (commercial red‑team-as-a-service, black‑market agents).
  • Commoditization may displace some junior penetration-testing tasks, shifting labor demand toward higher-skilled oversight, model-auditing, governance, and blue-team automation engineering.
• Concentration and supply-chain risk:
  • Model-making is concentrated among a few frontier providers; this creates systemic risk and rents for model owners, while also centralizing liability and governance leverage.
  • Closed-weight / API models let providers impose access controls, but also create single points of failure and private gatekeeping of capabilities.
• Incentives, liability, and insurance:
  • Diffuse moral responsibility and opaque causality complicate liability allocation and insurance pricing for cyber incidents; insurers may raise premiums or exclude agentic tooling exposures.
  • Liability rules (strict liability for certain deployments, provenance/auditability requirements) can reshape incentives for releasing or restricting models and scaffolds.
• R&D and innovation trade-offs:
  • Tighter controls (export rules, publication gating, restricted model weights) reduce some misuse risk but can impede legitimate R&D and defensive innovation; policies must weigh these trade-offs.
  • Tiered access (safe-for-work vs. restricted frontier), IRB-like review for high-risk releases, and mandated safety-by-design can better align incentives.
• Labor market and skill composition:
  • Short-run demand for routine pen-testing labor may fall; medium/long-run demand for specialists in AI oversight, security model engineering, provenance/audit, and compliance will rise.
  • Training and certification programs should pivot toward governance, auditing, and hybrid human-AI oversight skills.
• Policy & market recommendations (implied by paper):
  • Adopt supply-chain governance: model provenance, versioning, and audit trails; mandatory red-team audits for high-capacity models.
  • Implement stratified access and certification for offensive tool deployment (HITL defaults, authorization gating).
  • Create institutional oversight for high-risk releases (DURC-style review boards for cyber/AI).
  • Public investment in defensive public goods: funded shared detection, patched datasets, publicly audited defensive models to counterbalance private model concentration.
  • Clarify liability regimes and update cyber insurance to reflect agentic-tools risks.
  • Encourage value-sensitive design and fail-safe architectural constraints in scaffolds and agent deployments.
• Long-run outlook:
  • While short-term market dynamics favor attackers, the same technologies could democratize defensive tooling and lower defender costs if governance, public investment, and institution-building succeed — but this outcome is neither guaranteed nor immediate.