A cryptographic audit layer plus replay-certified counterfactuals lets auditors assign graded, tamper-evident blame in multi-agent AI systems; in benchmarks Audita recovers true responsibility where single-culprit judges fail and resists record forgery.

Main Finding

Audita is an audit layer for autonomous multi-agent systems that (1) records every inter-agent command in a tamper-evident, cryptographically verifiable log and (2) computes a certified, graded causal-attribution verdict over only that verified evidence. Under standard technical assumptions it provably prevents framing (a compliant agent cannot be made to look guilty), detects attempted blame-shifting coalitions, and certifies exactly what an evidence-limited auditor can and cannot establish. Empirically, Audita sharply reduces attribution error on live LLM-based pipelines (≈threefold reduction vs. an LLM judge) and recovers planted responsibility perfectly on a benchmark of physical-incident structures where prior methods systematically fail.

Key Points

• Two-layer design:
  • Evidence layer: every inter-agent message m = (author, payload, citations, committed effect, signature) is signed, cites predecessors, and is sealed into an append-only Merkle-committed record so authorship, ordering, and completeness are cryptographically checkable.
  • Causal layer: a production-gated, replay-certified causal engine computes graded responsibility per principal using a modified Halpern–Pearl causal semantics and returns a typed verdict (duty bit, breach label under spec and reasonable-agent standards, a responsibility interval with receipts, and conduit/exoneration flags).
• Graded causation: responsibility score ρp computed from minimal cause and witness sets (ρ = 1/(|X| + |W|)), with every claimed cause verified by counterfactual replay (intervention + re-execution).
• Verdict schema expresses distinctions single-culprit labels cannot: joint overdetermination, preemption, omission (including omission-only culpability), negligent-but-inert actors, and conduit actors (cause without breach).
• Proven guarantees (under three assumptions: replayable stack, recorded channel, sound key custody):
  • Theorem 1: A compliant, rule-following principal cannot be fabricated as culpable without its own certified breach.
  • Theorem 2: Attempts by a coalition to shift blame are themselves detectable and graded.
  • Theorem 3: Deleting/destroying evidence cannot silently exonerate a certified culprit.
  • Theorem 4: A formal completeness bound on what any auditor confined to the record can certify.
  • Theorem 5: Guarantees persist across nested/meta-orderings of commands.
• Robustness: tamper-evident record prevents forgery attacks from changing certified verdicts; certified verdicts stay invariant under record forgery experiments.

Data & Methods

• Message and record design:
  • Messages carry explicit citations and signed receipts; epochs are Merkle-rooted and chain to prior roots (append-only public commitment).
• Causal semantics and algorithm:
  • Backward slicing from declared adverse predicate Y=1 to gather candidate messages.
  • Production gate filters to messages with intact certified paths to the outcome.
  • Two-layer engine proposes minimal cause sets (X) and witness sets (W), certifies by counterfactual replay, and outputs per-principal graded responsibility intervals plus duty/breach findings and verification receipts.
• Theoretical model assumptions:
  • Replayable stack (deterministic re-execution or controlled nondeterminism),
  • Recorded channel (all inter-agent commands go through the recorded substrate),
  • Sound key custody (key material is not exfiltrated; insiders may be malicious).
• Evaluations:
• Live LLM multi-agent pipelines: a 4-role pipeline (planner, parallel solvers, aggregator with declared duty, verifier) solving GSM8K problems; three independent corpora (480 cases each) with calibrated faults and replay-verified labels. - Primary metric: responsibility error = L1 distance between reported and ground-truth graded responsibility profiles. - Results (mean over n=145 adverse incidents): LLM judge baseline error 0.589 ± 0.017; Audita causal-only 0.408 ± 0.020; Audita full (gold-informed) 0.170 ± 0.035; Audita deployable (no answer key) 0.159 ± 0.030. - Finding: almost all adverse incidents were joint solver–aggregator events (95–98%), showing single-culprit formats misrepresent common failure modes.
• Benchmark of physical incident structures: >1,000 incidents grounded in 42 public OSHA/NIOSH robot-accident reports; analytically derived graded ground truth (redundancy, preemption, omission, inert-twin, chains). - Audita (certified, graded) recovered planted responsibility exactly (responsibility error 0.000 across structures; exact-match rate 1.000). - Baselines (single-site counterfactual scoring, Shapley-value attribution, LLM judges, anomaly detectors) systematically failed on specific structures (e.g., redundancy, omission, inert twin). - Negative controls: on no-cause incidents Audita reports no culprit (0/300 manufactured), whereas judges and anomaly detectors often manufacture culprits.
• Robustness / scaling:
  • Audita’s advantage persists as underlying LLMs scale; certified verdicts remain exact under record forgery attacks that break unsigned substrates.

Implications for AI Economics

• Liability allocation and litigation:
  • Audita makes blame an evidence-based, graded calculation rather than contested log claims, reducing uncertainty in assigning economic responsibility among vendors, operators, algorithm providers, and insurers. This should lower dispute resolution costs and reduce litigation over “who did it.”
• Insurance pricing and moral hazard:
  • Graded, verifiable responsibility signals enable insurers to price coverage more accurately (risk-based premiums tied to observable duty/breach patterns) and to design contracts with clear auditability requirements, mitigating moral hazard across vendors/operators.
• Market for safety and certified audit services:
  • Demand will increase for certified-record infrastructure, replayable stacks, and third-party auditors; vendors offering these will capture economic value and could become gatekeepers for high-risk deployments.
• Regulatory compliance and compliance costs:
  • Audita provides a practical compliance mechanism for obligations like the EU AI Act’s auditability requirement. Regulators can mandate certified-recording and replay-certified causal analysis as a compliance standard, shifting compliance costs into up-front engineering and audit-certification markets.
• Investment and competition:
  • Operators and platform providers that adopt certified audits will enjoy lower liability risk and potentially lower capital costs; conversely, lack of certified audits may lead to higher insurance and operational costs and regulatory barriers to entry.
• Incentives for protocol design and standards:
  • Economic analysis should incorporate the externalities of unverifiable logs (strategic destruction or manipulation of evidence) — Audita reduces those rents. Standard-setting bodies should evaluate protocols that implement Audita-like properties (signed messages, citation chains, Merkle appends, replayability).
• Limits & economic trade-offs:
  • Audita’s guarantees depend on assumptions (e.g., sound key custody, recorded-channel coverage, replayability). Achieving these imposes costs (engineering, latency, storage, cryptographic key management) and may require governance changes (who controls keys/roots). Economists should model adoption thresholds where benefits (lower liability, insurance savings, regulatory access) exceed these costs.
• Policy recommendations for economists/advisors:
  • Encourage regulators and insurers to require verifiable audit layers for high-risk autonomous collectives.
  • Incorporate graded responsibility outcomes (not just binary fault) into compensation and liability regimes to better reflect overdetermination and omission risks.
  • Support public benchmarks and standards (the paper’s >1,000-incident benchmark) to evaluate auditing technologies and to reduce information asymmetries in the market for safe autonomy.

Short caveat: Audita shifts blame from contested logs to calculation over certified evidence, but it cannot certify beyond what is recorded (formal completeness bound). Practical deployment requires engineering to meet the replay/record/key assumptions and governance to manage keys and public commitments.