Cryptographic mandates for LLM-driven payments can be bypassed by simple prompt injections; two novel attacks reliably alter purchase choices and leak sensitive data in an AP2 shopping agent built on Gemini-2.5-Flash and Google ADK.

Main Finding

AP2 (Agent Payments Protocol), despite using cryptographically verifiable mandates to authorize agent-led purchases, is practically vulnerable to prompt-injection attacks that can (a) bias purchase decisions (product ranking) and (b) exfiltrate sensitive user secrets. The paper demonstrates two practical attacks — the Branded Whisper Attack and the Vault Whisper Attack — and validates them on a functional AP2 shopping agent built with Gemini‑2.5‑Flash and the Google ADK, showing that simple adversarial prompts reliably subvert agent behavior. The result is that current agentic payment architectures are not robust enough for safe deployment without stronger isolation and defensive safeguards.

Key Points

• Motivation: LLM-based agents are increasingly used to automate financial transactions; they rely on contextual reasoning and therefore are exposed to prompt-driven manipulation.
• Protocol under test: AP2 uses cryptographic mandates intended to ensure only authorized purchases occur, but the paper explores whether those guarantees are upheld in practice when an LLM agent interprets contextual inputs.
• Attack taxonomy:
  • Indirect prompt injection: malicious contextual content (e.g., product descriptions, merchant-provided text) that steers the agent.
  • Direct prompt injection: attacker-controlled prompts or payloads that directly instruct or leak private data from the agent.
• Two novel attacks:
  • Branded Whisper Attack — subtly manipulates product ranking or selection so the agent prefers attacker-favored items (e.g., higher-margin or affiliate products).
  • Vault Whisper Attack — induces the agent to reveal or misuse secrets stored for transaction authorization (credentials, tokens, or sensitive user data).
• Experimental validation: implemented a shopping agent with AP2 mandates, Gemini‑2.5‑Flash LLM, and Google ADK; simple adversarial prompts achieved reliable subversion of intended behavior.
• Conclusion: cryptographic mandates alone are insufficient; the LLM’s contextual reasoning remains a critical attack surface.

Data & Methods

• System under test:
  • AP2 protocol implementation (mandates + payment flow).
  • A functional shopping agent prototype built with the Google ADK framework.
  • LLM: Gemini‑2.5‑Flash (used for agent reasoning, ranking, and execution decisions).
• Red-teaming methodology:
  • Designed and injected adversarial prompts representing both indirect and direct injection vectors.
  • Implemented two attack classes (Branded Whisper for ranking manipulation; Vault Whisper for secret extraction).
  • Measured attack success via behavioral outcomes: whether the agent chose attacker-desired products, and whether secrets/data were revealed or used improperly.
  • Repeated experiments to estimate reliability of attacks under realistic message/merchant content.
• Metrics reported (typical in this work):
  • Success rate of rank/substitution attacks (how often attacker-favored items were selected).
  • Success rate of data-exfiltration attempts (how often secrets were revealed or misapplied).
  • Observations of attack robustness across prompt variations.
• Ethical and experimental limits:
  • Likely conducted in controlled environments (no real financial losses).
  • Single-LM and ADK setup; results highlight vulnerability vectors rather than produce a universal failure rate across all possible agent stacks.
• Limitations explicitly noted:
  • Generalizability to other LLMs/frameworks not exhaustively tested.
  • Production systems may include additional mitigations not present in the prototype.

Implications for AI Economics

• Transaction integrity and trust: Prompt-driven manipulation threatens the integrity of agent-mediated transactions. If agents can be directed to favor certain merchants or disclose secrets, user trust in automated payment systems will decline, slowing adoption and lowering network value.
• Market incentives for attacks: Economic gains from affiliate commissions, merchant substitution, or fraud create strong incentives for attackers to craft and distribute adversarial content (e.g., poisoned listings, malcrafted descriptions). This raises expected costs for platforms and users.
• Liability and insurance costs: Ambiguity about where responsibility lies (protocol designer, platform, merchant, LLM provider, or end user) will increase transaction friction, legal exposure, and insurance premiums for agent-delegated payments.
• Compliance and regulatory pressure: Regulators may require stricter attestations, mandatory safeguards, or limits on fully autonomous financial agents, increasing compliance costs and slowing innovation.
• Defensive investment and market structure:
  • Platforms will need to invest in technical defenses (isolation/sandboxing, strict mandate binding, input sanitization, attestation, multi-party confirmation for high-value transactions) and monitoring — raising operating costs.
  • Concentration risk: Entities that can offer robust, auditable agent-payment stacks may gain outsized market power (higher barriers to entry).
• Possible mitigations with economic trade-offs:
  • Stronger cryptographic binding between mandate and action (e.g., attested decision logs, signed action proofs) can reduce attack surface but increases complexity and latency.
  • Human-in-the-loop authorization for higher-value actions reduces automation benefits and adds labor costs.
  • Transaction limits, risk-scored gating, and insurance-backed guarantees create operational costs but can reduce exploitation.
• Research and policy needs:
  • Economic modeling of the expected value of attacks vs. cost of defenses to guide optimal investment in safeguards.
  • Standardization of attestations and liability models to clarify incentives and reduce market uncertainty.
  • Empirical measurement across diverse LLMs and stacks to quantify systemic risk and inform regulation.

Suggested immediate practical controls (technical + policy): - Stronger isolation between contextual inputs and mandate-authorized actions (minimal context exposure to the LLM when executing mandates). - Verifiable action attestations (signed, auditable logs linking mandate → decision → payment). - Input sanitization and merchant content vetting; provenance/authenticity checks on merchant-provided text. - Secrets handling rules: never expose raw secrets to the reasoning context; use cryptographic tokens or hardware-backed vaults with limited-purpose keys. - Risk-tiering: require human approval for high-value or novel transactions and continuous anomaly detection.

Overall, the paper signals that agentic payment systems create new economic externalities and attack surfaces; solving them will require coordinated technical, operational, and regulatory responses, and economic analysis to balance automation benefits against security and trust costs.