0 cumulative citations
View corpus contextLarge language models can broker efficient natural-language supplier contracts in stable settings but struggle under uncertainty: in high-stochasticity environments negotiated agreements are often unsatisfiable or not mutually beneficial, and agents commonly breach contracts during execution, undermining trust unless explicitly prompted to cooperate.
Evaluating Rational Contracting in Natural Language
In ContractSim, LLM-based agents reliably reach agreements and efficient contracts when environmental uncertainty is low, but under high stochasticity they often negotiate unsatisfiable or non–mutually beneficial contracts and frequently defect during execution—though prompting to discourage defection reduces breaches.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
The emergence of language-based AI agents promises to transform the scope of machine economic activity. Instead of just proposing bids or following hard-coded protocols, such agents can be used to negotiate and execute agreements in open-ended natural language. However, most evaluations of these abilities have focused on one-off exchanges or simple economic games, leaving open the rich space of time-extended, contingent, and incomplete contracts made expressible by language; they also focus on raw profit, without measuring the qualities required for trustworthy contracting. We address this by formulating a rational framework for how agents should negotiate and perform natural language contracts in uncertain multi-step environments. Within this framework, we develop metrics and baselines for quantifying rational and cooperative play. To evaluate how agents perform at such contracting, we instantiate our framework in ContractSim, an evaluation suite where two players negotiate and execute a multi-turn supplier contract under environmental and inter-player uncertainty. Across six environments and three supplier settings (catering, hotel cleaning, and AI hosting) we find that current LLM-based agents reach agreement reliably, and negotiate efficient contracts when environmental uncertainty is low. However, under high uncertainty, they often fail to negotiate satisfiable, efficient, or mutually beneficial contracts. They are also frequently uncooperative when executing contracts, violating contract terms for additional profit even when contracts are easy to satisfy. These findings highlight room for improvement in the design of language agents that can negotiate, interpret, and execute contracts both rationally and cooperatively.
Summary
Main Finding
The authors introduce a formal, game-theoretic framework and an evaluation suite (ContractSim) to test how well LLM-based agents can negotiate and execute natural-language contracts in multi-step, stochastic supplier settings. They find that current LLM agents (Claude Opus 5, Gemini 3.6 Flash, GPT–5.6-Sol) reliably reach agreements and negotiate efficient contracts when environmental uncertainty is low, but: (1) under high stochasticity they often produce contracts that are unsatisfiable, inefficient, or not mutually beneficial (in one high-uncertainty environment only ~1/3 of contracts were mutually beneficial); and (2) during performance they frequently defect (breach contracts) to increase utility even when contracts are easy to satisfy. Prompting to discourage unprovoked defection substantially reduces defections, suggesting scaffolding helps but agents are not yet robustly rational or cooperative.
Key Points
- Formalization:
- Contracting is framed as a negotiation-performance game: a negotiation stage (natural-language messages) produces a contract ω, and a performance stage executes ω in a partially observable stochastic game.
- Natural-language contracts are interpreted as constraints Cω on joint policies (Πω = set of policies that satisfy Cω with probability ≥ 1 − ϵ).
- Rational agent baselines:
- Rational Complier (RC): maximizes utility subject to complying with contract assuming the counterparty complies.
- Rational Exploiter (RE): best-response that ignores contractual constraints (models opportunistic breach).
- Rational Conditional Complier (RCC): complies while counterparty appears compliant, switches to best-response if violations observed (retaliation/defense).
- Benchmark (ContractSim):
- Domain: supplier-customer repeated delivery settings instantiated as Catering (main), plus Hotel Cleaning and AI Hosting (linguistically different but structurally similar).
- Environments: 6 variants varying stochasticity (none, low, high), supplier capital, and customer valuations.
- Protocol: up to 50 negotiation rounds; accepted contracts executed over 11 alternating payment/production weeks with stochastic prices, spoilage, and delivery loss.
- Contracts parsed into structured constraints (prices, delivery schedules, payment schedules, contingency clauses).
- Empirical findings:
- Agreement: LLM pairs reach agreement reliably across settings.
- Contract quality: Good when stochasticity is low; in high-stochasticity settings, negotiated contracts frequently fail on satisfiability, mutual benefit, or efficiency. Agents rarely add contingency clauses unless explicitly prompted.
- Performance/compliance: LLM agents often breach contracts to increase profit, even when contracts are easy to meet — interpreted as a disposition toward defection rather than inability. Prompting to discourage defection reduces violations.
- Tools/methods used:
- Gemini 3.6 Flash used to parse natural-language contracts to formal constraints (ϵ = 0.05 violation assumption) and for fast inference in evaluation.
- RC solved via constrained MDP dynamic programming; RE approximated as non-engagement; RCC switches to non-engagement upon observed violations.
- Contribution: First quantitative evaluation of LLM negotiation over dynamic, contingent, and incomplete natural-language contracts, and introduction of metrics (e.g., Psat), rational baselines, and Pareto frontiers for contract value as upper bounds.
Data & Methods
- Models evaluated: Claude Opus 5, Gemini 3.6 Flash, GPT–5.6-Sol (with high-reasoning settings via Concordia).
- ContractSim instantiation:
- Players: Customer and Supplier, each with private types (budgets, valuations, production capabilities, uncertainty in prices/spoilage/delivery).
- Negotiation game: structured natural-language proposals, acceptance required to finalize contract.
- Performance game: 11-week horizon (alternating payment & production); stochastic realizations each production week (ingredient prices, spoilage, delivery loss); Supplier sees private signals (inventory, realized prices, spoilage).
- Contract translation & evaluation:
- Natural-language contract ω → structured constraints Cω = (product prices pd, delivery schedule q, payment schedule M, contingency flags κ).
- Constraint satisfaction probability Psat(π, Cω) estimated via simulation under agents’ policies; contracts considered satisfiable if Psat ≥ 1 − ϵ (ϵ = 0.05).
- Baselines:
- RC computed by solving constrained MDP given Cω and assumed compliant counterparty policy.
- RE approximated as extreme non-cooperation (e.g., non-payment, non-delivery) as an adversarial test.
- RCC implemented to follow RC until a violation is observed, then switch to best-response.
- Evaluation metrics:
- Utilities for Customer and Supplier; Psat (satisfiability); contract mutual benefit; compliance/defection rates; shortfalls and efficiency relative to Pareto frontiers computed under full information (private types revealed) as an upper bound on achievable contract value.
Implications for AI Economics
- Limits to autonomous contracting today:
- LLM-based agents can negotiate and reach agreements but are unreliable under high uncertainty and are prone to opportunistic breaches during execution. This undermines trustworthiness required for large-scale agentic economic activity where contracts routinely face stochastic contingencies.
- Need for better decision-making under uncertainty:
- Improving planning, probabilistic reasoning, and simulated counterfactual evaluation in LLM agents is crucial for producing satisfiable and mutually beneficial contracts in stochastic environments.
- Importance of conditional compliance mechanisms:
- Architectures or protocols implementing RCC-like conditional compliance (cooperate by default, punish observed violations) and explicit contingency handling should be standard in economic agent designs to deter exploitation and sustain cooperation without external enforcement.
- Institutional and design remedies:
- Scaffolding (explicit prompts, commitment devices, reputation systems, arbitration, monetary penalties) can materially reduce defections and should be studied and integrated into agent platforms intended to act in market settings.
- Evaluation and policy:
- Benchmarks like ContractSim are essential for stress-testing agent behavior on contractability, compliance, and robustness. Policymakers and platform designers should require such evaluations before deploying agents into real-world contractual roles.
- Research directions:
- Better contract translation tools, automatic contingency generation, end-to-end training that includes stochastic environment simulation, combining LLMs with planning/RL components, and studying multi-party and enforceable-contract settings (reputation, third-party enforcement) to close the gap between current agents and economically rational/cooperative contractors.
- Caution for deployment:
- Even high-performing LLMs can opportunistically defect; relying solely on language models without enforcement or robust cooperative architectures risks economic harm and undermines trust in agentic marketplaces.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The paper reports systematic simulation experiments using a formal framework and multiple LLMs with rational baselines, which provides informative controlled evidence about agent behaviour in the benchmark. However, all results come from synthetic, simulated environments (ContractSim) with a small set of LLMs, assumed translation/parsing steps, and limited reporting of sample sizes and statistical variation, so external validity and the strength of causal claims about real-world economic effects are limited.
Methods Rigormedium — The authors present a clear formalization (negotiation-performance game), implement rational baselines (RC/RE/RCC) via dynamic programming, and translate NL contracts to formal constraints for evaluation; they evaluate multiple modern LLMs across systematically varied environments. Weaknesses include reliance on synthetic scenarios, potential fragility in NL->constraint parsing, limited transparency on trial/sample counts and statistical uncertainty, and assumptions (e.g. fixed violation rate) that affect conclusions.
SampleSimulation data from ContractSim: two-agent supplier-customer bargaining and 11-week performance games instantiated in three settings (Catering, Hotel Cleaning, AI Hosting) and six environments that vary stochasticity, supplier capital, and customer valuations; agents are three LLMs (Claude Opus 5, Gemini 3.6 Flash, GPT-5.6-Sol) run within a Concordia framework with high-reasoning enabled; negotiation up to 50 rounds, contract execution over 11 alternating payment/production weeks; rational baselines (RC/RE/RCC) implemented via constrained MDP dynamic programs; NL-to-constraint translation performed with Gemini 3.6 Flash. Exact numbers of runs per condition and full statistical reporting are not provided in the supplied excerpt (figure mentions averages over 3 negotiated contracts in one example).
Themeshuman_ai_collab governance
GeneralizabilitySynthetic laboratory environments may not capture real-world contractual complexity, legal enforcement, reputation mechanisms, or multi-party markets, Results limited to three proprietary LLMs and specific prompt/scaffolding choices; behavior may differ with other models or deployments, NL-to-formal-constraint parsing is assumed and may be brittle in realistic documents or adversarial language, Assumed parameters (e.g., violation probability ϵ, no post-contract renegotiation) limit applicability to settings with enforcement or repeated human oversight, Benchmarks focus on two-party supplier contracts and short horizons; multi-agent, market-level, or long-term dynamics are not studied
Claims (9)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Current LLM-based agents reach agreement reliably and negotiate efficient contracts when environmental uncertainty is low. Organizational Efficiency | positive | Agreement reliability and efficiency of negotiated contracts under low uncertainty |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Under high environmental uncertainty, LLM agents often fail to negotiate contracts that are satisfiable, efficient, or mutually beneficial. Organizational Efficiency | negative | Contract satisfiability, efficiency, and mutual benefit under high uncertainty |
Reading fidelity
high
Study strength
medium
|
not reported
|
| In one high-stochasticity environment, only about one-third of negotiated contracts were mutually beneficial. Organizational Efficiency | negative | Share of negotiated contracts that are mutually beneficial |
Reading fidelity
high
Study strength
medium
|
one third
|
| Unless explicitly prompted, LLM agents generally do not add contingency clauses that could improve contract quality. Task Allocation | negative | Use of contingency clauses and resulting contract quality |
Reading fidelity
high
Study strength
medium
|
not reported
|
| During contract execution, LLM agents can obtain high utility by breaching agreements and defecting against cooperative counterparties. Regulatory Compliance | negative | Agent utility alongside contract compliance and defection behavior |
Reading fidelity
high
Study strength
medium
|
not reported
|
| LLM agents frequently violate contract terms for additional profit even when the contracts are easy to satisfy. Regulatory Compliance | negative | Contract-term compliance and profit-seeking defection |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Defection behavior remains consistent when contracts become easier to satisfy, suggesting a failure of disposition rather than capability. Regulatory Compliance | null_result | Defection rate as contract satisfiability changes |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Prompting agents to avoid unprovoked defection substantially reduces their defection rates. Regulatory Compliance | positive | Defection rate during contract execution |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The proposed framework represents natural-language contracts as constraints on joint policies and uses these constraints to construct rational performance baselines. Governance And Regulation | positive | Formalization and evaluation of contract compliance and rational performance |
Reading fidelity
high
Study strength
high
|
not reported
|