0 cumulative citations
View corpus contextA formal mechanism links stakeholder contributions to signed compute licenses: breadth-weighted quadratic funding plus a two-threshold gate converts human support into self-enforcing compute budgets for deployed AI, implemented via hardware attestation. The design is rigorous and novel but depends on trusted identity/attestation infrastructure and leaves manipulation of the governing electorate by the agent as a key unresolved risk.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
We give a formal mechanism design model for the continuous participatory governance of a deployed AI agent. The mechanism is built on the principle that governance should control an AI agent through resource allocation so as to make authorization self enforcing via compute budgets. The mechanism seeks to establish the Safe AI paradigm that compute is an effective governance lever. We situate our work as a compliance or commons overlay on a deployer. One governance period is an extensive form game in which verified human stakeholders arrive sequentially and contribute, on a provision or a rejection market, in a governance currency that is deliberately distinct from the agents compute. A funding aggregator turns raw contributions into breadth weighted effective supports - a two threshold gate with hysteresis converts net support into a binary authorization that, through a coupling map bounded by an exogenously certified safety ceiling, releases a metered compute budget - realized in hardware as a signed compute license so that the decision is self-enforcing. We characterize the class of agents the mechanism can govern and isolate manipulation of the governing electorate by the governed agent as the central open problem. We also introduce several challenges addressing manipulation of governing electorate by the governed agents.
Summary
Main Finding
The paper formalizes a continuous participatory governance mechanism for deployed AI agents that governs operation by allocating verifiable compute budgets (signed compute licenses). Using a two-sided, breadth-weighted (quadratic-funding) provision/rejection market with securities for early commitment and a challengeable attestation process for outcomes, the mechanism makes external authorization self-enforcing via hardware-enforced compute caps. Key theoretical results show (i) authorization tracks breadth of support (not raw wealth) — a "breadth-weighted authorization" theorem, (ii) sided incentive-compatibility and incentives for early commitment, and (iii) run-feasibility conditions tying governance margins and a compute-subsidy parameter to physical runnable budgets.
Key Points
- Core design: Decouple a human-anchored governance currency from the agent’s compute; link them only through a monotone coupling map ρ that converts net breadth-weighted support into a metered compute budget β, capped by an exogenous safety ceiling Γ and realized as a signed compute license.
- Two-sided markets: Parallel Provision (authorization) and Rejection (halt) markets collect nonnegative contributions from verified human stakeholders; each side is aggregated via a quadratic-funding (QF) aggregator ϕ((√c) sum squared) so support scales with number of distinct contributors rather than wealth.
- Hysteresis & quorum: A two-threshold gate (κstart ≥ κhalt) with a participation floor H0 implements hysteresis (prevents flapping and anti-capture quorum) and requires absolute breadth on the supporting side.
- Securities & early-commitment: Contributors receive securities (cost-function market) that reward earlier contributions in refund events; securities pay bonuses if the proposal fails (incentivizes early, honest participation).
- Outcome verification: Verifier split into Vhard (cryptographic attestation: model ID, compute consumed, procedural facts) and Vsoft (semantic adjudication: verified-harm indicator ¯H). Semantic findings are challengeable via appeals.
- Belief elicitation: Use proper scoring / prediction markets where ground truth is available; otherwise use RBTS peer prediction to reward calibrated beliefs (νi) from a bounded budget.
- Transfers & liability: Contributions are escrowed; if authorized, escrow funds provisioning and oversight; if rejected, refunds + early-commitment bonuses. A deployer liability bond Λ is forfeited on verified harm and redistributed to harmed objectors; operating within the attested authorized envelope reduces deployer liability exposure (safe-harbor).
- Scope & admissibility: Mechanism applies to "club-or-commons" agents with bounded stakeholder communities, reversible and compute-scaled impact, genuine contestation, repeated operation, and attestable outcomes. Catastrophic/irreversible harms are out of scope (dominated by the exogenous safety ceiling).
- Central open problem: Manipulation of the governing electorate by the governed agent (through endogenous signals like the agent’s past behavior shaping stakeholder beliefs) — a robustness challenge without a public-goods analogue.
- Adoption pathways: Mandated compliance (regulator), commons/co-op deployments (DAO, municipalities), or internal lab gates. The mechanism is framed as an overlay on deployers, not replacing internal alignment efforts.
Data & Methods
- Nature of study: Formal, theoretical mechanism-design model and proofs — no empirical datasets.
- Mathematical framework:
- Extensive-form sequential game per licensing generation (players = verified human stakeholders; incomplete information on types).
- Types: each stakeholder i has a signed valuation θi and belief parameter εi (probability agent behaves acceptably).
- Quadratic-funding aggregator ϕ(c1,...,cm) = (Σ √ci)^2 to compute breadth-weighted supports S+ (supporters) and S− (objectors).
- Gate logic: Dg (binary authorization) determined by S+, S−, participation floor H0, hysteresis thresholds κstart/κhalt, and Safe(g) (exogenous certified-envelope predicate).
- Coupling to compute: β = Dg · min(Γ, ρ(S+ − S−, µ)), where µ is a compute-subsidy parameter and Γ is exogenous safety ceiling; ρ is continuous and strictly increasing in net support.
- Key formal results:
- Breadth-weighted authorization theorem: authorization scales with effective number of backers × intensity (via QF), not raw wealth.
- Sided incentive-compatibility and early-commitment equilibria through securities design (adapted from PPS family and cost-function markets).
- Lemma (Run-feasibility): conditions (involving κhalt/κstart, µ, and Γ) under which β ≥ h0 (the minimum compute needed to run) whenever Dg = 1.
- Mechanism components borrow/adapt from prior literature:
- Provision-point mechanisms (PPM, PPS), securities/early-commitment from PPS family.
- Quadratic funding (Buterin et al.) repurposed as support→compute conversion (µ read as compute subsidy rather than matching pool).
- RBTS peer prediction and prediction-market scoring for beliefs and forecasts.
- Hardware primitives assumed available exogenously: workload attestation, offline signed licensing, flexHEG-style enforcement.
- Assumptions & substrate requirements:
- Sybil-resistant identity substrate (verified distinct human stakeholders) is assumed external.
- Exogenous certified safety ceiling Γ and minimum compute h0 not settable by designer.
- Attestation infrastructure (Vhard) and credible semantic adjudication (Vsoft) required.
- Liability bond mechanism requires legal/financial enforceability.
Implications for AI Economics
- Compute-as-governable-resource: The paper operationalizes compute as an allocable, excludable scarce resource that can be monetized and governed. This reframes governance design problems into allocating constrained compute budgets rather than modifying internal models.
- New instruments and markets:
- Governance currency and parallel provision/rejection markets create a market for authorization preferences distinct from compute consumption markets.
- Quadratic funding-style breadth amplification favors dispersed small supporters; this affects redistribution and matching policy design (e.g., public compute pools acting as µ).
- Securities and belief-reward budgets imply novel financial instruments tied to governance participation (early-commitment securities, RBTS reward budgets).
- Platform and policy design trade-offs:
- Decoupling governance currency from compute enables regulators or publics to influence operation without direct control of deployers’ internal models. Adoption is eased by a liability safe-harbor for deployers that stay within attested authorizations.
- Designer knobs (H0, κstart, κhalt, µ, Λ) permit trade-offs between legitimacy (broad participation), operational feasibility (enough compute), and safety (caps via Γ). Economic calibration of these knobs will be crucial (e.g., how large must µ or Λ be to make authorization feasible and credible).
- Redistribution and public funding role:
- µ (compute subsidy) is functionally similar to a matching pool but interpreted as compute subsidy; public or philanthropic compute allocations could be routed to incentivize governed deployments.
- The model highlights how governance subsidies (compute, oversight) can be made conditional on participatory authorization.
- Limits and risks with economic consequences:
- Applicability limited to reversible, compute-scaled regimes with attestable outcomes — many high-risk domains (irreversible harms) are excluded, requiring alternative policies.
- Central robustness risk: governed agents may strategically influence stakeholder beliefs via their own behavior or information flows (endogenous ε signals). This creates an economic externality where deployed agents invest in manipulating future governance — a new class of regulatory arbitrage and capture risk requiring further mechanism design or legal mitigations.
- Dependence on identity, attestation, and adjudication infrastructures implies deployment costs and market power considerations (who controls attestation hardware, who provides µ, who adjudicates harm).
- Research and policy directions:
- Empirical calibration: estimate how QF-style breadth amplification and µ interact to produce practical compute budgets for real-world agents.
- Robustness to manipulation: design electorates and signaling/attestation protocols that limit agent-driven distortions of stakeholder beliefs (and study strategic dynamics across generations).
- Market design: analyze optimal sizing/sourcing of µ (compute subsidies), liability bond sizing Λ, and incentive-compatibility under heterogeneous stakeholder wealth and participation costs.
- Institutional design: explore legal enforceability of signed compute licenses, cross-jurisdictional liability, and platforms’ incentives to adopt overlays versus resisting them.
Summary: The paper proposes a principled, formally analyzed mechanism that leverages compute metering and hardware attestation to make participatory governance self-enforcing. It contributes a blend of quadratic funding, provision-point mechanics, securities, and attested licensing to prioritize breadth of political support over raw wealth. The approach opens economic avenues for subsidized, accountable compute allocation but raises important practical and theoretical questions about identity, attestation infrastructure, and strategic manipulation by governed agents.
Assessment
Claims (11)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| The proposed governance mechanism makes authorization self-enforcing by coupling a binary human authorization decision to a metered compute budget enforced through a signed compute license. Governance And Regulation | positive | Whether governance authorization is enforceable through compute allocation |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The mechanism's authorization decision is designed to weight the breadth of stakeholder support rather than raw wealth or total contributions. Governance And Regulation | positive | Breadth-weighted authorization |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The two-sided quadratic-funding gate can authorize deployment even when objectors contribute more in raw currency than supporters, if supporters have greater breadth-weighted effective support. Governance And Regulation | positive | Authorization decision under breadth-weighted aggregation |
Reading fidelity
high
Study strength
medium
|
n=6
|
| The mechanism uses hysteresis so that starting the agent requires a higher support margin than continuing an already authorized deployment, reducing authorization flapping while allowing halting when support falls below the halt threshold. Organizational Efficiency | positive | Stability and responsiveness of deployment authorization |
Reading fidelity
high
Study strength
high
|
not reported
|
| The certified safety envelope is a hard constraint on authorization: deployment is unauthorized whenever the exogenous safety predicate fails, regardless of stakeholder support. Ai Safety And Ethics | positive | Safety-constrained deployment authorization |
Reading fidelity
high
Study strength
high
|
Dg ≡ 0 outside the certified envelope
|
| Released compute is capped pointwise by the exogenously certified safety ceiling Γ and cannot be increased beyond Γ through stakeholder contributions. Ai Safety And Ethics | positive | Maximum compute/capability released to the agent |
Reading fidelity
high
Study strength
high
|
β ≤ Γ
|
| Under the model's monotonicity assumptions and Γ ≥ h0, an authorized deployment receives at least the minimum compute provision point h0 if and only if the coupling map evaluated at the halt margin reaches h0; equivalently, the subsidy parameter satisfies the corresponding minimum threshold. Organizational Efficiency | positive | Run feasibility of an authorized deployment |
Reading fidelity
high
Study strength
high
|
β ≥ h0 iff ρ(κhalt, µ) ≥ h0
|
| The participation floor H0 functions as an absolute quorum and anti-capture requirement, independently of the physical run-feasibility condition determined by the support margin, subsidy, and hardware ceiling. Governance And Regulation | positive | Legitimacy and resistance to capture in authorization |
Reading fidelity
high
Study strength
high
|
not reported
|
| The model separates cryptographically verifiable factual attestation from trusted semantic adjudication of harm, and makes harm findings challengeable through an appeal process. Ai Safety And Ethics | positive | Verifiability and accountability of AI deployment outcomes |
Reading fidelity
high
Study strength
medium
|
not reported
|
| The mechanism is intended only for agents with a bounded stakeholder community, reversible and compute-scaled impacts, genuine two-sided contestation, repeated operation, and attestable outcomes; catastrophic or irreversible harm is explicitly outside its scope. Governance And Regulation | negative | Scope and applicability of the governance mechanism |
Reading fidelity
high
Study strength
medium
|
not reported
|
| Manipulation of the governing electorate by the governed AI agent is identified as the mechanism's central unresolved problem. Ai Safety And Ethics | negative | Robustness of participatory governance to agent manipulation |
Reading fidelity
high
Study strength
speculative
|
not reported
|