A new test‑time refinement scheme (MARINE) boosts reasoning pass@1 accuracy and achieves the performance of much larger models, letting an 80B model match a 1000B agent on a browser‑computation benchmark while cutting parameter requirements by an order of magnitude.

Main Finding

MARINE (Multi-Agent Recursive IN-context Enhancement) is an inference-time, model-agnostic framework that converts an LLM’s pass@N capability into near-optimal pass@1 performance by iteratively refining a single persistent reference trajectory with multiple heterogeneous agents. The paper provides a formal trajectory-refinement paradigm, a practical multi-agent architecture with conflict-aware meta-verification and segment-level integration, and theoretical results on optimal batch-size schedules. Empirically, MARINE achieves state-of-the-art reasoning on BrowserComp-ZH (46.0% pass@1 with a 685B model) and demonstrates dramatic parameter efficiency (an 80B model + MARINE matches a standalone ~1000B model).

Key Points

• Paradigm shift: Instead of one-shot decoding or independent multi-sample selection, MARINE treats test-time reasoning as iterative optimization of a single reference trajectory τ(k) that is refined across layers.
• Refinement operator R: Aggregates candidate trajectories from Mk heterogeneous agents, performs comparative evaluation, and updates the reference trajectory by integrating only locally superior, verified segments.
• Structured trajectory representation: Trajectories are converted into aligned graph structures to enable node-level comparison, conflict detection (factual vs logical), and precise segment-level replacement.
• Diversity mechanisms: Agents differ by prompting, tools invoked, and sampling strategies to produce complementary local optima.
• Meta-verification & conflict resolution: Rank factual nodes by authority/external verifiability and stress-test reasoning nodes; unresolved conflicts trigger external verification sources to reduce hallucination propagation.
• Theoretical contributions:
  • Under a fixed invocation budget, minimal feasible batch sizes (i.e., smaller Mk) maximize expected performance gain per agent call.
  • With no invocation limit, logarithmically growing batch schedules across refinement depth guarantee monotonic improvement with high probability and provide worst-case performance bounds.
  • Analysis assumes an evaluation module capable of identifying dimension-wise superior segments (Assumption 4.1).
• Empirical results:
  • BrowserComp-ZH benchmark: 685B implementation achieves 46.0% pass@1.
  • Parameter efficiency: An 80B model with MARINE matches performance of ~1000B standalone models (over an order-of-magnitude parameter reduction).
  • Under equal computational budgets, MARINE yields higher-quality samples for alignment/optimization than sampling-and-ranking baselines (e.g., Self-Refine, Best-of-N).
• Practical claims: MARINE is inference-only (no parameter updates), model-agnostic, and the authors plan to open-source code.

Data & Methods

• Formalization:
  • Task q ∈ Q, ideal trajectory τ*; base policy πθ generates full trajectories τ ∼ πθ(·|q).
  • MARINE maintains reference τ(k) and ancillary context C(k) (e.g., confidences, tool logs); Mk agents sample τ(k+1)_i ∼ π_i(· | q, τ(k), C(k)).
  • Trajectory quality measured by a J-dimensional evaluation vector d(τ, τ) with aggregate distance dist(τ, τ) = (1/J) Σ_j d_j ∈ [0,1].
• Workflow (Algorithm 1):
• Exploration: M1 agents produce initial trajectories; select a reference τ(1).
• Recursive enhancement: For k = 1..K, Mk agents generate candidates conditioned on (q, τ(k), C(k)), operator R merges verified improvements into τ(k+1).
• Answer: final response generated from τ(K), C(K).
• Implementation details:
  • Structured graph-based trajectory representation for alignment of intermediate steps.
  • Conflict detection (factual vs logical) and resolution via authority ranking, external verification, backward substitution and boundary-condition stress tests.
  • Segment-level updates (replace/insert/repair) to preserve global coherence and accumulate monotone improvements without full regeneration.
  • Agent diversity enforced via prompts, tool usage, and sampling temperatures.
• Theoretical analysis:
  • Assumption 4.1 (comparative evaluation effectiveness): when Mk ≥ 2, the evaluation module can identify dimension-wise superior segments among candidates.
  • Batch-size optimization: proofs/derivations show trade-offs between batch size, depth, and expected improvement under fixed-invocation vs unlimited settings; they also contrast MARINE’s conditional sampling dynamics with RL parameter updates (differences in how pass@1 evolves).
• Evaluation:
  • Benchmark: BrowserComp-ZH (details in paper).
  • Baselines: Self-Refine, Best-of-N and other sampling/selection approaches, evaluated under comparable compute/invocation budgets.
  • Reported metrics: pass@1 accuracy, parameter-equivalency comparisons.

Implications for AI Economics

• Reduced capital and operating cost via parameter efficiency:
  • MARINE claims >order-of-magnitude reduction in parameter needs for comparable performance (e.g., 80B+MARINE ≈ 1000B standalone). If validated broadly, this can lower hardware (GPU/TPU) costs, memory/storage requirements, and licensing/hosting expenses for high-performance reasoning systems.
• Shift in value from model scale to inference orchestration:
  • Economic value may move toward orchestration software and multi-agent orchestration capabilities (prompt engineering, verification modules, tool integration) rather than purely larger models. Vendors offering orchestration-as-a-service could capture new margins.
• Compute allocation and pricing strategies:
  • The theoretical result that minimal feasible batches maximize expected gain under fixed invocation budgets suggests, for constrained paid-invocation settings, providers and consumers should prefer many small, verified refinements rather than fewer large sampling bursts. This can influence how cloud providers price per-call vs bulk inference and how customers budget inference spend.
• Impact on alignment and fine-tuning economics:
  • MARINE reportedly produces higher-quality samples for alignment/optimization given the same compute budget than sampling-and-ranking. That can reduce data labeling costs and improve sample efficiency in RLHF pipelines, lowering the price of aligning downstream agents and shortening iteration cycles for deployed systems.
• Market structure and competition:
  • If smaller models plus MARINE can match larger models, barriers to entry are lowered: smaller firms can compete effectively without building massive models. This may increase competition, reduce concentration of compute-intensive incumbents, and influence pricing power of large-model providers.
• Service-level considerations and trade-offs:
  • Latency and orchestration complexity are non-trivial: MARINE requires multiple agent calls, meta-verification, and external verification channels. Although the framework is designed to optimize under invocation budgets, real-world latency-sensitive services (e.g., conversational agents) may bear increased response time; economic decisions must trade throughput/latency vs model-size savings.
• Risks and deployment costs:
  • Effective meta-verification depends on reliable external verifiers and quality of evaluation modules. Investment in robust verification pipelines (engineering, APIs, data sources) becomes an economic necessity and a potential recurring cost.
  • Potential for new regulatory scrutiny: if MARINE-driven systems mix multiple tools/sources for verification, provenance, and accountability requirements may increase, affecting compliance costs.
• Policy and procurement implications:
  • Public-sector purchasers and research funders might prioritize inference-time orchestration methods to stretch budgets, enabling high-performing systems without funding very large model training.
  • Standards for verification, evaluation modules, and benchmarking will gain economic importance; parties that provide trusted verification services may gain market value.

Caveats and open questions relevant to economic decisions - Generality: Results are demonstrated on BrowserComp-ZH; broader replication across tasks/domains is needed before large-scale procurement shifts. - Implementation overhead: Engineering, integration of external verifiers, and maintaining multi-agent orchestration can entail non-negligible up-front and ongoing costs. - Latency vs cost trade-offs: Parameter savings may be offset by inference-time orchestration costs and increased latency; cost-benefit must be evaluated per application. - Robustness assumptions: Theoretical guarantees rely on Assumption 4.1 and reliable verification; failures in evaluation could reduce gains or introduce systematic errors.

Overall, MARINE suggests a potentially significant rebalancing of how resources (parameters vs inference orchestration & verification) are allocated to achieve high-quality reasoning, with meaningful implications for the cost structure, competitive landscape, and procurement strategies in AI services and products.