A low-cost PPO controller can match most of the accuracy of frontier LLM co‑scientists in a protein-characterization task while being faster, deterministic and cost-free; expert-prompted LLMs edge out accuracy but at materially higher token cost and lower consistency, and federated deployment imposes almost no performance penalty.

Main Finding

Federation topology has negligible impact on scientific-agent performance, but the choice of controller drives the dominant tradeoffs: a cheap, deterministic PPO policy delivers near-frontier accuracy (≈88%), perfect reproducibility, and minimal latency/cost, whereas LLM-driven harnesses (especially expert-prompted frontier models) give higher peak accuracy (Opus ≈92–94%) at substantially greater token cost and lower consistency; low-cost LLMs (o4-mini) performed much worse (~40–50%). Prompt sensitivity grows when proteins are novel (low BLAST confidence).

Key Points

• Controller choice dominated outcomes. Model selection (Opus vs o4-mini) mattered far more than federation or prompt style.
  • Claude Opus 4.8 (expert prompt): ≈92–94% accuracy (highest), but high token cost and lower trial-to-trial consistency.
  • o4-mini (zero/weak prompt): ≈40–50% accuracy (worst).
  • PPO (classical reinforcement learning, off-the-shelf stable-baselines3): ≈88% accuracy, zero token cost (no LLM calls), fastest latency, perfect determinism/consistency, but provides no language reasoning trace.
• Cost structure and tokens are material. Example pricing used: o4-mini $3.43 / 1M tokens; Opus 4.8 $18.97 / 1M tokens. Tool costs modeled as $5 per BLAST call, $10 per AlphaFold call.
• Federation topology (monolithic vs federated across labs) imposed a negligible penalty on task performance in this setup; orchestration and telemetry made cross-site deployments feasible without large accuracy loss.
• Prompt engineering matters, especially for hard/novel items: expert prompts increased LLM accuracy but also increased token usage and produced less consistent outputs across rollouts.
• PPO learned a near-universal AlphaFold-escalation strategy (often calling structure prediction), while expert-prompted Opus rarely called AlphaFold yet maintained high accuracy—different emergent strategies can reach similar accuracy with different costs.
• Observability vs reasoning tradeoff: deterministic controllers (PPO) give full reproducibility and stable decisions but no natural-language chain-of-thought; LLM harnesses provide reasoning traces useful for interpretability and discovery but are stochastic and more expensive.

Data & Methods

• Task and benchmark:
  • Verifiable protein function characterization: given UniProt accession/sequence, agent predicts functional annotations (Gene Ontology terms for Molecular Function and Biological Process).
  • Dataset: 314 unique proteins sampled to cover BLAST-confidence regimes (well-characterized bacteria for high BLAST identity; divergent eukaryotes N. gruberi and H. vulgaris for low BLAST identity). Dataset split (paper reports multiple splits; training/test subsets used for experiments and prompt tuning).
  • Evaluation: 5 rollouts per protein; metrics aggregated across 10 ablation experiments.
• Tools and signals:
  • BLAST (sequence homology) returns percent identity (treated as BLAST confidence) and supports GO-term transfer; AlphaFold returns pLDDT structural confidence (used to shape reward but not for GO transfer).
  • Tool costs and latencies explicitly modeled in reward.
• Experimental variables (ablation):
  • Two federation topologies: monolithic (all tools local) vs federated (tools hosted at separate lab instances, cross-gateway auth via Keycloak).
  • Two harness classes: classical PPO RL controller vs prompt-driven LLM harness.
  • Two prompt configurations: minimal (zero-shot) vs augmented/expert (explicit thresholds and routing strategy).
  • Two LLMs tested: o4-mini (low-cost) and Claude Opus 4.8 (frontier).
• PPO agent:
  • Observation: 9-dim numerical vector (seq length, BLAST conf, AlphaFold conf, accumulated cost, tool-called indicators).
  • Action space: Discrete(3) — call BLAST, call AlphaFold, STOP. Episodes ≤5 steps.
  • Training: stable-baselines3 PPO, default hyperparameters, trained quickly (~15 minutes on CPU), three seeds per condition.
  • Reward combined accuracy, per-tool cost penalties ($5 BLAST, $10 AlphaFold), wall-clock penalty, and bonuses (pLDDT, BLAST confidence).
• LLM harness:
  • Single-pass inference per protein via ADEPT orchestration, temperature 0.0 (greedy decoding).
  • Prompt templates refined on training data to ensure formatted ROUTING_SUMMARY traces; no gradient updates to models.
  • Token usage and cost estimated from traces (system prompt ~11k tokens excluded).
• Metrics:
  • Accuracy (binary per protein if any predicted GO term overlaps ground truth).
  • Consistency (agreement across 5 rollouts; 0.5–1.0 scale).
  • Tool efficiency (accuracy + confidence normalized over cost ratio).
  • Token/inference cost and wall-clock speed.

Implications for AI Economics

• Cost vs performance tradeoffs are actionable and large:
  • Deterministic, low-cost controllers (PPO-style) can achieve near-frontier accuracy for routine, well-specified, verifiable tasks at minimal marginal cost (no LLM tokens) and with perfect reproducibility — strong case for deploying such controllers to reduce operational budgets.
  • Frontier LLMs yield higher peak accuracy on hard tasks but impose significant token costs and produce inconsistent outputs, raising expected per-task costs and the need for repeated runs or human oversight.
  • Low-cost LLMs may be false economy for scientific tasks: o4-mini performed poorly here, so token-price alone is not a reliable proxy for cost-effectiveness.
• Federation is nearly free (in accuracy terms) but not irrelevant:
  • Cross-institution, federated tool hosting can be supported without major performance degradation; economic planning should therefore focus on controller and token costs rather than assuming federation will drive large accuracy penalties.
  • However, federation introduces modest operational overheads (auth, telemetry, slightly increased latency/calls) that need accounting in budget and SLAs even if accuracy is unaffected.
• Observability and auditability have economic value:
  • Deterministic policies simplify auditing, reproducibility, and regulatory compliance (lower verification/testing cost). They reduce hidden costs (repeated experiments, investigator time) tied to nondeterministic LLM outputs.
  • LLM reasoning traces are valuable for discovery and human interpretation but increase costs: tokens, human review, and probabilistic validation pipelines.
• Procurement and system design recommendations:
  • Use deterministic RL or rule-based controllers for routine, verifiable workflows to minimize per-task cost and guarantee reproducible outcomes.
  • Reserve expensive, prompt-engineered LLM reasoning for open-ended discovery, hypothesis generation, or when dealing with low-confidence / highly novel inputs; expect higher marginal cost and invest in prompt engineering and trace capture.
  • Adopt hybrid architectures: cheap deterministic controller by default, with an LLM fallback triggered under high uncertainty (e.g., BLAST confidence < threshold). This captures most economic upside with limited LLM spend.
  • Model and track token costs, tool call costs, and latency in procurement decisions. Large per-token price differences (example: $3.43 vs $18.97 per 1M tokens) multiply across high-throughput workflows.
• Risk & policy considerations:
  • Deterministic controllers lack natural language reasoning traces — this reduces transparency even as it improves reproducibility; organizations must weigh traceability vs determinism when regulatory auditability is required.
  • Investing in observability (per-tool cost attribution, per-call telemetry) is essential to translate technical tradeoffs into economic decisions and to enforce budget controls in federated deployments.

Limitations to keep in mind (affecting economic generalizability) - The task is narrowly scoped (GO-term transfer via BLAST + optional AlphaFold), so results may not generalize to all scientific domains or more open-ended multi-step discovery tasks. - PPO used was a modest off-the-shelf implementation and observation/action design was simplified; more sophisticated learned controllers might change the tradeoffs. - LLM experiments used temperature 0.0 and no policy fine-tuning—different deployment settings (higher temperature, chain-of-thought prompting, RLHF updates) can change cost/accuracy/consistency profiles.

Actionable takeaway - For routine, verifiable scientific workflows: prioritize cheap deterministic controllers (or hybrid default controllers with LLM fallbacks) to minimize cost and ensure reproducibility. Allocate frontier LLM spend selectively to tasks requiring flexible reasoning or when handling novel data with low-confidence signals.