A new system, Credo, turns opaque agentic LLM harnesses into reusable declarative primitives so teams can compile task-specific harnesses in minutes and dollars rather than re-running costly searches for hours and tens of dollars, largely preserving accuracy.

Main Finding

Credo recovers a typed, declarative description from an agent-discovered (searched) LLM "harness" and catalogs its reusable primitives (skeleton steps, beliefs, policies, prompt templates) with provenance and scope tags. A compiler can then assemble harnesses for new targets from the catalog instead of re-running an expensive harness search. Empirically, Credo reproduces source-harness behavior with negligible fidelity loss and enables substantial cost/time savings when reusing primitives across related tasks.

Key Points

• Problem: modern agentic harness discovery produces opaque imperative programs that embed logical steps, control signals, model choices, and prompt strategies — knowledge that is hard to inspect, audit, or reuse.
• Analogy: similar to past DB work that recovered declarative structure from UDFs to enable optimization; Credo applies that idea to harnesses.
• Four primitive types recovered from a searched harness:
  • Skeleton: a logical DAG of operator steps (what is computed and dependencies).
  • Beliefs: typed derived attributes (bool/enum/int/float/str) computed at plan points and read by policies.
  • Policies: guarded rules mapping beliefs to physical implementations (model, params, execution pattern).
  • Prompt templates: parametric natural-language templates discovered by search (bindable slots).
• Catalog construction: abstractor extracts primitives; level-tagger labels transfer scope (L1 = functional/transferable, L2 = source-specific); librarian stores primitives in two tiers (general/ and specific/), with provenance.
• Compilation: compiler selects primitives, binds slots to target-specific values, and assembles runnable harness.py; falls back to full search if no acceptable combination exists.
• Round-trip fidelity: compiling back from extracted primitives reproduces original harness accuracy and cost with minimal change (Δ accuracy within ±0.011; Δ cost ~±0.013 ¢/q).
• Cost/time savings: a single recover+compile costs ~$2.3–2.8 and takes minutes; original agentic search (Meta-Harness) costs ~$43–72 and takes hours (per-domain runs: total wall 2.8–9.2 hours depending on domain).
• Primitive reuse: cross-domain compilation (finance↔legal, math↔polyreason) often yields sizable improvements over brief-only baselines; success depends on the policies/beliefs the source primitives encode (transfer can fail if operating points differ).

Key quantitative examples from experiments - Round-trip (per-domain mean over seeds): RT preserves accuracy (finance +0.011, legal −0.007, math −0.010, polyreason +0.007) and cost (mean Δ within ±0.013 ¢/q). - Search vs compile cost/time: search ~$43–72 and hours; Credo recover+compile ~$2.3–2.8 and a few minutes. - Reuse highlights: - Legal→finance: accuracy rose from 15.7% (brief-only) to 28.3% (compiled), similar to searched harness (27.7%). - Finance→legal: accuracy rose from 43.4% to 66.3%, recovering 76% of distance to searched result (73.4%). - Math↔Polyreason: transfer success depends on policies; polyreason→math improved math accuracy from 39.6% → 44.6%/45.4%; math→polyreason sometimes reduced accuracy versus brief-only because of mismatched operating points.

Data & Methods

• Hardware & deployment: single dual-socket AMD EPYC server, 4× NVIDIA L40S GPUs; local open-weight models served with vLLM.
• Models used:
  • Local open weights: gemma3-12b (small), gemma4-31b (large).
  • Harness discovery agent & Credo abstractor/compiler: claude-opus-4-7 (API).
  • Accuracy judge: gpt-5-mini (API).
• Harness discovery: Meta-Harness (coding-agent search), 10 iterations per search; programs chosen from Pareto frontier of accuracy vs cost.
• Datasets / domains (each with 3 splits): four task families:
  • Retrieval: Finance (DocFinQA + FinQA), Legal (CUAD-common + MAUD). For retrieval tasks, gold evidence removed so retrieval is required.
  • Reasoning: Math (OlympiadBench + Omni-MATH), Polyreason (composed reasoning benchmarks).
  • Splits per domain: search (100 queries), in-distribution (100 queries) used to select best harness, cross-distribution (300 queries) for transfer evaluation.
• Metrics:
  • Task accuracy judged by gpt-5-mini per dataset grading rules.
  • Harness inference cost: per-question inference cost (¢/q).
  • Reported both accuracy and cost (no single combined metric).
• Experiments:
  • Catalog construction: extract primitives from searched harnesses; annotate with L1/L2 scope.
  • Round-trip fidelity: extract then compile back to runnable harness, compare original vs compiled on same in-distribution split.
  • Primitive reuse: compile harnesses for unseen targets using (a) brief-only, (b) single-domain L1 primitives, (c) open-library (all non-target domains) primitives; compare with target’s own Meta-Harness result.

Implications for AI Economics

• Large upfront search costs can be amortized: building a catalog once per source domain (search + extraction) and compiling per target reduces marginal harness construction cost dramatically (example ROI: ~$43–72 → ~$2 per compile).
• Returns to scale and network effects: as catalogs accumulate L1 primitives across sources, the marginal value of new primitives grows (lower cost of entering new tasks); this favors centralized/shared catalogs or marketplace models for reusable harness components.
• Productization and business models:
  • SaaS offerings can sell compiled harnesses or curated catalogs; pricing could reflect provenance, L1/L2 status, and empirical transfer performance.
  • Enterprises can invest in per-source catalog construction to lower per-task operational costs across many use cases (positive NPV if many targets).
• Optimization & resource allocation research opportunities:
  • Cost-based compilation/optimizer: choose primitives to meet accuracy constraints at minimal cost (analogous to DB query optimization).
  • Maintenance under drift: catalog upkeep under model upgrades, new benchmarks, or dataset shifts — economic trade-offs between re-searching vs reusing/adapting existing primitives.
• Auditability, compliance, and risk management:
  • Declarative primitives (with provenance and typed beliefs) enable better auditing of decision logic, easier regulatory compliance, and clearer attribution of failure modes — potentially lowering compliance costs.
• Risks & frictions:
  • Transfer failures impose implicit costs (time, accuracy regressions); economic models should include failure probability and costs of fallback searches.
  • Potential commoditization of harness-search capabilities may shift competition toward catalog quality, metadata/provenance, and specialization.
• Research questions for AI economics:
  • How to value a primitive/catalog entry? (marginal benefit across tasks, expected reuse rate)
  • Optimal investment per-source: when is it better to search anew vs compile/adapt?
  • Market design for primitive sharing: incentives, licensing, and pricing under externalities and information asymmetries.
  • Welfare and distributional effects: who captures savings (catalog providers, application owners, end-users)?

Takeaway: Credo shows that transforming opaque agentic harnesses into reusable, typed declarative primitives can materially reduce marginal costs and latency of deploying agentic workflows across related tasks, creating clear economic incentives to build and share catalogs — but realized benefits depend on transferability of policies/beliefs and on catalog maintenance over time.