A lightweight 'cracking' sub-agent that speculatively extracts structured facts while a document is loaded halves the token cost of agentic reasoning on a Wikipedia benchmark and can approach the orders-of-magnitude efficiency of an oracle pre-structured database, preserving answer quality.

Main Finding

Agentic data cracking — forking a lightweight, speculative sub-agent to extract grounded, reusable structured facts whenever an LLM agent opens a document — can dramatically cut the token and monetary costs of multi-step, document-fanning "data reasoning" queries while preserving agentic accuracy. On FanOutQA, an ideal pre-structured database would be 28× cheaper than reasoning over raw documents; a practical agentic cracking system (with modest reuse) already reduces cost by ~53% while keeping accuracy, with larger gains as reuse accumulates.

Key Points

• Problem: Data reasoning (multi-step queries that must extract scattered facts from many large documents) is prefill-heavy — agents repeatedly load large documents and pay large token costs. Example: single FanOutQA question can consume up to ~1M tokens and cost ~\$1 (Haiku model).
• Insight 1 (workload locality): Related queries reopen the same documents and tend to demand overlapping structure; queries reveal which structure is useful.
• Insight 2 (inference amortization): When a document is already prefixed in context (KV-cache or prompt cache), a second generation with a bounded decode is cheap — use it to extract reusable structure.
• Method (Agentic Data Cracking, Adc):
  • When the main reasoning agent opens a document to answer a query, it forks a cracking sub-agent that (outside the answer path) speculatively extracts grounded entity–relation/value facts likely to be useful for future queries.
  • Extraction is semantic and selective, constrained by an output-token budget, and followed by validation/normalization.
  • Extracted items are stored as "cracked objects" (RDF-like edges) with evidence, cardinality and canonical units.
  • A catalogue view lists available subject–relation pairs per document; reasoning agents can query this store and avoid opening the document when structure exists.
  • If a structured read misses, the agent falls back to opening the raw document (the system never opens documents solely for cracking).
• Cracked object model: c = ⟨s, r, o, κ, u, ε⟩ where s ∈ entities, r ∈ relations, o ∈ (entity ∪ typed scalar), κ ∈ {singular, list}, u = unit, ε = evidence (document id + region).
• Operational behavior: cracking runs off-path (doesn't delay current answer), reuses cached prefix to avoid another prefill, and emits schema-constrained JSON for post-processing and insertion.
• Empirical results (high-level):
  • Oracle pre-structured store on FanOutQA: 28× lower cost than raw-document agentic reasoning.
  • Practical cracking with one prior related question per test question: ~53% cost reduction while preserving accuracy.
  • Case-study workload (an evolving investigation): ~3× cost reduction.
  • At the 10th percentile of per-question savings, cracking already runs ~9× cheaper.
• Durability and transfer: the cracked-object store is a durable asset (a "data moat") that benefits subsequent queries and can transfer across models, unlike KV caches.

Data & Methods

• Benchmarks and models:
  • Primary benchmark: FanOutQA (Wikipedia-based, multi-document data reasoning).
  • Model used for experiments: Anthropic Claude-Haiku-4.5 (FanOutQA agent specs).
  • Prompt caching / KV-cache reuse enabled in experiments to realize shared-prefix savings.
• Baselines:
  • Agentic reasoning over raw documents (search → open → extract → code/joins).
  • Top-k retrieval / RAG baselines (fixed retrieval).
  • Oracle (manually pre-structured database) to measure upper-bound savings.
• Implementation of Adc:
  • Added cracking interface, cracked-object read/write calls, and catalogue query tool to the reasoning agent.
  • Cracking sub-agent decodes under a bounded token budget and returns JSON-constrained facts for validation and insertion.
  • Catalogue per document enumerates subject–relation pairs and cardinalities to help the agent decide to use structured reads vs. opening a document.
  • Post-processing normalizes numbers, units, dates, expands list relations into indexable edges, and inserts only grounded facts.
• Evaluation metrics:
  • Token and monetary cost per query.
  • Answer accuracy relative to agentic baseline.
  • Distributional savings (mean, percentiles) across questions and simulated reuse workloads.
• Workload simulations:
  • Extended FanOutQA with one related (earlier) question per test question to model reuse.
  • Case study using a harder multi-document example (e.g., cast members for Hitchcock & Scorsese) to illustrate broader fan-out and greater savings.
• Limitations of evaluation:
  • Primary results reported on FanOutQA and specific case studies; real-world enterprise collections may differ.
  • Effectiveness depends on prompt-caching/KV-cache availability and on workload locality (degree of reuse).

Implications for AI Economics

• Lower marginal cost of reasoning over large corpora:
  • By turning one-off prefill costs into amortized, reusable structure, Adc materially reduces the per-query operational expense for agentic services — increasing scalability and lowering price per served query.
• Emergence of durable data assets and "data moat":
  • The cracked-object store accumulates value with usage; organizations and services that run agents on the same corpora build a reusable knowledge substrate that benefits future queries and users, enhancing switching costs and competitive advantage.
• Changes in cost structure & investment incentives:
  • Firms may trade larger upfront/incremental investment in extraction and cataloging for lower ongoing inference spend. This shifts economics from purely pay-as-you-go decode costs to hybrid investment/maintenance models.
  • Multi-tenant or centralized cracked stores can further amortize extraction costs, encouraging shared infrastructure and platform-level offerings (e.g., "structured corpora as a service").
• Pricing and product design effects:
  • Providers can offer cheaper, higher-accuracy agentic reasoning tiers for workloads with reuse/locality; pricing models could reflect durability (e.g., credits for cracked-object reads vs raw-document opens).
• Operational and deployment considerations:
  • Requires support for KV/prompt cache reuse or local serving to realize low marginal cost for cracking. Without such infrastructure, gains shrink.
  • Storage, catalog maintenance, and correctness/validation impose additional costs and risks (storage overhead, stale/contradictory facts, need for provenance/evidence).
• Risks and caveats:
  • Value depends strongly on workload locality and repetition — low-reuse workloads see limited benefit.
  • Extraction errors or inconsistent canonicalization can propagate and cause systematic inaccuracies; strong validation and provenance (evidence links) are essential.
  • Privacy, compliance, and security: extracted structured facts may need access controls, redaction, or retention policies different from raw corpora.
• Research & market opportunities:
  • New product opportunities around shared cracked-object stores, cross-model durable knowledge layers, cataloging tools, and cost-aware agent schedulers.
  • Further research needs: optimal cracking policies (what to extract when), cost-benefit thresholds for speculative extraction, robust normalization and schema alignment across documents, and techniques to measure/encourage cross-user reuse.

Summary: Agentic data cracking is a pragmatic, workload-adaptive approach that converts some of the high recurring costs of document-prefill-heavy agentic reasoning into one-time (or amortized) extractions, producing large cost savings in realistic reuse scenarios while keeping agentic accuracy. Its adoption implies new infrastructure, productization, and economic trade-offs around upfront extraction effort, durable data assets, and multi-tenant amortization.