Language is the wrong substrate for high-stakes quantitative decisions: descriptions irreversibly lose the data needed for fidelity, reproducibility, provenance and calibrated uncertainty, so institutions should build 'Large Quantitative Models' trained directly on native records with explicit structure and provenance.

Main Finding

The paper argues that natural language is an insufficient substrate for building models that must make consequential quantitative decisions. The failure is structural: when a domain’s quantitative record is compressed into human description, information relevant to many quantitative queries is irreversibly lost. The authors propose a distinct model class — Large Quantitative Models (LQMs) — trained on domain-native quantitative records, with four core, inspectable properties (fidelity, reproducibility, lineage, calibrated uncertainty). They claim LQMs are necessary (not just desirable) to meet governance, regulatory and operational requirements in high-stakes quantitative domains (finance, insurance, healthcare, cybersecurity), and that attempts to make LLMs compliant (scaling, fine-tuning, tool use, retrieval) address the interface but not the substrate.

Key Points

• Formal definitions (representation-focused):
  • Fidelity: representation T(D) preserves all information needed for a family of queries Q (zero conditional entropy of q(D) given T(D)).
  • Reproducibility: outputs are a deterministic function of inputs, configuration and version.
  • Lineage: computable, preserved mapping from every output back to the source records that produced it.
  • Calibration with abstention: stated uncertainty has correct frequency coverage and system can abstain when evidence is insufficient.
• Formal information-theoretic critique:
  • Proposition 1: If a description T(D) is lossy for a quantitative query q, then any model Y built from T(D) has a lower bound on error (Fano + data-processing inequality). This bound depends only on the representation loss, not on model size or compute.
  • Conclusion: Loss introduced at the description stage is irrecoverable by scaling or bigger models.
• Four substrate-driven failure modes of language models:
  • Fidelity loss: description is an irreversible compression of the quantitative record.
  • Reproducibility loss: generation is sampling-based; intermediate explanations are often non-faithful.
  • Lineage failure: weights/implicit representations do not provide mechanistic provenance; post-hoc attribution ≠ provenance.
  • Calibration brittleness: verbalized confidence is learned from human expressions of confidence, not from coverage guarantees; abstention is not native.
• In-context provision, retrieval-augmented generation, and deterministic shells (retrieval + logging + verification) help but do not resolve substrate-level problems; they address the interface rather than provide native substrate properties.
• LQM definition (constructive claim): A model class with four positive properties:
  • Substrate-native: trained on the domain’s raw quantitative records (tabular, relational, temporal, high-D).
  • Structurally explicit: holds inspectable representations rather than only implicit weights.
  • Lineage-complete: every output maps computably to contributing records, preserved across transformations.
  • Uncertainty-calibrated: built-in coverage guarantees and an abstention mechanism.
• LQMs are distinct from, but related to, tabular/time-series foundation models, world models and classical single-task quantitative models.
• Practical/regulatory relevance:
  • Authors map the four required properties to governance regimes (US banking model-risk guidance, NAIC insurer AI bulletin, EU AI Act, medical device ML guidance) showing these regimes effectively require the properties LQMs provide.
• Evidence and corroboration:
  • Empirical literature shows LLMs often degrade on quantitative fidelity (numerical perturbation sensitivity, compositional failure, poorer performance on tabular tasks compared with domain-appropriate models).
  • Authors report a deployed system example where an LLM is confined to interface duties while a structured quantitative model performs the reasoning — presented as evidence that hybrid architectures with native-quantitative cores are feasible.
• Counterarguments addressed:
  • Scaling, fine-tuning, tool use, retrieval, and in-context provision are treated but argued not to substitute for a substrate-native quantitative representation.

Data & Methods

• Formal/theoretical analysis:
  • Uses information theory (data-processing inequality, Fano’s inequality) to formalize irrecoverability of quantitative information lost in human descriptions.
  • Precise, representation-level definitions of the four core properties (fidelity, reproducibility, lineage, calibration with abstention).
• Conceptual and comparative analysis:
  • Systematic comparison between language-substrate systems and the proposed LQM class.
  • Mapping of representation properties to regulatory/guidance requirements across sectors.
• Literature synthesis:
  • Survey of empirical studies showing LLM quantitative weaknesses (numeracy, compositional generalization, tabular prediction).
  • Review of related ML programs (world models, tabular/time-series FMs) and how they inform or contrast with LQMs.
• Case example / deployment evidence:
  • Describes (at high level) one instantiation and a deployed system where a structured quantitative model is used for reasoning and an LLM is used purely as the human interface. This serves as a proof-of-concept rather than a controlled experiment.
• Methodological scope:
  • The paper is primarily a position and formal-argument piece, combining elementary formal proofs, conceptual architecture proposals, literature review, and an illustrative deployment example. It does not present large-scale empirical benchmarks or randomized trials.

Implications for AI Economics

• R&D and investment priorities:
  • Shift funding from general-purpose LLM scale-centric efforts toward building domain-native quantitative infrastructure (data ingestion, provenance, structured modeling, uncertainty quantification).
  • Expect increased spending on data engineering, record systems, and lineage-preserving pipelines rather than solely on model parameter count.
• Market structure and competitive dynamics:
  • Firms with high-quality, well-governed quantitative records (financial institutions, insurers, hospitals, cybersecurity providers) gain an advantage: LQMs favor incumbents with privileged data and provenance infrastructure.
  • Opportunity for specialized vendors building LQM platforms, tools for provenance, and calibrated quantitative foundation models; could produce vendor concentration but also new entrants targeting vertical domains.
• Productization and monetization:
  • LQM-based products are better aligned with regulatory compliance and auditability — lowering compliance costs and legal risk for adopters in regulated industries.
  • New commercial offerings: audited-calibrated risk scores, lineage-backed decision systems, and reproducible model-as-a-service for high-stakes quantitative tasks.
• Regulatory and supervisory impacts:
  • LQMs naturally meet many audit, documentation and reproducibility requirements; regulators may demand LQM-like properties for approval in high-stakes domains, raising the bar for language-only solutions.
  • Standardization pressure: to achieve interoperability and inspection, industry standards for representation, lineage formats and coverage guarantees may emerge, with economic effects on switching costs and network externalities.
• Model risk, capital allocation and systemic risk:
  • Better-calibrated and lineage-complete models could reduce model risk, support more defensible capital and reserve calculations, and reduce operational surprises — with macroeconomic effects on risk pricing and stability.
  • Conversely, concentration of LQMs in a few large firms could amplify systemic risk if shared modeling assumptions or data biases affect many institutions.
• Labor and skills:
  • Increased demand for domain data engineers, provenance architects, uncertainty quantification experts and statisticians; decreased centrality of prompt-engineering as LLMs become interfaces rather than reasoning cores.
  • Organizational changes: stronger integration between domain experts, data engineering, and model governance teams.
• Implementation costs and adoption frictions:
  • Building LQMs requires investment in structured data capture, provenance systems, and infrastructure for reproducible training and inference — potentially high upfront costs, slowing adoption for smaller firms.
  • Long-term ROI: for high-stakes decision domains, the governance and accuracy benefits may outweigh costs, particularly where regulation or liability makes auditability essential.
• Welfare and distributional considerations:
  • Improved decision quality (pricing, triage, risk containment) can improve allocative efficiency and reduce costly errors, but gains may unevenly accrue to organizations able to invest in LQMs.
  • Privacy and data governance: reliance on raw records raises privacy risks; markets for privacy-preserving provenance and secure computation (federated LQMs, secure enclaves) may grow.
• Strategic recommendations for economists and policymakers:
  • Reassess expectations that LLM advances alone will displace specialized quantitative modeling — anticipate hybrid architectures and allocate policy/regulatory guidance accordingly.
  • Encourage standards for lineage, reproducibility and calibration to lower adoption barriers and reduce lock-in risks.
  • Support public-good datasets and tools for provenance and calibration to mitigate concentration and promote competition.

Summary takeaway: For economists, policymakers and firms concerned with high-stakes quantitative decisions, this paper recommends reorienting technical, regulatory and investment strategies away from treating language models as a universal substrate, and toward building and governing substrate-native Large Quantitative Models that provide fidelity, reproducibility, lineage and calibrated uncertainty by construction.