Generative AI produces a new class of 'synthetic knowledge' whose probabilistic, weakly grounded nature requires provenance-aware architectures and lifecycle governance; treating GenAI outputs as conventional information risks systemic reuse of brittle or misleading knowledge.

Main Finding

Generative AI produces a distinct class of epistemic artifacts—termed "synthetic knowledge"—that are probabilistic, weakly grounded, and opaque. When embedded and reused across organizational information systems, synthetic knowledge can systematically distort information quality and trust calibration, creating systemic epistemic risks (automation bias, contamination loops, recursive error amplification). Managing these risks requires provenance-aware, auditability-by-design architectures and lifecycle-sensitive governance, with significant economic implications for firms, markets, and regulators.

Key Points

• Definition: Synthetic knowledge = epistemic artifacts generated by probabilistic AI systems, lacking stable, inspectable lineage. Formally:
  • Ks = ⟨O, G, P, C⟩ where O = observable output, G = generative process, P = provenance trace (partial/absent), C = contextual conditions.
• Core epistemic properties:
  • Probabilistic truth status (generation via P(Y|X) = ∏ P(yt | y<t, X; θ))
  • Opaqueness / latent generative logic
  • Source indeterminacy (weak or missing provenance)
  • High context sensitivity (outputs vary with prompts and environment)
• Lifecycle model: generation → reuse → amplification → contamination. Reuse and institutionalization of unvalidated outputs amplify perceived legitimacy and can lead to recursive contamination (AI outputs re-ingested as inputs).
• Information quality and trust:
  • Perceived information quality IQp = f(C, L, U) (surface coherence C, linguistic fluency L, perceived uncertainty U) can diverge from epistemic quality.
  • Trust miscalibration metric: εT = |Tu − Rs| (user trust vs system reliability), synthetic knowledge tends to increase εT via surface fluency.
  • Epistemic accountability: Ae = h(Pv, Vm, Ho) (provenance visibility, validation mechanisms, human oversight).
• Systemic risk: errors often arise from recursive reuse across systems rather than isolated model failures. Governance gaps (treating GenAI outputs like conventional information) exacerbate risks.
• Recommended system design & governance: explicit uncertainty communication, robust provenance trails, auditability-by-design, lifecycle-aware risk controls, and aligning accountability with epistemic responsibility.

Data & Methods

• Method: theory-oriented literature synthesis / short review focused on conceptual integration rather than empirical meta-analysis.
• Scope: peer-reviewed IS and socio-technical design literature from 2024–2026, emphasizing studies on GenAI in organizational/institutional contexts.
• Inclusion criteria: peer-reviewed reviews addressing GenAI or related generative tech in organizational settings and touching on nature/quality/governance of AI outputs.
• Analytical approach: abductive iteration between literature-derived patterns and conceptual frameworks (information quality theory, decision support systems, trust/automation bias, IS governance).
• Limitations: conceptual/theory-building focus; no new empirical data collection or quantitative validation of proposed constructs.

Key formalizations in the paper: - Generation: P(Y|X) = ∏{t=1}^T P(y_t | y{<t}, X; θ) - Perceived IQ: IQ_p = f(C, L, U) - Trust calibration error: εT = |Tu − Rs| - Epistemic accountability: Ae = h(Pv, Vm, Ho) - Synthetic knowledge artifact: Ks = ⟨O, G, P, C⟩

Implications for AI Economics

Practical and research-relevant economic implications:

• Firm-level value and risk

  • Knowledge capital depreciation: Contamination of organizational knowledge bases can reduce effective human capital and decision quality, lowering firm productivity and valuation.
  • Hidden costs of adoption: Gains from efficiency/creativity may be offset by costs for validation, provenance infrastructure, audits, and liability management.
  • Investment incentives: Firms face incentives to invest in provenance, audit trails, and validation pipelines; the optimal investment depends on network externalities from recursive reuse.
  • Liability and insurance markets: Increased epistemic risk creates demand for new insurance products and contractual mechanisms allocating liability for decisions based on synthetic knowledge.

• Market structure & competition

  • First-mover vs. follower dynamics: Early adopters can reap productivity gains but may internalize higher epistemic-risk costs; market dynamics hinge on firms’ ability to manage lifecycle risk.
  • Information asymmetries: Opacity of synthetic knowledge increases information asymmetry between providers, integrators, and consumers (e.g., vendors vs. enterprise buyers), affecting pricing and procurement contracts.
  • Barriers to entry: Costs for building trustworthy, provenance-aware systems may raise barriers for smaller firms, concentrating advantage with larger firms that can fund governance infrastructure.

• Externalities and systemic risk

  • Interconnected externalities: Contamination loops create negative externalities across organizational networks (e.g., industries or public-sector ecosystems), justifying policy/regulatory responses.
  • Public-good aspect of provenance standards: Standardized provenance and audit mechanisms yield broad social benefits; private markets may underprovide them.

• Policy and regulation

  • Need for rules on provenance, audit trails, and minimum validation in high-stakes domains (finance, healthcare, public policy).
  • Potential for certification regimes, mandatory disclosure of model provenance/augmentation, and liability clarity to correct market failures.
  • Competition policy: Monitor how governance costs and information opacity influence market concentration.

• Research agenda for AI economics

  • Quantify externalities: Measure how synthetic knowledge reuse propagates errors across firms/markets and estimate social cost.
  • Cost–benefit models: Formalize trade-offs between productivity benefits and governance/validation costs; derive optimal governance investments.
  • Contracting and liability: Study contracting solutions, warranties, and insurance mechanisms to allocate epistemic risk.
  • Market for trust: Model how provenance visibility, certification, and signaling affect demand for GenAI services and pricing.
  • Empirical lifecycle studies: Track real-world lifecycle dynamics (generation → reuse → contamination) and their economic impacts across sectors.
  • Policy evaluation: Analyze how regulatory interventions (provenance mandates, auditability requirements) affect innovation, adoption, and welfare.

Suggested empirical approaches - Event studies linking GenAI deployment + governance interventions to firm outcomes (productivity, errors, stock market reaction). - Network analyses tracing reuse flows of AI-generated artifacts across platforms and organizations. - Field experiments testing different provenance/audit interfaces on decision quality and reliance (measure εT). - Cost-accounting studies estimating incremental governance and validation costs required to maintain epistemic accountability.

Overall, the paper reframes generative AI’s economic calculus: benefits from productivity and innovation must be evaluated jointly with lifecycle-driven epistemic risks and governance costs. Addressing synthetic knowledge externalities—through technical design, contracting, and regulation—will be central to realizing net economic gains from GenAI.