General intelligence may be an evolutionary double-edged sword: long-lived lineages often lack it, and intelligence can generate unique existential risks; therefore engineers and policymakers should treat AGI — and especially artificial consciousness — with far greater precaution.

Main Finding

The paper challenges the widespread assumption that domain-general intelligence is intrinsically and unambiguously valuable when judged on evolutionary (long-run) timescales. Using evolutionary comparisons and philosophical argument, the author argues that general intelligence may be neither necessary nor reliably advantageous for long-term species persistence and may uniquely generate existential risks for the lineage that possesses it. This uncertainty about long-term survival value creates strong ethical reasons—grounded in deontological ethics and the precautionary principle—to be cautious about building artificial general intelligence (AGI), especially conscious AGI.

Key Points

• Framing and definitions

  • Adopts a situative conception of general intelligence: intelligence evaluated relative to specific contexts rather than an abstract, context-independent capacity.
  • Distinguishes survival value (persistence), instrumental value (usefulness), and intrinsic/moral value; focuses primarily on survival value.

• The intuitive case for general intelligence

  • Humans exemplify domain-general cognition and have achieved unprecedented ecological and technological dominance.
  • General intelligence is a “meta-capability” enabling cumulative culture, technology, and high instrumental value across many domains.
  • Empirical evidence connects cognition to individual-level benefits (e.g., lower mortality in some birds; cognitive capital and economic development).

• The evolutionary challenge

  • Anthropocentric bias: we (generally intelligent beings) select success metrics that favour intelligence (technology, cultural complexity).
  • Long-term persistence evidence: many of Earth’s longest-lived lineages (cyanobacteria, horseshoe crabs, sharks, nautiloids, crocodilians) lack anything like human-style general intelligence yet have persisted for hundreds of millions to billions of years.
  • Trade-offs and vulnerabilities: large brains are metabolically costly, lengthen development, reduce fecundity—traits associated with higher endangerment risk (cited studies such as Sol et al. 2007; Gonzalez‑Voyer et al. 2016).
  • Mass extinctions do not systematically favour cognitively sophisticated taxa; specialization and other ecological features often determine survival.
  • Existential risk paradox: general intelligence may be the only biological strategy that can create self-generated existential threats (technological self-harm), a structural liability not reducible solely to “misalignment” concerns. Even well-aligned AGI could inherit this liability because capabilities that enable unprecedented control/manipulation can also enable species-level harm.

• Ethical implications drawn

  • Given uncertainty or possible negative long-term survival value, there is a moral duty to apply strong precaution before instantiating new kinds of generally intelligent beings, especially if they are conscious.
  • Deontological and precautionary reasoning implies responsibility to ensure conditions for flourishing of any new intelligent beings we create.
  • The argument does not call for abandoning AI research outright, but for reframing priorities, governance, and moral calculus around AGI creation.

Data & Methods

• Methodological approach: conceptual analysis combining evolutionary biology, comparative evidence, philosophical argument, and literature synthesis rather than new empirical data collection or formal modeling.
• Evidence types used:
  • Comparative persistence statistics and fossil records (e.g., cyanobacteria, horseshoe crabs, sharks, nautiloids, crocodilians).
  • Empirical studies linking cognition/brain size to outcomes: cognitive buffer hypothesis (Sol et al. 2007) and work showing larger brains correlate with higher extinction risk/endangerment (Gonzalez‑Voyer et al. 2016).
  • Evolutionary theory and examples of trait-mediated extinction vulnerability (e.g., dinosaur gigantism, specialized Pleistocene megafauna).
  • Conceptual analysis of mass extinctions as natural experiments and exploration of selection pressures across timescales.
• Limitations acknowledged by the author:
  • The argument is not an empirical proof that general intelligence reduces long-term persistence—rather it raises underappreciated uncertainty and negative possibilities.
  • The paper focuses on survival value; it does not deny instrumental or intrinsic goods provided by intelligence.
  • Much of the claim rests on interpretation of comparative and theoretical evidence rather than on direct causal identification of intelligence → extinction.

Implications for AI Economics

• Risk-adjusted valuation of AGI investments

  • Standard expected-value calculations for AGI should incorporate long-run extinction/terminal-risk tail probabilities that may be generated structurally by general intelligence capabilities. This can dramatically lower the social expected value of unchecked AGI development.
  • Market valuations of firms pursuing AGI likely underprice global-externality tail risks (classic market failure). Private incentives thus may oversupply risky AGI development absent regulation or collective action.

• Social planner and policy design

  • A social-welfare framework that gives weight to long-term persistence (future generations) should impose stringent constraints or require large safety investments before permitting AGI deployment.
  • Option value: delaying AGI development until safety, governance, and survivability conditions are demonstrably improved has positive value under deep uncertainty.

• Public goods and coordination

  • AGI safety and survivability are global public goods. Because private actors capture only part of the benefits but can impose catastrophic downside externalities, there is a role for public funding of safety research, international coordination, and binding governance regimes.
  • Mechanisms: conditional licensing, international treaties, liability regimes, moratoria on certain types of consciousness-capable AGI, and mandated stress-testing for survivability.

• Insurance, regulation, and incentives

  • Insurance markets may be incomplete or unavailable for existential risks. Regulatory measures—risk-weighted capital requirements, phased deployment, transparency mandates—should be considered.
  • Create incentives for narrow/specialized AI applications that deliver economic value while lowering species-level risk, or redirect R&D toward provably safe architectures.

• Valuing conscious AGI and moral externalities

  • If AGI could be conscious, economic models must incorporate moral status and welfare of created agents (non-market moral weights), plus the responsibility to guarantee conditions for their flourishing—raising additional costs to deployment and governance.
  • Cost–benefit analyses must incorporate the moral costs of creating sentient beings under uncertain long-run prospects.

• Practical prescriptions for economists and policymakers

  • Incorporate deep-tail extinction risk terms into cost–benefit analyses and investment appraisals for AGI projects.
  • Recognize and correct for externalities via regulation, public funding for safety research, and international coordination.
  • Use precautionary decision rules (e.g., minmax-regret, conditional moratoria) when uncertainty about long-term persistence is high and consequences are irreversible.
  • Prioritize institutional safeguards that ensure any created general intelligence will be embedded in environments that make flourishing and long-run survivability plausible.

Overall, the paper suggests that economic analyses and policy frameworks for AGI must go beyond standard productivity and growth considerations to explicitly value long-run species persistence, tail existential risks, and the moral status of potentially conscious artificial agents. Failure to do so risks systematically mispricing the social costs and benefits of AGI development.