Reinforcement-learning portfolios can meaningfully improve sustainability metrics while keeping returns stable: agents trained with ESG and sector CO2 inputs produce portfolios with markedly better ESG/emission scores than traditional Modern Portfolio Theory allocations, though pure return-focused agents still deliver the highest profitability.

Main Finding

Reinforcement learning (PPO) that combines predicted returns (from LSTMs), current financial data, ESG scores, and sector-level CO2 emissions can produce portfolios that substantially improve sustainability metrics while maintaining competitive financial performance. Return-maximizing RL agents achieve the highest profitability but poorer sustainability scores; sustainability-focused agents improve ESG/emission metrics at the cost of returns; balanced agents provide stable returns and meaningful sustainability gains versus finance-only strategies and vs. Modern Portfolio Theory (MPT) baselines.

Key Points

• Objective: build an AI-driven portfolio strategy for Nasdaq Helsinki stocks that prioritizes low-carbon / sustainable assets while maintaining acceptable financial returns.
• Algorithmic stack:
  • Time-series forecasts: LSTM models for predicted asset returns.
  • Reinforcement learning: Proximal Policy Optimization (PPO) agent acting in a portfolio environment.
  • Missing-data imputation: ANN regression for IPO/missing values.
  • Hyperparameter tuning: genetic algorithm.
• Inputs to the RL agent: LSTM-predicted returns, current returns, ESG scores, sector-level CO2 emission scores, engineered features.
• Reward design: multi-objective scalar reward combining financial return and sustainability (ESG/emissions) with tunable weights to emphasize return, sustainability, or a balance.
• Baseline: Modern Portfolio Theory portfolios (mean-variance optimization / efficient frontier).
• Empirical outcomes:
  • Return-oriented PPO: highest profitability, lower sustainability metrics.
  • Sustainability-oriented PPO: improved ESG and emission scores, reduced returns.
  • Balanced PPOs: stable financial performance with significant improvements in ESG and CO2 indicators compared to finance-only approaches and MPT.
• Limitations highlighted: sector-level emissions used when firm-level disclosures absent; omitted macroeconomic variables; sample limited to Finnish-listed equities; potential underfitting and sensitivity to historical regime changes.

Data & Methods

• Universe: Nasdaq Helsinki-listed firms (Finnish equities).
• Data sources:
  • Historical stock market prices/returns.
  • ESG metrics from providers (discussion mentions LSEG, Sustainalytics; diverse scales and methodological inconsistency).
  • Sector-level CO2 emission scores where firm-level emissions unavailable.
• Preprocessing & features:
  • Feature engineering and normalization/scaling for multi-objective reward compatibility.
  • Synthetic data visualizations and ESG distribution analyses for sectoral patterns.
• Forecasting:
  • LSTM architectures trained per asset for return prediction; model evaluations and directional accuracy used as inputs to RL.
  • ANN regression used to impute missing IPO-related data.
• Reinforcement learning design:
  • Environment: custom PortfolioEnv encoding portfolio state, rebalancing, transaction costs and action space (allocation adjustments).
  • Algorithm: PPO policy network trained with episodes; exploration and training regimen described.
  • Reward: mathematical formulation combining financial return and sustainability score; weighting strategies tested to produce return-oriented, sustainability-oriented, and balanced agents.
  • Hyperparameter search via genetic algorithm to optimize PPO settings.
• Validation & evaluation:
  • Data split into training, validation, and evaluation ranges.
  • Models compared across multiple evaluation ranges and against MPT portfolios.
  • Performance reported across financial metrics (returns) and sustainability measures (ESG quartiles, sector CO2 scores).

Implications for AI Economics

• Multi-objective RL is a viable approach for aligning investors’ sustainability preferences with portfolio performance; it offers a flexible mechanism to trade off financial returns and externalities (carbon emissions) via reward weighting.
• Policy relevance: RL strategies that internalize ESG/CO2 information can help channel capital toward low-carbon firms, supporting regulatory goals (e.g., CSRD) and national decarbonization targets.
• Market design and productization: asset managers can use RL frameworks to create configurable sustainable funds (tunable between return and impact objectives).
• Data challenges matter economically: reliance on heterogeneous ESG providers and sector proxies introduces measurement error and comparability problems; standardization of ESG and mandatory firm-level emissions disclosures would improve model quality and market efficiency.
• General equilibrium & adoption considerations:
  • Widespread use of sustainability-weighted RL strategies could change asset prices and sectoral capital allocation — potentially reducing the returns premium for green assets over time.
  • Concerns about robustness, interpretability, and regime shifts limit immediate institutional adoption; stress-testing, explainability, and causal analysis should accompany deployment.
• Research & policy directions suggested:
  • Expand to multi-country/universe settings and firm-level emissions to assess scalability and cross-market effects.
  • Incorporate macroeconomic variables, transaction frictions, liquidity constraints, and model risk into RL environments.
  • Explore alternative RL algorithms, risk-aware rewards, and methods for explainability and regulatory compliance.

If you’d like, I can extract key figures/tables or produce a one-page executive summary highlighting quantitative results and model comparisons from the thesis.