About & Methodology
The Commonplace tracks academic research on AI's economic impact. An automated pipeline pulls new papers every day, grades the evidence, extracts claims, and writes summaries. It is a research-acceleration tool, not a peer-review system. Every direction, evidence grade, paper type, and outcome category you see is assigned by a large language model and has not been checked by a human researcher. This page explains how the system works and where its limits are. Read it before you rely on anything here.
The sections, and when to use each
Two pages overlap the most: Evidence and Explore. The short version is that Evidence works at the level of individual claims, and Explore works at the level of whole outcome categories.
Evidence vs. Explore, in one line each
- Evidence answers "what do papers say about X?" It returns individual claims you can filter and read.
- Explore answers "how do the outcomes compare?" It returns one row per category, so you can rank topics by agreement, strength, or activity before drilling in.
Reading the labels in the digest
Every paper in the Monday digest carries two tags and, in parentheses, its study design. The two tags are independent: one says how the paper relates to what we already knew, the other says how strong the evidence behind the claim is. Study design is a separate fact about how the study was run.
Relation — how it moves the picture
- New: a finding on a question the standing evidence had not settled (also used for capability benchmarks and theory, which inform but cannot confirm an economic-outcome view).
- Confirms: same outcome, same direction, comparable population as the standing view — an independent study that raises confidence.
- Extends: same direction, but a new setting, population, or mechanism.
- Tension: a direction or magnitude mismatch, not yet strong enough to overturn anything.
- Challenges: reserved for a same-estimand finding with identification at least as strong as the standing base. A single study does not flip a verdict.
Evidence status — how strong the support is
- Established: multiple well-identified studies point the same way.
- Suggestive: real but limited — one study, or several with caveats.
- Framework: a theoretical model or argument, not an estimate.
- Descriptive: measures or documents something without identifying a causal effect.
In parentheses you may see the study design (RCT, quasi-experiment, panel, descriptive). That is how the study was run, and it is kept separate from the two tags above on purpose.
How a paper gets here
Each paper moves through these stages before it appears in the knowledge base.
Fetch
Papers are pulled daily from OpenAlex, arXiv, and Semantic Scholar (plus institutional feeds such as NBER, IZA, BLS, BEA, and Census) using keyword queries across nine AI-economics topics. Around 300–400 candidates come back per run; cross-source deduplication on DOI, arXiv ID, and fuzzy title removes repeats.
Get the full text
For papers that clear the relevance bar, the pipeline resolves an open-access PDF (via Unpaywall and a chain of OA fallbacks), downloads it, and extracts the text. Roughly four in five relevant papers end up with usable full text; the rest fall back to the abstract. A quality check flags text that is truncated or garbled so it isn't mistaken for the real thing.
Summarize (gpt-5-mini)
A structured summary is generated from the full text where it exists, and from the abstract otherwise. Summaries are not checked against the source for accuracy.
Assess and extract claims (gpt-5-mini)
The model classifies each paper by type (RCT, quasi-experimental, correlational, theoretical, survey, review, or other), assigns an evidence strength, scores relevance 1–10, and pulls out specific empirical claims with a direction (positive, negative, mixed, or null) and a confidence level. Papers scoring below 7 are deleted.
These labels are machine-assigned and unverified. "High evidence" means the model judged the study strong, working from whatever text it had — the full paper for most papers, the abstract for the rest. It does not mean a methodologist reviewed it.
Categorize claims (gpt-5-mini)
Each claim is sorted into one of 34 outcome categories (wage effects, firm productivity, automation exposure, and so on). The model also records how faithfully it read the claim from the paper and how strong the underlying study design is.
Detect tensions (SQL rule, not a model)
A plain database rule flags pairs of claims that share an outcome category but point opposite ways. These are candidates only. The rule cannot tell whether two claims describe the same population, time period, or outcome definition, so many flagged pairs are not real disagreements. The site says "tensions," never "contradictions," on purpose.
Synthesize (gpt-5)
For categories with at least 100 claims, a longer synthesis is written by gpt-5 in two passes: one to assemble the evidence, one to write the narrative. Where a synthesis page shows Reviewed by a human with a date, a person read and approved it. Where it shows Auto-generated, not human-reviewed, no one has checked the text.
Why these tools, and not others
The model choices are engineering and cost tradeoffs, not claims about which model is "best."
gpt-5-mini for the high-volume work
Summarizing, assessing, categorizing, and headline-writing run on every paper, every day — hundreds of calls per run. gpt-5-mini is cheap and fast enough to do that across the whole corpus nightly. A frontier model on each of those steps would multiply the daily cost several times over for a job where the marginal quality gain is small and nothing is human-reviewed anyway.
gpt-5 for synthesis only
Syntheses are low-volume (about a dozen pages, refreshed occasionally) but reason over a large evidence dossier, sometimes 200,000+ tokens, and the writing quality matters. That is worth a more capable model. We ran a side-by-side evaluation of candidate synthesis models against a fixed rubric (citation accuracy, hedging discipline, structure) and kept gpt-5 because it scored best on that test, not by assumption.
Why not Claude Opus, or one model for everything?
A top-tier model on every per-paper step is hard to justify at this volume when the output isn't reviewed. The pipeline is also built on the OpenAI SDK with keys already in place, so staying within one provider keeps the daily job simple and reliable. This is a deliberate cost ceiling, not a verdict that gpt-5-mini is the most capable model available.
Plain SQL for tensions and links
Detecting opposing claims and linking related ones is a deterministic rule, so it runs in SQL rather than through a model. That keeps it auditable, free, and repeatable. The cost is precision: the rule over-flags, which is why every tension is labeled a candidate.
Known limitations
Most evidence is observational
Only about 18% of the corpus uses causal or quasi-experimental designs (RCTs, difference-in-differences, IV, RDD, synthetic control). Roughly 5% are randomized controlled trials, and about 2.5% of claims are graded "high evidence." The rest is correlational, descriptive, or theoretical. The site shows this distribution rather than hiding the weaker studies, because context decides how much weight a finding deserves.
The labels are machine guesses
Paper type, evidence grade, direction, and outcome category all come from gpt-5-mini, which makes mistakes. It can misread a study design, flip an effect's direction, or file a claim under the wrong category. There is no human verification step in the current pipeline.
Tensions are not confirmed contradictions
The tension rule flags opposing claims within a category. It cannot check whether they share a population, context, or outcome definition. Treat every tension as a prompt to read both papers, not as a settled disagreement.
Effect sizes are sparse and local
When a specific number appears (say, "14% productivity gain"), it comes from one study in one setting. Effects vary widely across populations, sectors, and time. Don't treat a single extracted figure as the field's answer.
Coverage is partial
The pipeline indexes a slice of English-language academic work from a handful of sources. Working papers come in through arXiv and NBER/IZA feeds; published journal articles can lag by months. Good research will sometimes be missing.
Syntheses age
A synthesis reflects the evidence available the day it was written. New papers are added constantly, so pages are marked "update pending" when fresh evidence has arrived since. Even a current synthesis only covers what cleared the relevance bar at generation time.
Using this well
- Read the original paper before you cite a specific claim or number. Every claim links straight to its source.
- Read paper type and evidence grade together. A high-confidence claim from a correlational study is weaker than a medium-confidence claim from an RCT.
- Treat effect sizes as directional, not precise. The field rarely has enough studies to settle on a single number.
- When you hit a tension, open both papers before deciding whether it's a real disagreement.
- Use syntheses to get oriented, not to reach a verdict. They inherit every limitation above.
Questions or corrections? Contact Alex Farach.
Topics tracked
Nine areas, from firm-level productivity to macroeconomic policy.
AI & Labor Productivity
Output effects, firm-level productivity gains, task automation, and the gap between adoption and measurable impact.
AI & Labor Markets
Employment, wages, job displacement, occupational shifts, and how labor demand restructures.
Human-AI Collaboration
Augmentation versus automation, human-AI teaming, and where the two are complements.
AI & Skills/Training
Upskilling, reskilling, skill obsolescence, and returns to AI-related human capital.
AI & Organizational Design
Firm structure, management practices, and how organizations reconfigure around AI.
AI & Innovation
R&D productivity, scientific discovery, patenting, and AI as a general-purpose technology.
AI & Inequality
Wage gaps, digital divides, and distributional effects across skills, regions, and groups.
AI Adoption & Diffusion
Adoption barriers, diffusion patterns, sectoral variation, and what drives uptake.
AI Governance & Policy
Regulation, labor-policy responses, governance frameworks, and policy evaluation.
Sources
Three academic databases, plus institutional feeds, for coverage of both published work and preprints.
OpenAlex
Open scholarly metadata over 250M+ works, filtered to the Social Sciences domain (Economics, Sociology, Management).
50 results per query · 9 topic queries
arXiv
Preprint server, filtered to econ.*, cs.CY, cs.HC, and cs.AI.
40 results per query · 9 topic queries
Semantic Scholar
AI-powered academic search by the Allen Institute, filtered to Economics, Business, Sociology, Political Science, and Computer Science.
40 results per query · 9 topic queries
The pipeline runs daily at 4:00 AM ET with a 7-day rolling lookback. After deduplication, papers are enriched in parallel.
MCP server
Connect an AI assistant to The Commonplace through the Model Context Protocol. Read-only access to papers, claims, and semantic search.
Tools
search_papers
Full-text search with filters for theme, evidence strength, methods rigor, and paper type.
search_claims
Query graded claims by outcome category, direction, evidence weight, and effect type.
semantic_search
Vector similarity search over papers or claims using natural language.
get_paper
Full paper details: assessment, claims, relationships, and linked HuggingFace datasets.
Connection setup
The server uses Streamable HTTP transport with Bearer token authentication. Add this to your client configuration:
Claude Desktop (claude_desktop_config.json)
{
"mcpServers": {
"commonplace": {
"type": "streamable-http",
"url": "https://commonplace.workforcefutures.net/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}Claude Code (.claude/settings.json or project .mcp.json)
{
"mcpServers": {
"commonplace": {
"type": "streamable-http",
"url": "https://commonplace.workforcefutures.net/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}For an API key, contact Alex via workforcefutures.net.