Generative AI tools routinely collapse or rot contextual cues in workplace use, forcing users into ad hoc fixes; the paper argues firms should stop indiscriminately hoarding context data and instead design interactional practices that keep AI systems aligned with users' real-world contexts.

Main Finding

Generative AI systems fail to reliably account for the social and professional contexts in which expert users operate. Attempts by developers to solve this by collecting ever more context data (larger context windows, RAG, metadata, user histories) produce limited gains and introduce new frictions (privacy, bias, “context rot” and “context collapse”). As a result, professionals spend extra effort (prompting, verification, workflow workarounds) to make outputs usable, reducing the promised productivity gains and slowing adoption. The authors argue for a shift away from indiscriminate data accumulation toward interactional design practices that embed GenAI into users’ actual contexts of use (what they call context engineering done through interaction, not just data aggregation).

Key Points

• Definition gaps: “Context” means different things to model developers (tokens, context windows, retrieved documents, metadata) and to professionals (projects, clients, deliverables, tacit procedures, social relationships). This mismatch produces systematic failures.
• Failure modes observed: generic outputs, persistent hallucinations, off-topic responses, and bias toward common representations that omit niche, domain-specific needs.
• Developers’ current technical responses: expand context windows, retrieval-augmented generation (RAG), collect user metadata and histories, and append documents to prompts. These are treated as a route to better domain fit and a competitive “moat.”
• Limits of the data-centric approach:
  • Multiple distinct contexts collapse together (context collapse) or degrade over time (context rot), reducing relevance.
  • Accumulating more data raises privacy, IP, and legal exposures and environmental/computational costs.
  • Over-reliance on synthetic outputs risks “model collapse” where subsequent training degrades fidelity.
• Users’ compensatory behaviors:
  • Prompt engineering and careful seeding of context.
  • Manual verification, multi-step workflows, and repeated clarification with the model.
  • Tactical avoidance of certain tasks for which AI is unreliable.
• Recommended conceptual shift: move from indiscriminate collection of context-relevant data to interactional practices that solicit and manage the precise, situational context needed for a task (e.g., ephemeral, project-scoped context, user-in-the-loop clarifying questions, workflows that integrate human expertise and checks).

Data & Methods

• Study design: qualitative, semi-structured interviews plus demonstration/prompting exercises.
• Sample: n = 15 professionals recruited from the U.S./Canada who regularly use GenAI in their work. Participants spanned domains such as medicine, law, software and semiconductor design, 3D and graphic design, animation, music production, accounting, and research.
• Recruitment: screened respondents (n = 235) and selected participants to optimize domain diversity and frequency of AI use.
• Data collection:
  • Interviews with three sections: background/use patterns; open-ended exploration of interactions and frustrations; task demonstrations and prompting exercises (11/15 participants shared demos).
  • Collected 29 real prompt descriptions and 22 descriptions of prompting approaches.
• Analysis: grounded theory with iterative coding (open → axial → selective), producing a codebook and higher-order concepts from interview transcripts (ASR-corrected).
• Limitations noted by authors: small, self-selected sample; geographic limitation (U.S./Canada); gender and demographic skew; qualitative design limits statistical generalizability—but yields rich, practice-oriented insights.

Implications for AI Economics

• Realized productivity gains may be substantially lower than headline claims:
  • Time/effort spent on verification, context-setting, and workflow adaptation offsets efficiency benefits; measured ROI will be lower and adoption slower.
• Platform competition and switching costs:
  • “Context moats” based on vast user-specific data may be less valuable if that data poorly captures actionable, project-specific context. Vendors that support interactional context capture and ephemeral/project-scoped context may win users despite smaller historical data stores.
• Labor-market effects:
  • Increased demand for verification, curation, and prompt-engineering work—complementary tasks that preserve expert gatekeeping rather than fully automating work.
  • Need for upskilling: workers must learn to integrate GenAI safely and productively into workflows, changing task composition rather than eliminating roles immediately.
• Cost structure and investment implications:
  • Building systems that indiscriminately collect and store context is costly (privacy compliance, storage, compute, legal risk). Investment may shift toward UX/interaction design, integration with collaboration tools, and human-in-the-loop systems that deliver more usable outputs per unit of data.
• Product-market opportunities:
  • Tools that implement interactional context engineering (clarifying dialog, ephemeral project context, structured capture of tacit workflow rules) have potential market value.
  • Domain-specific tooling that combines minimal, targeted data collection with high-quality interactional capture may yield higher effective productivity than large monolithic models alone.
• Measurement & policy implications:
  • Productivity studies should account for secondary labor (editing/verification) and context-management effort to avoid overstating AI gains.
  • Policymakers and firms should weigh privacy/regulatory costs of aggressive context collection against modest improvements in output relevance.
• Long-term model risks affecting economic value:
  • Model collapse and degraded training fidelity from synthetic feedback loops can reduce long-term model reliability, increasing maintenance costs and potentially reducing trust and adoption across firms.

Overall, the paper implies a reorientation of commercial and research priorities: rather than investing primarily in larger data troves and broader context windows, firms and investors should prioritize interactional design, human-in-the-loop workflows, and targeted context capture to unlock real productivity improvements and sustainable economic value from GenAI in the workplace.