Paid AI models outperform free counterparts at producing functional assistive-device software: Gemini Pro implemented 14 of 16 requested features with just 1.25 prompts on average, while free models produced at most four functions, suggesting subscription models materially improve nontechnical professionals' ability to generate usable code.

Main Finding

Paid large AI models substantially outperform free models at converting natural-language, nontechnical prompts into working assistive-device software. In this study Gemini Pro (Google) and ChatGPT Plus (GPT-5) produced far more functional features with fewer prompts than free alternatives, suggesting paid models are more effective, robust, and usable by nontechnical professionals designing low-cost personalized assistive products.

Key Points

• Study task: ask AI models (no technical input) to generate Python software that turns an arcade gamepad into an adapted, mouse-like controller through 16 progressively complex functions.
• Population of models: 7 total — 2 paid models (Gemini Pro, ChatGPT Plus) and 5 free models (including Gemini Free, DeepSeek).
• Performance summary:
  • Gemini Pro: 14 of 16 functions implemented; average 1.25 prompts per function (SD 0.45).
  • ChatGPT Plus (GPT-5): 11 of 16 functions implemented; average 1.31 prompts per function (SD 0.48).
  • Free models: between 0 and 4 functional outcomes; DeepSeek and Gemini Free were the best among free options.
• Qualitative advantage of paid models: superior contextual understanding, greater tolerance for natural-language phrasing, and reduced conversational drift.
• Human oversight and prompt-literacy remain necessary to ensure correctness, reliability, and ethical deployment.

Data & Methods

• Experimental design:
  • 16 standardized, progressively complex natural-language prompts requesting specific functional capabilities.
  • No technical input from researchers; prompts framed as nontechnical design requests to simulate use by health/education professionals.
  • Models were prompted and allowed to produce code; researchers recorded the number of functions successfully implemented and the average number of prompts required (with standard deviations reported for top performers).
  • No technical corrections or coding-level instruction were provided by researchers (interactions aimed to remain within a natural-language design process).
• Outcome measures:
  • Primary: count of implemented functions (out of 16).
  • Secondary: average number of prompts required to reach a working implementation per function.
• Sample limitations:
  • Single application domain (assistive controller) and single implementation language (Python).
  • Seven models tested — small model pool and snapshot in time.
  • “Success” defined by implemented functionality; nuance of code quality, robustness, security, or maintainability not deeply quantified here.

Implications for AI Economics

• Value of model quality and monetization:
  • Paid models show clear productivity returns for nontechnical users; this supports subscription/API pricing and justifies willingness to pay among professionals who gain time- and cost-savings.
  • Model quality translates into economic value beyond raw compute: better contextual understanding reduces iteration costs and human supervision.
• Distributional effects and access inequality:
  • Superior paid models can widen gaps between well-resourced organizations (who can afford subscriptions) and resource-constrained practitioners, potentially worsening the digital divide in assistive-technology development.
  • Public-sector and low-income providers may need subsidized access or public alternatives to capture these productivity gains.
• Labor and skill complementarities:
  • These models act as skill multipliers for nontechnical workers (health, education), reducing the need for specialist programmers for many custom assistive solutions — a form of task automation/augmentation that is skill-biased toward prompt-literacy rather than coding.
  • Investment in prompt-design training (human capital) is likely to have high returns for organizations adopting these tools.
• Market structure and competition:
  • Freemium models that maintain only limited capability risk failing to deliver real utility for complex applied tasks; this may concentrate value in better-funded incumbents unless open or competitive alternatives improve.
  • Demand for reliable, domain-specific performance may drive firms toward premium tiers or specialized enterprise offerings.
• Externalities and policy considerations:
  • Safety, reliability, and ethical deployment require oversight; poor outputs from free models could produce harms if used without checks.
  • Policymakers should consider subsidizing access for public-health/education providers, supporting open-model development, and funding evaluation frameworks for AI-generated assistive tech.
• Research and economic evaluation needs:
  • Cost–benefit analyses comparing subscription costs to labor/time saved in real-world deployments.
  • Broader, longitudinal studies across domains and larger model samples to quantify general equilibrium effects (labor demand, market entry for small providers, welfare gains from cheaper assistive tech).

Limitations to keep in mind: narrow domain, limited model set, and focus on immediate functional outputs rather than long-term reliability, maintenance costs, or user testing. Further work should evaluate real-world deployments, user outcomes, and the tradeoffs between paid-access productivity and public-access equity.