Vibe coding speeds up visualization prototyping but stumbles on polish: an empirical study finds large language model–driven generation cuts initial development time but struggles with detailed interactions, state coordination and reliable mappings, turning developer work into a costly loop of semantic alignment and manual fixes.
Towards Effective Generation of Interactive Visualizations with Vibe Coding: An Empirical Study
A mixed-methods study of 78 CS students shows vibe coding accelerates prototyping of interactive visualizations but often fails on fine-grained interactions, multi-view consistency, and code reliability, shifting developer effort toward prompt engineering and verification.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
No provider observation is available for this paper.
Missing data, not a zero citation count.
Constructing interactive visualizations has traditionally required substantial human effort, involving both technical implementation and design decision-making. Recently, vibe coding, a programming paradigm leveraging Large Language Models to generate, interpret, and refactor code from natural language specifications, has emerged as a promising approach to reduce the burden. However, the capabilities and limitations of vibe coding in building interactive visualizations remain unexplored. To address this gap, we conducted a user study with 78 participants that were tasked with constructing interactive visualizations using vibe coding. We further collected users feedback through questionnaires, interviews, and case analyses. Based on this study, we examine (1) the capabilities and (2) user experience of vibe coding in generating interactive visualizations, and (3) the practical human-agent collaboration strategies adopted. Our findings provide the first systematic assessment of vibe coding for interactive visualization construction, revealing both its strengths and limitations, explaining the shift in developer labor and identifying the hybrid collaboration strategies participants adopted. Furthermore, our study offers insights for more intuitive and robust vibe coding practices.
Summary
Main Finding
Vibe coding (LLM-assisted code generation from natural language) substantially speeds prototyping and initial delivery of interactive visualizations but falls short on fine-grained interaction design, multi-view/system-level consistency, and code reliability. The net effect is a shift in developer labor from manual coding toward semantic alignment, prompting, validation, and selective manual intervention, producing a hybrid human–AI workflow rather than full automation.
Key Points
- Productivity and limits
- Strong gains for rapid prototyping and basic charts: participants reported faster initial delivery and framework setup.
- Performance degrades with complexity: one-shot success rates fall from 92.3% (static charts) to 19.2% (multi-view coordination).
- Generated code is often functional but requires manual review and refinement (many reported confusing interaction logic, state-synchronization bugs, and suboptimal visual mapping).
- User experience
- Major cognitive friction arises from intent expression and semantic alignment: ~43.6% cited semantic understanding difficulties; 37.2% worried about re-prompting introducing new biases.
- Iterative prompting and checking cause fatigue; developers frequently switch between dialog and visual previews (25.6% reported this switching cost).
- Developers lack predictable intermediary feedback linking natural-language instructions to visual outcomes.
- Collaboration strategies
- Hybrid workflows dominate: 61.5% decomposed tasks and added features incrementally; 59.0% provided sample data; 32.1% used reference diagrams.
- Manual edits remain common: >59% needed to modify ~10–30% of generated code; only 28.2% felt only minor adjustments were necessary.
- Decision heuristics: marginal-effort tradeoffs, modification granularity, and cognitive-load considerations guide whether to manual-edit or re-prompt the agent.
- Representative cases
- Failure mode: agents may violate visualization principles (e.g., local vs global color normalization), producing misleading encodings.
- Success mode: agent can act as design catalyst (suggesting alternative encodings like dumbbell charts) when coupled with human guidance.
Data & Methods
- Participants: 78 university students (53 male, 25 female) with CS backgrounds and prior vibe-coding experience; LLMs used included Codex, Claude Code, Cursor, GitHub Copilot.
- Procedure:
- 1-week interactive visualization construction task using vibe coding (participants chose topics/datasets).
- Post-task questionnaire (20 minutes) covering delivery efficiency, code reliability, intent alignment, cognitive load (5-point Likert items).
- Semi-structured interviews (~25 minutes) for a subset of participants.
- Measurements and analysis:
- Quantitative: descriptive stats (mean, SD, median, IQR) on Likert items (e.g., Q1–Q21 covering data encoding, interactions, bugs, trust).
- Qualitative: thematic coding of open responses and interviews (initially via Google Gemini, then manual review).
- Select quantitative findings:
- One-shot success rates by task: Static charts 92.3% (72/78), Basic interactions 84.6% (66/78), View transformations 60.3% (47/78), Dynamic animations 30.8% (24/78), Multi-view coordination 19.2% (15/78).
- Common practitioner tactics include incremental prompting (61.5%), sample data (59.0%), and manual edits (many needed to change 10–30% of code).
Implications for AI Economics
- Partial automation and task polarization
- Vibe coding automates routine, well-specified visual-coding tasks (prototyping, simple charts) but not complex/systemic tasks (multi-view coordination, nuanced interaction logic). Expect increased automation pressure on lower-complexity visualization tasks while preserving or increasing demand for skilled work in complex design, verification, and orchestration.
- Labor reallocation and complementarities
- Worker roles will shift from line-by-line implementation toward higher-level semantic tasks: prompt engineering, specification writing, validation, debugging, and design judgment. These are complementary to LLMs and likely to be higher-skill, higher-value activities.
- Firms may reassign or upskill developers to roles emphasizing human-in-the-loop control, visual design expertise, and AI orchestration, rather than pure coding.
- Productivity vs. verification costs
- Though overall delivery speed improves, the need for iterative prompting, manual corrections, and verification imposes non-trivial cognitive and time costs. Economic gains from faster prototyping may be partially offset by increased QA, review, and oversight labor—raising the effective cost per deployable, production-quality visualization.
- Wage and skill-premium effects
- Demand for workers who can effectively coordinate with LLMs (prompting, rapid specification, debugging generated code) will increase, potentially increasing wage premia for these skills. Conversely, purely routine coding tasks may face downward wage pressure.
- Market structure and service opportunities
- New markets for hybrid toolchains and services (visualization verification, AI-assisted QA, visual-version-control, domain-aware LLMs) are likely to grow. Vendors offering better intermediary feedback (previews, state tracing, shared-state management) will capture value.
- Product differentiation and homogenization risk
- The study notes homogenization risk: default agent outputs can converge on generic templates, reducing diversity and differentiation in visual design unless human intervention or specialized models are used. This can compress product differentiation in low-end visualization services, pushing premium value to bespoke, human-driven solutions.
- Investment and R&D priorities
- High economic returns are likely for investments that reduce verification costs and semantic-alignment friction (tools for state-aware code generation, visualization-principle-aware LLMs, visual preview + language binding). Policies or investments that improve LLMs’ domain sensitivity (e.g., preserving global normalization) will increase effective automation.
- Policy and governance considerations
- Given reliability concerns (misleading encodings), industries using automated visualization pipelines (journalism, public dashboards, policy analytics) may need standards, certification, or audit trails to manage misinformation risk—creating regulatory compliance costs and new auditing service markets.
- Aggregate adoption dynamics
- Adoption will be heterogeneous: small teams and novices benefit most from prototyping speed; large organizations will invest in hybrid toolchains and governance to manage verification and consistency across systems. Overall, vibe coding will raise aggregate productivity in visualization creation but with important distributional and institutional effects due to required complementary investments (skills, QA, tooling).
If you want, I can: - Draft a short economic model/sketch (task automation + verification cost) to quantify potential labor reallocation and welfare impacts. - Produce a slide-ready summary highlighting the top economic implications for managers and policymakers.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — Mixed-methods user study with a reasonably large sample (N=78) and both quantitative (Likert) and qualitative (interviews, cases) data provides informative empirical evidence about capabilities and user experience, but the study lacks experimental controls, objective performance metrics, or causal identification; results rely heavily on self-report and a non-representative, self-selected student sample.
Methods Rigormedium — Study uses standard HCI methods (task-based lab/remote work over one week, post-task questionnaires, semi-structured interviews, thematic coding), reports descriptive statistics and qualitative coding; however, there is no control/comparison group, tasks and datasets were self-selected (reducing comparability), outcome measures are primarily subjective, and potential selection and confirmation biases (experienced users, single university) are not addressed.
Sample78 university students with computer-science backgrounds (53 male, 25 female) experienced with interactive visualizations and 'vibe coding'; 21 relied mainly on LLMs, 54 used LLMs as assistants, 3 used LLMs for troubleshooting; commonly used models/tools included Codex, Claude Code, Cursor, GitHub Copilot; participants were given one week to build an interactive visualization of their choice and completed post-task Likert questionnaires and optional semi-structured interviews.
Themeshuman_ai_collab productivity
GeneralizabilitySample is university students with CS backgrounds—limits transferability to professional developers or non-technical users, Participants had prior experience with LLMs/vibe coding, biasing results toward more favorable/efficient use, Tasks and datasets were self-chosen, producing heterogeneity that prevents standardized performance comparison, Measures are largely self-reported (Likert scales) and qualitative, lacking objective time-on-task or productivity metrics, Single-institution recruitment and unspecified geographic/cultural context limit broader applicability, Rapidly evolving LLM models/tools mean findings may not generalize to newer models or toolchains
Claims (13)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Vibe coding accelerated the delivery of interactive visualization systems, particularly the creation of initial prototypes, according to participants' self-reports. Task Completion Time | positive | Perceived time required to create prototypes and complete final delivery |
Reading fidelity
high
Study strength
low
|
n=78
|
| Vibe coding enabled rapid setup of basic frameworks and charts during from-scratch development, as reported by 35.9% of participants. Task Completion Time | positive | Rapid setup of basic visualization frameworks and charts |
Reading fidelity
high
Study strength
low
|
n=78
35.9%
|
| One-shot generation success was highest for static charts and basic interactions, and declined for view transformations, dynamic animations, and multi-view coordination. Output Quality | mixed | Successful one-shot generation of visualization features |
Reading fidelity
high
Study strength
medium
|
n=78
92.3% for static charts; 84.6% for basic interactions; 60.3% for view transformations; 30.8% for dynamic animations; 19.2% for multi-view coordination
|
| Participants did not consider vibe-coded output stable enough to be delivered directly without manual review. Error Rate | negative | Perceived code stability and readiness for direct delivery |
Reading fidelity
high
Study strength
low
|
n=78
|
| Confusing interaction logic was the most frequently reported implementation problem, cited by 25.6% of participants; mismatched state synchronization and improper visual mapping were each cited by 12.8%. Error Rate | negative | Frequency of reported interaction, state-synchronization, and visual-mapping problems |
Reading fidelity
high
Study strength
low
|
n=78
25.6% confusing interaction logic; 12.8% mismatched state synchronization; 12.8% improper visual mapping
|
| Semantic-understanding discrepancies and high barriers to prompt construction were the most frequently cited difficulties in using vibe coding, reported by 43.6% of participants. Worker Satisfaction | negative | Difficulty of expressing intent and achieving semantic alignment with the agent |
Reading fidelity
high
Study strength
low
|
n=78
43.6%
|
| Modifying prompts was perceived as potentially introducing new biases and reducing developers' sense of control by 37.2% of participants. Worker Satisfaction | negative | Perceived control over development and unintended changes after prompt modification |
Reading fidelity
high
Study strength
low
|
n=78
37.2%
|
| The interaction cost of repeatedly switching between the agent dialog and the visual-interface window increased cognitive load, according to 25.6% of participants. Worker Satisfaction | negative | Cognitive load associated with observing and validating interface changes |
Reading fidelity
high
Study strength
low
|
n=78
25.6%
|
| Participants commonly decomposed visualization tasks into steps and incrementally added features; 61.5% reported using this strategy. Task Allocation | positive | Use of incremental task decomposition during AI-assisted development |
Reading fidelity
high
Study strength
low
|
n=78
61.5%
|
| Providing sample data was a common collaboration strategy, used by 59.0% of participants to help the agent understand data structure. Task Allocation | positive | Use of sample data to communicate data structure to the agent |
Reading fidelity
high
Study strength
low
|
n=78
59.0%
|
| When asking the agent to modify existing visualization code, 59.0% of participants still manually modified 10%–30% of the code. Task Allocation | mixed | Share of code requiring manual modification after agent-assisted changes |
Reading fidelity
high
Study strength
low
|
n=78
59.0% manually modified 10%–30% of the code
|
| Only 28.2% of participants believed that agent-assisted modification could work with minor adjustments. Task Allocation | negative | Perceived need for manual correction after agent-assisted code modification |
Reading fidelity
high
Study strength
low
|
n=78
28.2%
|
| Participants selected between manual editing and agent-guided modification based on marginal efficiency cost (42.3%), modification granularity (38.5%), and understanding or cognitive-load cost (26.9%). Task Allocation | mixed | Criteria used to allocate code-modification work between humans and the agent |
Reading fidelity
high
Study strength
low
|
n=78
42.3% marginal efficiency cost; 38.5% modification granularity; 26.9% understanding and cognitive load
|