LLM assistants often convince developers they've met HIPAA non-functional requirements, yet expert-verified accuracy is low; verbose, information-dense replies hurt user satisfaction while proactive behavior helps it.

Accuracy and Satisfaction in Multi-Turn LLM Dialogues for NFR Assessment

Ali Pourghasemi Fatideh, Wilder Baldwin, Maria Dhakal, Collin McMillan, Sepideh Ghanavati · June 23, 2026

arxiv correlational medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Developers frequently accept LLM assessments of HIPAA-related non-functional requirements, but those assessments perform poorly against expert ground truth; longer and more information-heavy responses reduce user satisfaction while proactive agent turns increase it.

LLM-based dialogue assistants have become mainstream tools for software developers, yet current evaluation benchmarks focus exclusively on functional correctness. This leaves a critical gap in assessing the quality and accuracy of these conversations when handling Non-Functional Requirements (NFRs), which are inherently vague, context-dependent, and involve many parts of a program. Evaluating how well these systems support collaborative reasoning about NFRs requires methods that go beyond single-turn accuracy to capture both the correctness of the system's outputs and the quality of the multi-turn interaction. In this paper, we investigate the accuracy and quality of multi-turn conversations between developers and an LLM-based agent in the domain of Health Insurance Portability and Accountability Act (HIPAA) regulatory compliance. We hired 49 programmers to interact with GitHub Copilot to assess 148 HIPAA-derived NFRs against the iTrust codebase, a system designed to comply with HIPAA regulations, across three dimensions: requirement satisfaction level, reasoning, and code localization. We find that developers tend to agree with LLM assessments, but accuracy against expert ground truth is low. We model user satisfaction and find that longer system responses and more information-providing turns negatively affect user satisfaction, whereas proactive interactions positively affect it. Our findings provide insights for designing LLM-based dialogue systems that support NFR assessment.

Summary

Main Finding

LLM-based dialogue assistants (GitHub Copilot with gpt-5.1-codex-max) produce multi-turn assessments of regulatory NFRs that developers overwhelmingly rate as high-quality (91–94% agreement), yet these assessments are often inaccurate relative to expert ground truth (low F1/BERTScore). Dialogue features matter for satisfaction: verbose answers and many information-providing turns reduce user satisfaction, while proactive interaction turns increase it.

Key Points

Perceived quality vs accuracy
- Participant agreement: 91% (satisfaction level), 94% (reasoning), 94% (code location).
- Accuracy vs expert ground truth:
  - Requirement satisfaction level F1 = 0.381 (4-category task).
  - Reasoning BERTScore F1 = 0.520; cosine similarity = 0.774; ROUGE-1/2/L low.
  - Code location F1 = 0.203.
- Conclusion: responses feel convincing to developers but are frequently incorrect (risk of overtrust).
Dialogue characteristics & satisfaction (PARADISE-inspired modeling)
- Longer system responses and a higher count of information-providing turns → negative association with user satisfaction.
- More proactive interaction turns by the agent → positive association with satisfaction.
Data and corpus construction highlights
- Started from 650 HIPAA-derived statements → filtered to 180 → final 148 NFRs scoped to iTrust.
- Expert-ground-truth created via two-expert annotation and reconciliation across three dimensions: satisfaction level (Satisfied / Weakly Satisfied / Weakly Denied / Denied), reasoning text, and code locations (file + line ranges).
Study participants and dialogue stats
- Recruited 49 programmers; after quality filtering, 34 participants remained.
- 34 dialogues, 406 turns total; average ≈ 12 turns/dialogue.
- Avg words: prompts ≈ 61, responses ≈ 129 (per response), per-turn ≈ 190.
Tools, metrics, and annotations
- Chatbot via GitHub Copilot CLI using gpt-5.1-codex-max.
- Evaluation metrics: precision/recall/F1 (categorical), BERTScore/cosine/ROUGE (text), set-based F1 for code locations.
- Dialogue-turn labels drawn from PARADISE, MT-Eval, MT-Bench-101; annotation reached high inter-rater reliability (Cohen’s κ ≈ 0.81–0.83 after refinement).
Transparency
- Authors released dataset and tooling repository: https://github.com/sh3rLock3d/DialogueNFR

Data & Methods

NFR corpus: 148 HIPAA-derived non-functional requirements validated for relevance to iTrust (EHR codebase).
Ground truth:
- Two-expert adjudication process produced labels for three dimensions: satisfaction level (4-class), textual reasoning, and code localization (file + line numbers).
Human study:
- Participants performed multi-turn assessments using the Copilot agent, were instructed to first submit all NFRs in an initial prompt and then pursue follow-ups.
- Each NFR assigned to at least two developers; survey included attention checks and PARADISE-based satisfaction questionnaire.
Annotation & modeling:
- Turn-level annotation taxonomy built from PARADISE, MT-Eval, and MT-Bench-101 categories; iterative annotation achieved high κ.
- Accuracy measured by matching system final outputs to ground truth (precision/recall/F1), semantic similarity (BERTScore, cosine), and set overlap for code locations.
- User satisfaction modeled as a function of dialogue features (turn counts, types, response lengths, proactive behaviors).
Sample & scale:
- 148 NFRs, 34 high-quality participant dialogues, 406 turns; pilot validated task feasibility.

Implications for AI Economics

Productivity vs verification costs
- Surface-level productivity gains (developers accept LLM outputs) can be offset by error-correction and compliance remediation costs when outputs are inaccurate. Firms may realize time savings in the short term but incur downstream verification and liability costs.
Labor demand reallocation
- Increased use of LLM assistants for NFR assessment could shift labor demand from routine code-writing toward verification, auditing, and compliance-specialist roles (higher-skill, higher-wage tasks). Demand for human experts to validate LLM outputs for regulated domains will likely rise.
Market for verification & auditing tools
- Low accuracy on code localization and requirement satisfaction creates market opportunities for tools that (a) surface uncertainty/calibration signals, (b) automatically cross-check LLM assertions against code, or (c) provide lightweight auditing for regulatory compliance.
Risk externalities & regulation
- Overtrust in persuasive-but-incorrect LLM outputs introduces externalities: compliance failures can lead to fines, reputational damage, and systemic risk in regulated sectors (healthcare, finance). This strengthens the case for regulated validation standards or mandatory human-in-the-loop checks for high-stakes domains.
Value of interface & UX design for economic outcomes
- Dialogue behavior influences satisfaction and perceived trust. Designing for concise, proactive assistance (ask clarifying questions, make short targeted suggestions) may improve effective adoption and reduce costly missteps—affecting realized ROI of LLM integration.
Evaluation metrics & procurement decisions
- Policy- and procurement-oriented decisions should not rely solely on single-turn functional benchmarks. Economic valuations of LLM deployments must incorporate multi-turn accuracy, calibration, and the cost of false positives/negatives in domain-specific NFRs.
Insurance, liability, and contracting
- Firms deploying LLM-assisted workflows in compliance contexts may face higher insurance premiums or need contractual clauses allocating responsibility for AI-generated compliance assessments.
Research & investment priorities
- Investments that improve model calibration, uncertainty communication, and precise code-localization will have outsized economic returns in regulated industries compared to marginal gains in single-turn code generation metrics.

If you want, I can: - Extract the most economically salient quantitative figures into a one-page brief for executives. - Draft recommended product design interventions (e.g., uncertainty indicators, brevity priors, mandatory human validation checkpoints) with estimated cost/benefit considerations.

Assessment

Paper Typecorrelational Evidence Strengthmedium — Provides direct empirical measurements of LLM-assisted assessment accuracy against expert ground truth and models correlates of user satisfaction, but uses a modest sample (49 programmers, 148 NFR evaluations), a single codebase (iTrust) and a single LLM product (GitHub Copilot), without experimental randomization to identify causal effects, limiting external validity and causal claims. Methods Rigormedium — Study uses human subjects, real multi-turn interactions, and expert-labeled ground truth for evaluation and employs statistical modeling of satisfaction, but lacks randomized or controlled variation, has a relatively small and potentially non-representative participant pool, and focuses on one domain and one assistant implementation which constrains robustness checks and identification. Sample49 hired programmers interacted with GitHub Copilot to evaluate 148 HIPAA-derived non-functional requirements (NFRs) applied to the iTrust codebase; data comprise multi-turn dialogues, participant ratings on requirement satisfaction, reasoning and code localization labels, and expert ground-truth annotations for accuracy; measures of interaction features (response length, turn types, proactive agent behavior) were used to model user satisfaction. Themeshuman_ai_collab productivity GeneralizabilitySmall participant sample (49) may not represent broader developer populations (experience levels, industries)., Single LLM/product (GitHub Copilot) — results may not generalize to other models, versions, or UIs., Single codebase (iTrust) and domain (US HIPAA compliance) limit applicability to other software systems and regulatory contexts., Lab/paid-task setting may differ from real-world development workflows and incentives., Evaluated NFRs are HIPAA-derived; other NFR types (performance, reliability) may yield different results.

Claims (9)

Claim	Direction	Outcome	Confidence & Evidence	Details
LLM-based dialogue assistants have become mainstream tools for software developers, yet current evaluation benchmarks focus exclusively on functional correctness. Ai Safety And Ethics	null_result	scope of evaluation benchmarks (functional correctness focus)	Reading fidelity high Study strength low	not reported 0.15
Non-Functional Requirements (NFRs) are inherently vague, context-dependent, and involve many parts of a program, making them difficult to assess with single-turn correctness benchmarks. Other	null_result	characteristics of NFRs (vagueness, context-dependence, broad code impact)	Reading fidelity high Study strength low	not reported 0.15
Evaluating how well LLM-based dialogue systems support collaborative reasoning about NFRs requires methods that go beyond single-turn accuracy to capture both the correctness of system outputs and the quality of multi-turn interaction. Ai Safety And Ethics	null_result	evaluation methodology adequacy for NFR assessment	Reading fidelity high Study strength speculative	not reported 0.05
We hired 49 programmers to interact with GitHub Copilot to assess 148 HIPAA-derived NFRs against the iTrust codebase across three dimensions: requirement satisfaction level, reasoning, and code localization. Other	null_result	study sample and experimental setup	Reading fidelity high Study strength high	n=49 0.5
Developers tend to agree with LLM assessments of NFRs. Decision Quality	positive	agreement between developers and LLM assessments	Reading fidelity high Study strength medium	n=148 0.3
Accuracy of developer+LLM assessments against expert ground truth is low. Decision Quality	negative	accuracy of assessments relative to expert ground truth	Reading fidelity high Study strength medium	n=148 0.3
Longer system responses and more information-providing turns negatively affect user satisfaction. Worker Satisfaction	negative	user satisfaction	Reading fidelity high Study strength medium	n=49 0.3
Proactive interactions (by the system) positively affect user satisfaction. Worker Satisfaction	positive	user satisfaction	Reading fidelity high Study strength medium	n=49 0.3
Our findings provide insights for designing LLM-based dialogue systems that support NFR assessment. Ai Safety And Ethics	positive	design guidance for LLM dialogue systems	Reading fidelity high Study strength speculative	not reported 0.05