Understanding Human-AI Collaboration in Cybersecurity Competitions

Capture-the-Flag (CTF) competitions are increasingly becoming a testbed for evaluating AI capabilities at solving security tasks, due to the controlled environments and objective success criteria. Existing evaluations have focused on how successful AI is at solving CTF challenges in isolation from human CTF players. As AI usage increases in both academic and industrial settings, it is equally likely that human players may collaborate with AI agents to solve challenges. This possibility exposes a key knowledge gap: how do humans perceive AI CTF assistance; when assistance is provided, how do they collaborate and is it effective with respect to human performance; how do humans assisted by AI compare to the performance of fully autonomous AI agents on the same challenges. We address this gap with the first empirical study of AI assistance in a live, onsite CTF. In a study with 41 participants, we qualitatively study (i) how participants'perception, trust, and expectations shift before versus after hands-on AI use, and (ii) how participants collaborate with an instrumented AI agent. Moreover, we also (iii) benchmark four autonomous AI agents on the same fresh challenge set to compare outcomes with human teams and analyze agent trajectories. We find that, as the competition progresses, teams increasingly delegate larger subtasks to the AI, giving it more agency. Interestingly, CTF challenges solving rates are often constrained not by model's reasoning capabilities, but rather by the human players: ineffective prompting and poor context specification become the primary bottleneck. Remarkably, autonomous agents that self-direct their prompting and tool use bypass this bottleneck and outperform most human teams, coming in second overall in the competition. We conclude with implications for the future design of CTF challenges and for building effective human-in-the-loop AI systems for security.

Summary

Main Finding

Human-AI collaboration in live CTFs shows substantial potential but is primarily limited by human-side interaction quality (prompting, context specification, verification) rather than model reasoning alone. As competitions progress, humans increasingly delegate larger subtasks to AI; when humans provide high-quality prompts and technical steering, AI assistance raises performance substantially. Fully autonomous agents that self-direct prompting and tool use can bypass the human bottleneck and outperform most human teams (one agent reached 4,900 points — 2nd among top-10 human teams) at modest API cost (~$96), but agent performance depends strongly on agent architecture and backbone model capability.

Key Points

Study design and scale
- Live, in-person university-level CTF with 17 newly authored challenges (total 7,700 points) across forensics, crypto, reverse, web, etc.
- 95 people participated in the CTF; 41 enrolled in the human-AI study (recorded pre/post surveys and use of the instrumented assistant).
- Recorded 2,299 chat messages across 168 chat logs from 38 participants using the assistant (CTFriend).
- Parallel autonomous-agent evaluation: four agent frameworks × three Claude-family models = 12 configurations on the same fresh challenge set.
RQ1 — Perception, trust, and expertise
- After hands-on use, participants’ expectations and trust in AI decreased (F1). Common complaints: flawed reasoning, hallucinations, non-working code, and guardrail-related failures (F2).
- Participants rated AI as clear in understanding and explanation but lacking in complete, actionable solutions (F3).
- AI expertise partially substitutes for CTF domain expertise: participants with low domain but high AI skill achieved competitive scores, while intermediate domain but low AI skill performed poorly (F4–F5).
RQ2 — Human-AI collaboration dynamics
- Interaction patterns shift under time pressure: many teams moved toward end-to-end delegation to the assistant rather than incremental co-piloting (F6).
- Effective collaboration correlates with high-quality, technical prompts and prompt-engineering strategies; novices often used low-quality prompts, engaged in repetitive “answer shopping,” and suffered more failures (F7–F8).
- Some low-domain-expertise users successfully used the AI as a learning scaffold to solve challenges they otherwise could not (F9).
RQ3 — Autonomous agents vs humans
- Advanced autonomous agents that combine long-horizon planning and robust tool support perform best; restricted toolsets or fixed wrappers lead to early plateaus (F11–F12).
- Backbone model capability sets the performance ceiling; with weaker models, different frameworks converge to low performance (F13).
- Some tasks are easier for agents (e.g., high-throughput, self-driven prompting) while others remain easier for humans (e.g., intensive environment interaction); this complementarity motivates hybrid workflows (“pair hacking”) where agents run autonomously and humans provide sparse steering/verification (F14–F15).
- Best agent achieved 4,900 points at ~$96.32 API cost and used roughly 1/5 of the cumulative runtime compared to humans.

Data & Methods

Experimental setting
- Live 24-hour, onsite CTF (university-level, IRB-approved). Participants watched a 2-minute assistant onboarding video, used the CTFriend assistant during the 24h competition, and completed pre/post surveys.
- Participant demographics: N=41 study participants (82.9% undergraduates; 56.1% CS majors). Self-reported CTF expertise and AI expertise were collected and binned.
Human-side analysis
- Surveys: pre- and post-competition surveys measuring expectations, perceived usefulness, trust, error types, and intent to reuse AI.
- Interaction logs: 2,299 human-AI messages analyzed qualitatively to extract interaction patterns, failure modes, delegation dynamics, and prompt strategies.
Autonomous-agent benchmarking
- Evaluated 12 agent configurations (4 frameworks × 3 Claude-family models) on the same 17-challenge set used in the live event.
- Measured total points earned, runtime, and API cost; analyzed agent trajectories and failure modes (e.g., tool use limits, environment-interaction brittleness).
- Compared agent scores and resource use against human teams’ results.
Limitations noted by authors
- Single-event university-level CTF; challenge design and participant pool limit generalizability to other domains and competition formats.
- Some participant reports of errors may reflect misunderstanding of model behavior rather than model failure; autonomous agents did not always encounter identical issues.

Implications for AI Economics

Productivity and cost-efficiency
- Autonomous agents can deliver high throughput at low marginal cost (example: 4,900 points for ~$96), indicating strong cost-effectiveness for certain security tasks. This suggests potential labor substitution in routine, high-volume parts of security workflows.
- However, agent performance is sensitive to architecture and model capability; investments in better backbone models and richer tool integration yield non-linear returns.
Skill-biased impacts and returns to AI expertise
- Value accrues to workers who possess AI-use skills (prompt engineering, tool orchestration). The study shows AI expertise can substitute for domain experience to a meaningful extent — implying wage premiums for AI-skilled security workers and returns to training programs focused on AI tooling.
- Poor human prompting is a primary bottleneck; economic returns to training (e.g., prompt-engineering certification, tooling UX improvements) may be high.
Complementarity vs substitution
- Tasks that require sparse human judgment, verification, or complex environment interactions remain complementary: hybrid workflows (agent doing throughput work; humans providing verification/steering) appear economically efficient. Organizations should optimize staffing to pair fewer expert humans with automated agents for scale.
- Competitive pressures in time-sensitive settings drive over-delegation to AI, increasing reliance on agent reliability; this raises operational risk and potential costs from errors (time wasted, false positives, security oversights).
Product design and market implications
- Tooling and interface design that reduce the human prompting burden (better context capture, templates, structured prompts, higher-level abstractions) will have strong economic value.
- Markets will favor agent platforms that combine long-horizon planning, robust tool integrations, and high-capacity models. Conversely, limited wrappers or fixed tool sets will underperform and lose market share.
Regulatory, safety, and compliance costs
- Guardrails and content restrictions introduced friction in competition settings; in real-world security work there will be trade-offs between safety/compliance and productivity. Firms must budget for these operational safety costs and potential delays introduced by model constraints.
Evaluation and incentives
- CTFs and benchmarks should evolve to measure human-AI teams, not only autonomous agents, to capture real-world economics of labor+AI. Leaderboards and procurement decisions will need to account for combined productivity, verification overhead, and downstream risk mitigation costs.

Overall, the paper indicates that AI can be economically transformative in cybersecurity tasks, but the realized gains depend on human-AI interface quality, worker AI skills, agent design, and appropriate human oversight. Investments into tooling, training, and hybrid workflows are likely to yield outsized economic returns.

Assessment

Paper Typedescriptive Evidence Strengthmedium — Head-to-head, contemporaneous measurement of human teams and multiple autonomous agents on the same fresh CTF challenge set provides direct, concrete evidence about comparative performance in that task domain; however, the sample is small (41 participants), non-random, task-specific (CTF puzzles), and short-term, limiting the strength of any broader causal claims about automation or labor displacement. Methods Rigormedium — The study uses real-world, instrumented behavioral data and objective outcome metrics (solve rates, rankings), plus systematic benchmarking of four autonomous agents and qualitative analysis of failure modes; but it lacks randomization or experimental manipulation, has limited sample size and participant selection information, and appears vulnerable to confounders (skill heterogeneity, novelty effects, contest incentives) and limited reproducibility without more detail on agent implementations and challenge selection. SampleLive, onsite Capture-the-Flag competition with 41 human competitors/teams using an instrumented AI assistant on a fresh set of challenges; four autonomous AI agents were run on the same challenge set for benchmarking; data include challenge solve rates, team/agent rankings, instrumented logs of AI interactions and prompting behavior, and qualitative observations/interviews about perceptions and trust. Themeshuman_ai_collab productivity skills_training GeneralizabilitySmall sample size (41 participants) limits statistical power and representativeness, Contest CTF tasks are narrow and stylized proxies for cybersecurity work and may not reflect operational, long-duration security tasks, Participants likely self-selected and may be more skilled or motivated than average cybersecurity workers, Short-term, single-event setting—novelty and learning effects may bias behavior and outcomes, Only four autonomous agent architectures/tools tested—results may not generalize to other models, toolchains, or deployment contexts, Onsite/lab environment differs from distributed, real-world workflows and organizational constraints, Outcome metrics (challenge solves/rankings) capture performance in puzzle-like tasks but not supervision, verification, or cost of false positives/negatives in real operations

Claims (10)

Claim	Direction	Outcome	Confidence & Evidence	Details
In a live onsite Capture-the-Flag (CTF) study (41 participants), human teams increasingly delegated larger subtasks to an instrumented AI as the competition progressed. Task Allocation	positive	degree/size of subtasks delegated to the AI over time (delegation rate and subtask size)	Reading fidelity high Study strength medium	n=41 teams delegated increasingly larger subtasks to AI over time 0.18
Human limits—specifically ineffective prompting and poor context specification—became the primary bottleneck to solving challenges, rather than model reasoning capability. Decision Quality	negative	attribution of challenge-solve failures to prompting/context issues versus model reasoning errors	Reading fidelity medium Study strength medium	n=41 failure-mode attribution: prompting/context issues were primary bottleneck rather than model reasoning 0.11
Four autonomous agents were benchmarked on the same fresh CTF challenge set alongside human teams. Team Performance	null_result	agent performance metrics on the fresh CTF challenge set (success rates, trajectories, tool use)	Reading fidelity high Study strength medium	benchmarking of four autonomous agents on the same fresh CTF challenge set 0.18
Self-directed autonomous agents (those that autonomously generated prompts and selected tools) bypassed human prompting failures and outperformed most human teams on the challenge set. Team Performance	positive	challenge solving rates and relative rankings of self-directed agents versus human teams	Reading fidelity medium Study strength medium	n=41 self-directed agents outperformed most human teams (bypassing prompting failures) 0.11
One autonomous agent finished second overall on the fresh challenge set. Team Performance	positive	overall ranking (2nd place) on the challenge set	Reading fidelity high Study strength medium	n=41 one autonomous agent finished 2nd overall 0.18
As the competition progressed, teams relied more on the AI for larger subtasks (increasing delegation and reliance). Task Allocation	positive	frequency of delegation and average scope/complexity of delegated tasks over competition time	Reading fidelity high Study strength medium	n=41 increasing frequency and scope of delegated tasks as competition progressed 0.18
Participants’ perceptions, trust, and expectations about the AI shifted after hands-on use (qualitative observation). Worker Satisfaction	mixed	qualitative changes in participant perceptions, trust, and expectations after hands-on AI usage	Reading fidelity medium Study strength medium	n=41 qualitative shifts in participant perceptions, trust, and expectations after hands-on use 0.11
Primary failure mode for human–AI teams was poor human prompting/insufficient context specification rather than deficiencies in the model's reasoning. Error Rate	negative	proportion of failed attempts attributable to human prompting/context issues vs. model reasoning failures	Reading fidelity medium Study strength medium	n=41 primary failure mode attributed to poor prompting/insufficient context vs model reasoning 0.11
The study is the first empirical investigation of human–AI assistance in a live CTF setting with a direct comparison to autonomous AI agents on the same fresh challenges. Other	null_result	novelty claim (existence of prior comparable live CTF human–AI empirical studies and direct agent comparisons)	Reading fidelity medium Study strength medium	n=41 author-stated novelty claim: first live CTF human–AI empirical investigation with agent comparisons 0.11
Practical takeaway: effectiveness of human–AI teaming in security tasks depends heavily on human ability to formulate context-rich prompts; autonomous workflows that self-manage prompting and tool selection can be more effective. Team Performance	mixed	relative effectiveness (challenge solve rates/rankings) conditional on human prompt quality versus agent self-directed prompting	Reading fidelity medium Study strength medium	n=41 effectiveness depends on human prompt quality; autonomous self-managing prompting can be more effective 0.11

In a live capture‑the‑flag contest, autonomous agents that self‑direct prompting and tool use beat most human teams, exposing human prompting and context specification as the key bottleneck to effective human–AI collaboration.