A GenAI multi-agent system at Germany’s Federal Employment Agency automates IT change requests into structured development tickets, cutting routine clerical work and producing more context-aware entries than legacy automation. The solution runs on-premises to meet strict data-security requirements, showing a path for public-sector IT modernisation—though evidence is from a single-case deployment.

Main Finding

A locally hosted, LLM-based multi-agent system (Reader, Planner, Creator, plus a developing Reviewer) can be safely integrated into a high-security public-administration environment to convert unstructured IT change requests (RfCs / user stories) into structured Jira development tickets. The prototype—orchestrated with CrewAI and powered by on-premise models (Aleph Alpha, LLaMA, Mistral) on a 4×A100 GPU cluster—produced plausible, usable tickets for medium-complexity requests, reduced routine work, and demonstrated advantages over traditional rule-based automation in flexibility, context-awareness, and maintainability, while preserving human-in-the-loop governance and strict data protection.

Key Points

• System architecture
  • Four specialized agents: Reader (extract info), Planner (decompose into work items), Creator (format Jira ticket), Reviewer (consistency/duplication/completeness; in development).
  • Agents orchestrated via CrewAI; Creator posts tickets to Jira using REST API.
  • Human-in-the-loop remains for final approval; Reviewer feedback loop planned to improve outputs.
• Deployment and security
  • Entirely on-premise to meet data protection and regulatory constraints.
  • Local inference on BA’s GPU cluster (4×A100 80GB); supports up to ~16k token inputs with token-management strategies.
• Prompting and language strategy
  • Persona prompting (role simulation) improved Planner consistency.
  • English prompts aided task decomposition; German prompts preserved domain terminology—tradeoffs led to mixed strategy.
• Evaluation
  • Test set: 50 authentic RfCs spanning low/medium/high complexity.
  • Judged by three domain experts (avg. 12 years’ experience); inter-rater agreement Cohen’s kappa = 0.81.
  • Results: high robustness and plausibility for low/medium complexity; very long or ambiguous documents sometimes required manual post-processing.
• Comparison to traditional automation
  • Outperformed RPA-like approaches on semi-structured natural language tasks via contextual reasoning, iterative decomposition, and multi-agent dialogue.
  • Promised better long-term maintainability because of adaptability, but requires ongoing model governance and human oversight.

Data & Methods

• Research design
  • Pragmatic design-science case study co-developed with domain experts and IT staff at the German Federal Employment Agency (BA).
  • Iterative build–evaluate–refine cycles with authentic operational constraints.
• Data
  • Real RfC and user-story documents from ALLEGRO system (varying lengths/formats/domains).
  • Jira ticket skeleton and formatting rules used as structured context.
• Implementation details
  • Multi-agent orchestration: CrewAI framework.
  • Local LLMs: Aleph Alpha, LLaMA, Mistral (no external API calls).
  • Hardware: on-premise GPU inference: 4×NVIDIA A100 80GB.
  • Input preprocessing: token trimming, contextual filtering for long documents.
  • Jira integration via REST API with full logging and traceability.
• Evaluation procedure
  • Generated tickets segmented into Title (ID + descriptive text), Goal, and Tasks to permit granular scoring.
  • Weighted scoring: title emphasis on descriptive clarity; goal/tasks split between formatting and content.
  • Human assessment by three domain experts; Cohen’s kappa = 0.81 indicates strong inter-rater reliability.
• Limitations of the evaluation
  • N = 50 tickets; Reviewer agent not fully integrated; no monetary ROI or time-savings numbers reported in the paper.

Implications for AI Economics

• Productivity and labor effects
  • Short-term: potential substantial reduction in routine, time-consuming manual work (ticket drafting, initial triage), enabling reallocation of skilled staff to higher-value tasks (design, oversight, exception handling).
  • Medium/long-term: technology acts as a complement for higher-skill administrative work (planning, review) while substituting for low-skill, repetitive tasks—implying shifts in job content rather than wholesale elimination in this context.
• Cost structure and capital investment
  • Higher upfront capital costs relative to cloud-based APIs: on-premise GPU hardware, model fine-tuning/hosting, and secure infrastructure.
  • Ongoing costs include model maintenance, evaluation, data management, and staff training; savings accrue through labor reallocation and process speedups but require careful ROI measurement.
• Scalability and generalizability
  • On-premise approach supports sensitive-government use cases but increases marginal deployment cost across agencies; shared public-sector platforms or federated approaches could amortize costs.
  • Multi-agent pattern is reusable across administrative workflows that involve unstructured text (reports, requests, applications), offering broad scaling potential.
• Flexibility, maintainability, and risk management
  • Compared to brittle rule-based automation (RPA), LLM-based agents are more adaptable to changing input formats and requirements, reducing maintenance churn for textual tasks.
  • However, they introduce new governance needs: model updates, prompt engineering, monitoring for hallucinations, duplication checks, and explainability — all requiring institutional investment.
• Policy, regulation, and societal acceptance
  • Data protection constraints push towards on-premise deployments in many jurisdictions; this increases public-sector costs but preserves trust and legal compliance.
  • Human-in-the-loop design and transparent logs (e.g., Jira traceability) help with accountability and public legitimacy—important for adoption in sensitive public services.
• Research and evaluation gaps relevant to economic decisions
  • Missing: quantified time savings, cost-benefit analysis, longer-term labor market impacts, and robustness under adversarial or ambiguous inputs.
  • These are necessary for a full economic appraisal (budgeting, hiring/training strategies, procurement choices).

Suggested economic takeaways for decision-makers - Expect meaningful efficiency gains in processing semi-structured natural-language administrative tasks; plan capital expenditure for secure on-premise infrastructure where data regulation requires it. - Budget for sustained governance: human oversight, model maintenance, and continuous evaluation—these are material costs, not one-off. - Consider hybrid deployment options (federated/shared models across agencies) to spread fixed costs while preserving data localization. - Prioritize measuring time-to-ticket and downstream development-cycle impacts to build a concrete ROI case and guide workforce transition policies.

Limitations and next steps - The study demonstrates technical feasibility and plausibility but does not provide hard monetary ROI or time-savings metrics—needed for full economic decision-making. - Future work: larger-scale longitudinal deployments, quantified productivity measures, cost-benefit modeling, and cross-agency comparative studies to assess general equilibrium labor effects.