Binary pass/fail metrics understate risks in agentic AI: a four-pillar assessment (LLMs, Memory, Tools, Environment) uncovers runtime behavioral deviations in a production CloudOps agent that traditional evaluation missed.

Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems

Sreemaee Akshathala, Bassam Adnan, Mahisha Ramesh, Karthik Vaidhyanathan, Basil Muhammed, Kannan Parthasarathy · December 14, 2025

arxiv descriptive low evidence 7/10 relevance Full text usable extracted full text Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: July 28, 2026

Sreemaee Akshathala unresolved corpus identity
Bassam Adnan unresolved corpus identity
Mahisha Ramesh unresolved corpus identity
Karthik Vaidhyanathan unresolved corpus identity
Basil Muhammed unresolved corpus identity
Kannan Parthasarathy unresolved corpus identity

Semantic Scholar

Latest observation: July 30, 2026

Sreemaee Akshathala provider ID
Bassam Adnan provider ID
Mahisha Ramesh provider ID
Karthik Vaidhyanathan provider ID
Basil Muhammed provider ID
Kannan Parthasarathy provider ID

The paper introduces a four-pillar Agent Assessment Framework (LLMs, Memory, Tools, Environment) and shows that conventional binary task completion metrics miss runtime behavioral uncertainties in agentic AI, as demonstrated in a production Autonomous CloudOps deployment.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

4 cumulative citations

Semantic Scholar · Observed July 31, 2026

View corpus context

Recent advances in agentic AI have shifted the focus from standalone Large Language Models (LLMs) to integrated systems that combine LLMs with tools, memory, and other agents to perform complex tasks. These multi-agent architectures enable coordinated reasoning, planning, and execution across diverse domains, allowing agents to collaboratively automate complex workflows. Despite these advances, evaluation and assessment of LLM agents and the multi-agent systems they constitute remain a fundamental challenge. Although various approaches have been proposed in the software engineering literature for evaluating conventional software components, existing methods for AI-based systems often overlook the non-deterministic nature of models. This non-determinism introduces behavioral uncertainty during execution, yet existing evaluations rely on binary task completion metrics that fail to capture it. Evaluating agentic systems therefore requires examining additional dimensions, including the agent ability to invoke tools, ingest and retrieve memory, collaborate with other agents, and interact effectively with its environment. These challenges emerged during our ongoing industry collaboration with MontyCloud Inc., when we deployed an agentic system in production. These limitations surfaced during deployment, highlighting practical gaps in the current evaluation methods and the need for a systematic assessment of agent behavior beyond task outcomes. Informed by these observations and established definitions of agentic systems, we propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment. We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations overlooked by conventional metrics, demonstrating its effectiveness in capturing runtime uncertainties.

Summary

Main Finding

The paper presents an end-to-end Agent Assessment Framework (ASF) for agentic AI systems that evaluates behavior beyond binary task completion. By decomposing assessment into four pillars—LLM, Memory, Tools, and Environment—and applying static, dynamic, and judge-based evaluations, the framework uncovers behavioral deviations (e.g., policy violations, incorrect tool orchestration, faulty memory retrieval) that conventional task-success metrics miss. Validation on a multi-agent CloudOps use case (MOYA / MontyCloud) shows agents often complete tasks yet exhibit important runtime failures that matter for reliability, safety, and operational cost.

Key Points

Motivation: Agentic systems (LLMs + tools + memory + multi-agent coordination) are non-deterministic; this propagates runtime uncertainty that task-completion metrics cannot capture.
Four assessment pillars:
- LLM: instruction-following, safety & alignment, and LLM-as-judge evaluation.
- Memory: storage correctness (updates, duplication), retrieval accuracy (single-hop, multi-hop, temporal, open-domain), and update/visibility latency.
- Tools: tool selection/classification, parameter mapping, sequencing, error interpretation, performance, and reliability under failure.
- Environment: observability (telemetry, logs), workflows/guardrails, controllability and configurability (resettable states, fault injection).
Three evaluation modes:
- Static analysis: compare agent trajectories / outputs to ground-truth labels (precision/recall/F1, BLEU/ROUGE, sequence correctness, tool classification accuracy, safety-policy checks).
- Dynamic execution: runtime monitoring, sandboxed tool executions, fault injection, guardrail violation tracking.
- Judge-based evaluation: qualitative scoring using LLM-as-judge and Agent-as-judge protocols to capture reasoning, alignment, and interpretability beyond numeric metrics.
Test case generator: produces controlled, reproducible scenarios per pillar to stress specific behaviors and enable repeatable assessment.
Representative metrics (sample): instruction adherence score, sequence correctness, test case coverage, safety violation count, memory update correctness & latency, retrieval precision/recall/F1, tool selection accuracy, parameter accuracy, recovery success rate, guardrail violation count.
Empirical validation: experiments on MOYA with three CloudOps scenarios showed that while baseline metrics reported task completion, ASF detected substantial failures—especially in tool orchestration and memory retrieval—that would be missed otherwise.
Practical assumptions: environment must provide sufficient observability (telemetry, traces, logs) to support dynamic analysis and RCA.

Data & Methods

Validation context: Autonomous CloudOps domain (industrial collaboration with MontyCloud Inc.), using the MOYA multi-agent framework and three production-motivated CloudOps scenarios of increasing complexity (examples: EC2 deployment & networking, S3 bucket setup & access policies, multi-step incident remediation).
Test-case generation: Controlled test cases specifying context, expected behaviors, and success criteria for each pillar; scenarios include both normal and injected-fault conditions.
Static metrics and labels: ground-truth sequences for tool calls, expected parameter mappings, golden memory entries/retrievals, and policy constraints to compute metric scores (precision/recall/F1, BLEU/ROUGE, classification accuracy, sequence correctness).
Dynamic execution: agents executed in sandboxed or instrumented environments; telemetry and logs captured to detect guardrail violations, unauthorized actions, error messages, recovery attempts, and runtime latencies.
Judge-based evaluation: structured prompts send execution logs to an LLM-judge and/or an auditor agent (Agent-as-Judge) which scores and justifies assessments on instruction-following, safety, memory usage, and tool correctness.
Outcome: Comparison of conventional task-success rates vs. pillar-specific failure modes; reported result: conventional metrics often indicate success while the ASF reveals operationally significant misbehaviors (e.g., repeated wrong tool invocations, stale memory retrievals, unsafe parameter choices).
Reproducibility: authors provide code and experiments (GitHub link referenced) and emphasize environment configurability to reset states and reproduce test runs.

Implications for AI Economics

Measurement and valuation:
- Productivity gains from agentic systems can be overstated if measured only by task-completion metrics. ASF metrics (failure rates, recovery costs, variance in outputs) should be incorporated into ROI and productivity models.
- Include runtime uncertainty and behavioral variance when estimating expected benefits; use distributional estimates (mean + tails) rather than point estimates.
Operational costs and adoption:
- Non-deterministic failures raise monitoring, observability, and human oversight costs (continuous telemetry, fault analysis, rollback capabilities). These should be budgeted as recurring operational expenditures.
- Tool orchestration and memory faults create downstream remediation costs and potential downtime—key drivers of total cost of ownership for agentic deployments.
Risk, insurance, and liability:
- Behavioral failures (policy violations, unauthorized actions) increase downside tail risk for firms deploying agents in production. This affects insurance pricing, contractual SLAs, and compliance liabilities.
- Regulators and auditors will want interpretable assessments (judge outputs, telemetry traces). Standardized frameworks like ASF could become part of compliance checklists and third-party audits.
Market structure and investments:
- Demand will grow for specialized testing, observability, and safety tooling companies that implement ASF-like assessments (sandboxing, judge evaluation, memory/test generators).
- Firms may prefer modular tool ecosystems (easier to validate and recover) over monolithic tool APIs—affecting vendor competition and integration strategies.
Labor and task allocation:
- Economic models of labor displacement should adjust for quality and reliability differentials: supervision and review labor remains valuable due to non-deterministic agent behavior. Net labor impact depends on costs of oversight versus task automation gains.
Recommendations for empirical work and policy analysis:
- When estimating economic impacts of agentic AI, use pillar-specific failure metrics (e.g., tool-error frequency, memory-staleness rate) to model ongoing costs and risk.
- Incorporate costs of observability, test-case generation, and judge-based evaluation into deployment cost models.
- Model tail risks (rare but high-cost failures) explicitly; consider stress-test scenarios analogous to ASF’s fault injection when computing expected losses.
- For regulators and procurement, require standardized assessment outputs (audit logs, judge justifications, test coverage) as part of procurement and certification processes.

Summary: The ASF offers a systematic, operationally-grounded way to quantify agentic AI reliability. For AI economics, it highlights hidden costs and risks that standard task-level metrics hide, implying more conservative valuations, higher operational budgets for monitoring/testing, and new markets for verification and observability services.

Assessment

Paper Typedescriptive Evidence Strengthlow — The paper proposes an evaluation framework and validates it using a single industry collaboration and a representative Autonomous CloudOps use case; it does not provide causal inference, broad empirical testing, randomized comparisons, or large-scale external validation, limiting the strength of empirical evidence. Methods Rigormedium — The authors systematically derive evaluation pillars from observed deployment issues and established definitions, and they validate the framework with experiments in a production-like CloudOps setting; however, the validation appears limited in scope (single partner, one domain), lacks counterfactuals or robustness checks across diverse systems, and does not quantify statistical uncertainty extensively. SampleQualitative and experimental validation conducted through an industry collaboration with MontyCloud Inc.; experiments and deployment observations come from a production Autonomous CloudOps agentic system combining an LLM, tool integrations, memory, and multi-agent coordination in a cloud operations workflow. Themesproductivity adoption GeneralizabilitySingle-company (MontyCloud) deployment may reflect firm-specific architectures, practices, and workloads., Evaluation validated only on a CloudOps/autonomous ops use case, limiting transferability to other domains (e.g., customer service, finance, manufacturing)., Findings may depend on the particular LLM, tool-chain, memory design, and system engineering choices used in the deployment., Non-deterministic behaviors observed may vary with different model versions, prompts, or runtime configurations not explored., No large-scale, cross-organizational or cross-task validation to establish broader applicability.

Claims (10)

Claim	Direction	Outcome	Confidence & Evidence	Details
Recent advances in agentic AI have shifted the focus from standalone Large Language Models (LLMs) to integrated systems that combine LLMs with tools, memory, and other agents to perform complex tasks. Adoption Rate	positive	shift in research and system design focus toward integrated agentic systems	Reading fidelity high Study strength medium	not reported 0.18
These multi-agent architectures enable coordinated reasoning, planning, and execution across diverse domains, allowing agents to collaboratively automate complex workflows. Organizational Efficiency	positive	ability to coordinate reasoning, planning, and execute automated workflows	Reading fidelity high Study strength medium	not reported 0.18
Evaluation and assessment of LLM agents and the multi-agent systems they constitute remain a fundamental challenge. Ai Safety And Ethics	negative	difficulty/limitations in current evaluation methods for agentic systems	Reading fidelity high Study strength medium	not reported 0.18
Existing methods for AI-based systems often overlook the non-deterministic nature of models. Ai Safety And Ethics	negative	coverage of model non-determinism by existing evaluation methods	Reading fidelity high Study strength medium	not reported 0.18
This non-determinism introduces behavioral uncertainty during execution, yet existing evaluations rely on binary task completion metrics that fail to capture it. Ai Safety And Ethics	negative	ability of binary task completion metrics to capture behavioral uncertainty	Reading fidelity high Study strength medium	not reported 0.18
Evaluating agentic systems therefore requires examining additional dimensions, including the agent ability to invoke tools, ingest and retrieve memory, collaborate with other agents, and interact effectively with its environment. Task Allocation	positive	coverage of tool use, memory operations, collaboration, and environment interaction in evaluations	Reading fidelity high Study strength medium	not reported 0.18
These challenges emerged during our ongoing industry collaboration with MontyCloud Inc., when we deployed an agentic system in production. Organizational Efficiency	negative	operational challenges observed during production deployment	Reading fidelity high Study strength medium	not reported 0.18
These limitations surfaced during deployment, highlighting practical gaps in the current evaluation methods and the need for a systematic assessment of agent behavior beyond task outcomes. Ai Safety And Ethics	negative	practical gaps in evaluation methods revealed by deployment	Reading fidelity high Study strength medium	not reported 0.18
We propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment. Governance And Regulation	positive	existence and composition of the proposed assessment framework	Reading fidelity high Study strength medium	not reported 0.18
We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations overlooked by conventional metrics, demonstrating its effectiveness in capturing runtime uncertainties. Error Rate	positive	detection of behavioral deviations/runtime uncertainties by the proposed framework versus conventional metrics	Reading fidelity high Study strength medium	not reported 0.18