Dimension
Goal and success
Failure mode
The agent finishes a plausible task that does not satisfy the original request.
Evidence to inspect
Compare the user goal, completion criteria, final output, and user-visible confirmation in one task trace.
Repair
Add explicit acceptance criteria and an independent result check before completion.
Dimension
Context and state
Failure mode
Later steps lose constraints, source identity, or decisions made earlier in the workflow.
Evidence to inspect
Inspect prompts, retrieved records, state transitions, summaries, and handoff payloads.
Repair
Use typed state, provenance fields, bounded summaries, and handoff validation.
Dimension
Tools and permissions
Failure mode
A tool accepts ambiguous input or can write more data than the task requires.
Evidence to inspect
Review tool schema, validation, authentication scope, write target, and failure response.
Repair
Narrow the contract, validate targets, default to dry-run, and require approval for high-impact writes.
Failure mode
The team relies on a successful demo and cannot detect regressions or unsafe edge cases.
Evidence to inspect
List representative tasks, failure cases, graders, thresholds, and prior run results.
Repair
Create repeatable eval cases for task quality, tool selection, policy boundaries, and recovery behavior.
Failure mode
Cost, latency, retries, or silent partial failures cannot be reconstructed after the run.
Evidence to inspect
Inspect structured logs, trace identifiers, model and tool timings, errors, and final status fields.
Repair
Add trace-level logs, bounded retry reasons, explicit partial status, and actionable alerts.
Dimension
Release and recovery
Failure mode
A production action is treated as complete without confirmation or a rollback path.
Evidence to inspect
Check approval records, production response, native-state verification, owner, and rollback procedure.
Repair
Separate candidate validation from release, verify the native destination, and rehearse rollback.