Use case
Evaluate customer support agents on real resolutions
Support agents must resolve tickets safely, use the right tools, and leave an audit trail. AgentClash evaluates full support trajectories and keeps replay evidence when tone, policy, or resolution quality regresses.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
Support eval signals
Built for reviewable agent decisions
Score resolution correctness, policy adherence, tool usage, escalation behavior, and customer-facing artifact quality.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
Support eval workflow
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Start with escaped tickets
Bring your first workload into the loop
Promote real support failures into challenge packs and regression suites so the same mistake cannot return after a model update.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
Dataset CI gates
Fail builds when a candidate regresses against a pinned baseline.
CI/CD agent gates
Block pull requests when agent behavior gets worse.
FAQ
Support agent evaluation FAQ
Can AgentClash evaluate multi-turn support flows?
Yes. Multi-turn challenge packs support scripted, simulated, and human phases for realistic support conversations.
How do teams measure policy adherence?
Challenge packs encode required actions, forbidden tool use, and validator checks so scorecards reflect policy—not just friendly language.
Can support evals gate releases?
Yes. Compare candidate and baseline scorecards in CI before deploying a new support agent or model route.