Feature
Agent replay that makes failures reviewable
When an agent gate fails, reviewers need the path — not a summary. AgentClash replay timelines show tool calls, observations, artifacts, and scorecard context in one place.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
Why replay matters
Built for reviewable agent decisions
Replay turns agent eval from a score into an auditable story of what the agent tried and what the environment returned.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
Replay workflow
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Debug with docs
Bring your first workload into the loop
Pair replay with interpret-results guidance and challenge packs so every debugged failure becomes a reusable test.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Trajectory evaluation
See how replay supports full-path agent scoring.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
Dataset CI gates
Fail builds when a candidate regresses against a pinned baseline.
CI/CD agent gates
Block pull requests when agent behavior gets worse.
FAQ
Agent replay FAQ
What does AgentClash replay include?
Tool calls, observations, logs, artifacts, timing, and links to the scorecard dimensions that passed or failed.
Can replay be shared with reviewers?
Yes. Runs support shareable views so PMs, support leads, and engineers can review the same evidence.
How does replay help CI failures?
When a gate fails, replay shows exactly which step diverged from baseline so you do not guess from a single error string.