Feature
Agent scorecards that justify a release decision
Scorecards turn long agent runs into a reviewable verdict. AgentClash aggregates correctness, cost, latency, tool strategy, and validator evidence so baselines and candidates are easy to compare.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
What scorecards include
Built for reviewable agent decisions
A scorecard should be more than a single number — it should explain why a run passed or failed.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
From run to scorecard
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Interpret results
Bring your first workload into the loop
Read the interpret-results guide, then wire scorecard thresholds into CI gates and release policy.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Interpret results
Learn how to read scorecards and replay evidence after a run.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
Dataset CI gates
Fail builds when a candidate regresses against a pinned baseline.
CI/CD agent gates
Block pull requests when agent behavior gets worse.
FAQ
Agent scorecard FAQ
What appears on an AgentClash scorecard?
Correctness signals, validator evidence, cost, latency, tool usage summaries, and dimension scores your challenge pack defines.
Can scorecards compare two runs?
Yes. Baseline versus candidate comparisons are first-class for regression testing and CI gates.
Can scorecards be exported or shared?
Yes. Runs keep scorecard views in the product and support shareable evidence for reviewers.