Trajectories
Agent trajectory evaluation with reviewable evidence
The final answer is not enough. AgentClash evaluates the trajectory — tool choices, observations, retries, artifacts, and stop conditions — then preserves replay evidence for auditors and release owners.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
Why trajectories matter
Built for reviewable agent decisions
Two agents can return the same answer while taking wildly different paths. Trajectory evaluation catches unsafe shortcuts, runaway loops, and brittle tool strategies.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
Trajectory eval workflow
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Inspect runs with docs
Bring your first workload into the loop
Use replay and scorecards to debug trajectory regressions, then encode the workload as a challenge pack for CI.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Agent replay
See how AgentClash preserves replay evidence for reviewers.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
Dataset CI gates
Fail builds when a candidate regresses against a pinned baseline.
CI/CD agent gates
Block pull requests when agent behavior gets worse.
FAQ
Trajectory evaluation FAQ
What is agent trajectory evaluation?
Trajectory evaluation scores the sequence of actions and observations an agent took to complete a task, not just the final output string.
How does AgentClash store trajectory evidence?
Each run keeps replay events, tool calls, logs, artifacts, and scorecards so reviewers can reconstruct the path that produced the result.
Can trajectory evals gate releases?
Yes. Compare candidate and baseline trajectories via scorecards, then fail CI when correctness, cost, latency, or evidence quality regresses.