Use case
Evaluate research agents with evidence quality
Research agents live or die on sourcing, synthesis, and artifact quality. AgentClash scores whether an agent found the right evidence, cited it correctly, and finished the investigation under budget.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
Research eval signals
Built for reviewable agent decisions
Measure coverage, citation quality, artifact completeness, and whether the agent stopped with a useful deliverable.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
Research eval workflow
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Encode investigations as packs
Bring your first workload into the loop
Turn recurring research workflows into challenge packs so every model or prompt change reruns the same investigation fairly.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
Dataset CI gates
Fail builds when a candidate regresses against a pinned baseline.
CI/CD agent gates
Block pull requests when agent behavior gets worse.
FAQ
Research agent evaluation FAQ
What should research agent evaluation score?
Task completion, evidence quality, artifact completeness, tool discipline, and whether the final synthesis matches the sources collected.
Can evals include web or file tools?
Yes. Challenge packs define which tools agents may use and which artifacts must be produced for a pass.
How do teams debug a failed research run?
Replay shows each search, fetch, note, and synthesis step so reviewers can see where the investigation went off track.