# Agent evals that catch production regressions

Agent evals should answer whether a change made your agent worse. AgentClash imports traces or curated examples, compares candidates to baselines, and turns failures into release gates.

Source: https://www.agentclash.dev/agent-evals
Markdown export: https://www.agentclash.dev/md/agent-evals

## What useful agent evals capture

If your agent eval only checks the final string, you miss the tool misuse, RAG drift, latency spikes, and artifact gaps that show up in production.

- OpenTelemetry-compatible trace import
- Pinned datasets and golden test cases
- Baseline versus candidate regression checks
- Replay trails for tool calls, outputs, and artifacts
- Scorecards for correctness, cost, latency, and evidence
- CI gates for prompt, model, RAG, and tool changes

## From one eval to a reusable gate

### 1. Import the evidence

Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.

### 2. Pin the baseline

Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.

### 3. Replay the evidence

Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.

### 4. Gate the release

Compare candidate and baseline runs, then fail CI before a regression reaches users.

## Run your first agent eval

Use datasets and challenge packs for repeatable workloads, then promote failures into regression cases your team can run in CI.

- [LLM agent evaluation](https://www.agentclash.dev/llm-agent-evaluation): Evaluate LLM agents on full trajectories, not one-shot answers.
- [Compare tools](https://www.agentclash.dev/compare): See how AgentClash differs from prompt-eval platforms.
- [Datasets overview](https://www.agentclash.dev/docs/guides/datasets-overview): Import examples, record baselines, sync regression suites, and gate CI.
- [Dataset CI gates](https://www.agentclash.dev/docs/guides/dataset-ci-gates): Fail builds when a candidate regresses against a pinned baseline.
- [CI/CD agent gates](https://www.agentclash.dev/docs/guides/ci-cd-agent-gates): Block pull requests when agent behavior gets worse.

## Agent eval FAQ

### What is an agent eval?

An agent eval is a repeatable test that runs an agent on a task, scores the full trajectory, and compares the result to a baseline, dataset, or competitor.

### How is AgentClash different from prompt evals?

Prompt evals score one model response. AgentClash evals multi-turn agents that use tools in a sandbox and scores correctness, cost, latency, and evidence quality across the run.

### Can agent evals run in CI?

Yes. AgentClash can compare candidate and baseline scorecards, including dataset baselines, and fail a pull request when the configured release gate regresses.