# CI/CD regression gates for AI agents

Model, prompt, RAG, and tool changes should not ship on vibes. AgentClash runs repeatable agent workloads in CI, compares candidates to baselines, and blocks merges when behavior regresses.

Source: https://www.agentclash.dev/ci-cd-agent-evaluation
Markdown export: https://www.agentclash.dev/md/ci-cd-agent-evaluation

## What CI/CD agent eval needs

Release engineering needs pinned datasets, stable scoring, and enough replay evidence to debug a failed gate without reproducing the issue manually.

- OpenTelemetry-compatible trace import
- Pinned datasets and golden test cases
- Baseline versus candidate regression checks
- Replay trails for tool calls, outputs, and artifacts
- Scorecards for correctness, cost, latency, and evidence
- CI gates for prompt, model, RAG, and tool changes

## CI gate workflow

### 1. Import the evidence

Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.

### 2. Pin the baseline

Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.

### 3. Replay the evidence

Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.

### 4. Gate the release

Compare candidate and baseline runs, then fail CI before a regression reaches users.

## Wire gates with docs

Start with dataset or challenge-pack gates, then promote real failures into cases your pipeline can rerun on every change.

- [Agent evals](https://www.agentclash.dev/agent-evals): Real-task agent evals with replay evidence and CI gates.
- [LLM agent evaluation](https://www.agentclash.dev/llm-agent-evaluation): Evaluate LLM agents on full trajectories, not one-shot answers.
- [Compare tools](https://www.agentclash.dev/compare): See how AgentClash differs from prompt-eval platforms.
- [Agent regression testing](https://www.agentclash.dev/platform/agent-regression-testing): Product overview for trace and dataset-driven regression testing.
- [Datasets overview](https://www.agentclash.dev/docs/guides/datasets-overview): Import examples, record baselines, sync regression suites, and gate CI.
- [Dataset CI gates](https://www.agentclash.dev/docs/guides/dataset-ci-gates): Fail builds when a candidate regresses against a pinned baseline.
- [CI/CD agent gates](https://www.agentclash.dev/docs/guides/ci-cd-agent-gates): Block pull requests when agent behavior gets worse.

## CI/CD agent evaluation FAQ

### How do agent eval gates fit into CI/CD?

A dataset or challenge pack runs on every candidate change, AgentClash compares the scorecard to a baseline, and the pipeline fails when configured thresholds regress.

### What happens when a gate fails?

Reviewers open replay evidence, inspect tool calls and artifacts, and either fix the regression or update the baseline intentionally.

### Can gates cover cost and latency budgets?

Yes. Scorecards include correctness and evidence quality plus cost and latency signals you can enforce in release policy.