# Datasets overview

Generate pinned dataset evals, record baselines, sync regression suites, and gate releases with agentclash dataset commands.

Source: https://www.agentclash.dev/docs/guides/datasets-overview
Markdown export: https://www.agentclash.dev/md/docs/guides/datasets-overview

Dataset evals run a **pinned version** of labeled examples through a challenge pack and deployment, then compare candidate runs against recorded baselines. They complement ad-hoc eval runs when you need reproducible regression signal tied to curated examples.

## Lifecycle

1. **Create or import a dataset** in the workspace (web UI or API).
2. **Pin a version** with examples linked to challenge cases.
3. **Run a dataset eval** — AgentClash executes each example and aggregates pass/fail.
4. **Record a baseline** from a green eval run.
5. **Gate CI** with `agentclash dataset test` (see [Dataset CI Gates](https://www.agentclash.dev/md/docs/guides/dataset-ci-gates)).

```bash
export AGENTCLASH_API_URL="https://api.agentclash.dev"
export AGENTCLASH_TOKEN="<token>"
export AGENTCLASH_WORKSPACE="<workspace-id>"

# Start an eval against a pinned version
agentclash dataset eval <datasetId> \
  --version <dataset-version-id> \
  --pack <challenge-pack-version-id> \
  --challenge support \
  --deployment <deployment-id>
```

## Record a baseline

A baseline snapshots the per-example outcomes of one completed, green eval run so later candidates can be compared against it. Recording a baseline is an API call (no CLI subcommand yet): `POST /v1/workspaces/{workspaceID}/datasets/{datasetID}/baselines` with the eval `run_id` and an optional `label`. The response returns a baseline `id` — pass that to `agentclash dataset test --baseline <id>` to gate candidates against it.

```bash
curl -sS -X POST \
  "$AGENTCLASH_API_URL/v1/workspaces/$AGENTCLASH_WORKSPACE/datasets/<datasetId>/baselines" \
  -H "Authorization: Bearer $AGENTCLASH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"run_id": "<eval-run-id>", "label": "support v1 baseline"}'
```

## Baselines and regression suites

Syncing a pinned dataset version into a regression suite copies each example's provenance — its dataset-example ID and trace metadata — into the workspace regression suite, so manifest-based release gates can reference the same curated examples. (Under the hood these land in the `workspace_regression_cases` table keyed by `dataset_example_id`.)

```bash
agentclash dataset sync-regression-suite <datasetId> \
  --version <dataset-version-id> \
  --pack <challenge-pack-version-id> \
  --challenge support \
  --suite-name "Support dataset regression"
```

Re-running sync is idempotent: cases that already exist for an example are skipped, so you can safely re-sync after adding new examples.

## CI gate output formats

`agentclash dataset test` supports human text, JSON gate payloads, and JUnit XML for CI parsers. Use `--max-regressions 0` to fail on any newly failing example versus the baseline.

> Info: Dataset gates are complementary to [CI/CD agent gates](https://www.agentclash.dev/md/docs/guides/ci-cd-agent-gates) and [eval workflows](https://www.agentclash.dev/md/docs/challenge-packs/eval-workflows-and-gates). Use datasets when examples are curated and pinned; use eval scorecards when you are comparing agent revisions on a live pack.

## See also

- [Synthetic dataset generation](https://www.agentclash.dev/md/docs/guides/synthetic-dataset-generation) — Fast Self-Instruct and Agentic Self-Instruct in workspaces
- [Dataset CI Gates](https://www.agentclash.dev/md/docs/guides/dataset-ci-gates) — GitHub Actions recipe and API surface
- [CI/CD workload recipes](https://www.agentclash.dev/md/docs/guides/ci-cd-workload-recipes) — pick realistic agent workloads
- [Interpret results](https://www.agentclash.dev/md/docs/guides/interpret-results) — read scorecards and timelines