# Fleet overview

Run packs × agents × models × repeats as one eval set with a live matrix, warehouse exports, budgets, and transcript scanners.

Source: https://www.agentclash.dev/docs/fleet
Markdown export: https://www.agentclash.dev/md/docs/fleet

**Fleet** is AgentClash's cloud-scale eval plane. You declare one manifest (packs × agents × models × repeats). AgentClash expands it into combinations, runs them with bounded concurrency and isolated sandboxes, then gives you a live matrix, queryable case results, optional budgets, and post-run transcript scanners.

Use Fleet when a single `run create` or eval session is not enough: nightly sweeps, multi-agent bakeoffs, multi-pack regression, or self-hosted fleets that need worker autoscaling.

## What you get

| Surface | Role |
|---|---|
| Eval-set manifest (`evalset/v1`) | Declarative packs × agents × models × repeats |
| CLI `agentclash evalset` | Submit, follow, logs, report, cancel |
| Web matrix | Live cells per combination under workspace → Eval sets |
| Case warehouse | Search, report, CSV/JSONL export of per-case outcomes |
| Budgets | Soft USD caps and workspace emergency stop |
| Scanners | Optional post-completion transcript analysis |
| Self-host scale | Helm chart, KEDA queues, Kubernetes sandboxes, metrics |

## Prerequisites

Before you submit an eval set:

1. A workspace with API access (`agentclash auth login`, workspace selected).
2. Challenge pack versions available in the workspace (or catalog slugs).
3. Agent deployments wired with provider credentials / BYOK.
4. A sandbox provider configured for native packs (`e2b`, `docker`, or `kubernetes` on self-host).

Hosted users typically do step 1–3 on [agentclash.dev](https://www.agentclash.dev). Self-hosters also need Temporal, Postgres, workers, and a sandbox path; see [Self-host at scale](https://www.agentclash.dev/md/docs/fleet/self-host-scale).

## Shortest path

```bash
export AGENTCLASH_API_URL="https://api.agentclash.dev"

agentclash auth login --device
agentclash workspace use <workspace-id>

agentclash evalset init
# edit agentclash.evalset.yaml — packs, deployments, repeats, budget

agentclash evalset submit agentclash.evalset.yaml --dry-run
agentclash evalset submit agentclash.evalset.yaml --yes --follow
agentclash evalset report <eval-set-id> --format csv
```

Open the set in the product UI: **Workspace → Eval sets →** your set for the live matrix and case explorer.

## How it relates to runs and eval sessions

- A **run** is still the unit of execution and scoring.
- An **eval session** groups repeated runs of one pack lineup.
- An **eval set** is the Fleet layer: many packs × agents × models × repeats, orchestrated as child sessions/runs under one set ID and `matrix_key` identity.

See [Runs and evals](https://www.agentclash.dev/md/docs/concepts/runs-and-evals) for the noun map, then continue with [Eval-set manifests](https://www.agentclash.dev/md/docs/fleet/eval-set-manifests).

## Docs in this section

- [Eval-set manifests](https://www.agentclash.dev/md/docs/fleet/eval-set-manifests) — YAML fields, expansion, limits
- [CLI evalset workflow](https://www.agentclash.dev/md/docs/fleet/cli-evalset) — submit through report
- [Matrix and warehouse](https://www.agentclash.dev/md/docs/fleet/matrix-and-warehouse) — UI + export APIs
- [Budgets and scanners](https://www.agentclash.dev/md/docs/fleet/budgets-and-scanners) — spend caps and findings
- [Self-host at scale](https://www.agentclash.dev/md/docs/fleet/self-host-scale) — Helm, sandboxes, queues, observability