# Fleet: eval sets at cloud scale, plus two new ways to author a pack

The Fleet epic landed in fifteen pull requests — case-level fan-out, Kubernetes sandboxes, per-account LLM throttling, eval-set manifests, a live matrix dashboard, post-hoc scanners, budgets with emergency stop, OTel metrics, and a Helm chart. Alongside it: one-command local setup for contributors, custom OpenAI-compatible endpoints, pack drafts generated from a tryout or a plain-English description, and a machine-readable public site.

Period: Aug 07 – Aug 15, 2026
Source: https://www.agentclash.dev/changelog/2026-08-07
Markdown export: https://www.agentclash.dev/md/changelog/2026-08-07

## Changes

- **Added**: Eval-set manifests (`evalset/v1`) — parse, dry-run combination expansion, persistence, and orchestration of child sessions across packs, agents, and models.
- **Added**: `agentclash evalset submit`, `status`, `logs`, `report`, and `cancel` for driving eval sets from the terminal or CI.
- **Added**: Live eval-set matrix dashboard plus a case-results warehouse with search, report, and export.
- **Added**: Per-set budgets with emergency stop, and a post-hoc scanner framework that flags patterns across a finished eval set.
- **Added**: Kubernetes sandbox provider with a conformance suite, sandbox capacity budgets and warm pool, bounded workflow fan-out, queue partitioning, and case-level fan-out for native run-agents.
- **Added**: Helm chart with KEDA-based Temporal worker autoscaling and published GHCR images for self-hosted deployments.
- **Added**: Custom OpenAI-compatible provider accounts — persisted per-account base URLs with SSRF protection at both save time and dial time.
- **Added**: Two new ways to start a challenge pack: promote a completed agent tryout into an editable draft, or describe your app in plain English and get a generated pack blueprint in the builder.
- **Added**: Machine-readable public site — stable Markdown URLs for every indexable route, `llms.txt`, `Accept: text/markdown` negotiation, and published OpenAPI, JSON Schema, and CLI schema contracts.
- **Added**: One-command contributor setup — `make setup`, `make start`, `make doctor`, and `make check`, with Temporal in Docker and a dev profile that runs with zero API keys.
- **Added**: `/healthz/ready` readiness probe that reports whether an instance can actually serve traffic, separate from the liveness check.
- **Improved**: OpenTelemetry metrics, stall detection, and a ready-to-import alert pack for run and eval-set health.
- **Improved**: Oversized run-event payloads now spill to object storage, and outbound LLM calls are throttled per account across workers.
- **Fixed**: Onboarding funnel fixes — dead demo CTA on every docs page, CLI errors that never pointed at signup, and jargon that hid what the product does.
- **Fixed**: Fleet matrix cells now read as human labels, and backend images build from the repo root so the shared runtime module resolves.
- **Fixed**: Benchmarks hub reordered to lead with measured proof and drop empty scoreboard columns.