# Turn production agent traces into datasets

The best seed material often comes from real agent runs. DataSmith ingests OTLP JSON and span JSONL locally for training export. AgentClash imports traces into workspace datasets for eval baselines and CI gates.

Source: https://www.agentclash.dev/trace-to-dataset
Markdown export: https://www.agentclash.dev/md/trace-to-dataset

## Two paths from traces

Training teams use DataSmith ingest-otel for SFT and DPO pipelines. Platform teams use AgentClash trace import to promote production failures into regression coverage.

- OpenTelemetry-compatible trace import
- Pinned datasets and golden test cases
- Baseline versus candidate regression checks
- Replay trails for tool calls, outputs, and artifacts
- Scorecards for correctness, cost, latency, and evidence
- CI gates for prompt, model, RAG, and tool changes

## Trace ingestion workflow

### 1. Import the evidence

Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.

### 2. Pin the baseline

Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.

### 3. Replay the evidence

Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.

### 4. Gate the release

Compare candidate and baseline runs, then fail CI before a regression reaches users.

## Ingest traces

Start with one exported trace bundle, convert to seeds or candidates, then expand with Agentic Self-Instruct generation.

- [Agent evals](https://www.agentclash.dev/agent-evals): Real-task agent evals with replay evidence and CI gates.
- [LLM agent evaluation](https://www.agentclash.dev/llm-agent-evaluation): Evaluate LLM agents on full trajectories, not one-shot answers.
- [Compare tools](https://www.agentclash.dev/compare): See how AgentClash differs from prompt-eval platforms.
- [DataSmith OTLP docs](https://github.com/Atharva-Kanherkar/datasmith/blob/main/docs/otel.md): Local trace ingestion reference.
- [Synthetic data generation](https://www.agentclash.dev/synthetic-data-generation-agents): Expand seeds with weak-vs-strong generation.
- [Agent regression testing](https://www.agentclash.dev/platform/agent-regression-testing): Gate releases on trace-derived coverage.
- [Datasets overview](https://www.agentclash.dev/docs/guides/datasets-overview): Import examples, record baselines, sync regression suites, and gate CI.
- [Dataset CI gates](https://www.agentclash.dev/docs/guides/dataset-ci-gates): Fail builds when a candidate regresses against a pinned baseline.

## Trace to dataset FAQ

### What trace formats does DataSmith support?

OTLP JSON exports and flattened span JSONL. See the DataSmith docs/otel.md guide for field expectations.

### Does AgentClash import the same traces?

AgentClash supports OTel-compatible trace import into workspace datasets for eval and regression workflows, complementary to DataSmith training export.

### Should traces become seeds or finished examples?

Usually seeds. Run Agentic Self-Instruct afterward to expand grounded, judge-filtered examples rather than training on raw spans alone.