# Synthetic data generation for AI agents that teaches, not noise

Bulk self-instruct produces volume without signal. AgentClash and DataSmith run a weak-vs-strong judge loop that accepts examples only when a strong solver succeeds and a weak solver struggles: the useful difficulty zone for fine-tuning and eval.

Source: https://www.agentclash.dev/synthetic-data-generation-agents
Markdown export: https://www.agentclash.dev/md/synthetic-data-generation-agents

## Why weak-vs-strong beats prompt-only generation

Meta FAIR Autodata showed that agentic self-instruct with solver separation beats standard synthetic pipelines across CS, legal, and math settings. DataSmith implements the practical loop; AgentClash runs it hosted for regression-ready datasets.

- OpenTelemetry-compatible trace import
- Pinned datasets and golden test cases
- Baseline versus candidate regression checks
- Replay trails for tool calls, outputs, and artifacts
- Scorecards for correctness, cost, latency, and evidence
- CI gates for prompt, model, RAG, and tool changes

## From seeds to export or CI gates

### 1. Import the evidence

Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.

### 2. Pin the baseline

Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.

### 3. Replay the evidence

Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.

### 4. Gate the release

Compare candidate and baseline runs, then fail CI before a regression reaches users.

## Start generating

Use DataSmith locally for training export, or AgentClash workspaces for eval baselines and CI gates on the same examples.

- [Agent evals](https://www.agentclash.dev/agent-evals): Real-task agent evals with replay evidence and CI gates.
- [LLM agent evaluation](https://www.agentclash.dev/llm-agent-evaluation): Evaluate LLM agents on full trajectories, not one-shot answers.
- [Compare tools](https://www.agentclash.dev/compare): See how AgentClash differs from prompt-eval platforms.
- [DataSmith platform page](https://www.agentclash.dev/platform/datasmith): SDK + hosted generation overview.
- [Introducing DataSmith](https://www.agentclash.dev/blog/introducing-datasmith-synthetic-agent-data): Launch blog with pipeline details.
- [Synthetic generation docs](https://www.agentclash.dev/docs/guides/synthetic-dataset-generation): Run Agentic Self-Instruct in AgentClash.
- [Datasets overview](https://www.agentclash.dev/docs/guides/datasets-overview): Import examples, record baselines, sync regression suites, and gate CI.
- [Dataset CI gates](https://www.agentclash.dev/docs/guides/dataset-ci-gates): Fail builds when a candidate regresses against a pinned baseline.

## Synthetic data generation FAQ

### What is synthetic data generation for AI agents?

It is the process of creating labeled training or eval examples for agent workloads using LLM-driven pipelines instead of manual curation alone.

### How does Agentic Self-Instruct improve data quality?

A challenger proposes examples, weak and strong solvers attempt them, and a judge accepts rows only when the score gap indicates the example is learnable but not trivial.

### Should I use DataSmith or AgentClash for generation?

Use DataSmith for offline SFT, DPO, and Hugging Face export. Use AgentClash when generated examples should feed dataset evals, baselines, and CI regression gates.