Glossary
What is Agentic Self-Instruct?
Agentic Self-Instruct generates synthetic training examples by proposing tasks, running weak and strong solvers, and accepting rows only when the strong path succeeds and the weak path struggles.
Candidate
Baseline
Control
replay timeline
ci verdict
Correctness improved, latency within budget, and required artifacts were preserved for review.
agentclash run create --follow
How it differs from self-instruct
Built for reviewable agent decisions
Classic self-instruct prompts a model for more examples. Agentic Self-Instruct adds solver rollouts and a judge so difficulty is measured, not assumed.
OpenTelemetry-compatible trace import
Pinned datasets and golden test cases
Baseline versus candidate regression checks
Replay trails for tool calls, outputs, and artifacts
Scorecards for correctness, cost, latency, and evidence
CI gates for prompt, model, RAG, and tool changes
Workflow
Typical roles
Import the evidence
Start from OpenTelemetry traces, curated datasets, support transcripts, or a real failure your team already saw.
Pin the baseline
Record the current accepted behavior so every prompt, model, RAG, or tool change has a fair comparison point.
Replay the evidence
Inspect tool calls, outputs, artifacts, latency, cost, and judge evidence when a candidate gets worse.
Gate the release
Compare candidate and baseline runs, then fail CI before a regression reaches users.
Go deeper
Bring your first workload into the loop
Read the Agentic Self-Instruct landing page and synthetic generation docs, or install DataSmith for local runs.
Agent evals
Real-task agent evals with replay evidence and CI gates.
LLM agent evaluation
Evaluate LLM agents on full trajectories, not one-shot answers.
Compare tools
See how AgentClash differs from prompt-eval platforms.
Agentic Self-Instruct hub
SEO landing with workflow and FAQ.
DataSmith platform
Open-source SDK overview.
Glossary index
More AgentClash terms.
Datasets overview
Import examples, record baselines, sync regression suites, and gate CI.
FAQ
Agentic Self-Instruct FAQ
Who popularized Agentic Self-Instruct?
Meta FAIR's Autodata paper formalized weak-vs-strong agentic self-instruct for synthetic data generation and meta-optimization.
What is the useful difficulty zone?
Examples where a strong solver passes and a weak solver fails, indicating the row can teach the weak model something new.
Where can I run it?
DataSmith (pip install datasmith) for local export, or AgentClash workspaces for hosted generation tied to eval gates.