AgentClash

Challenge packs

Tools, primitives & policy

How tool_policy and tools.custom map to engine primitives in backend/internal/engine and sandbox.ToolPolicy.

AgentClash stacks three tool notions (also summarized in Tools, network, and secrets):

  1. Workspace tool resources — org-level infrastructure objects (not covered by pack YAML)
  2. Pack composed toolstools.custom[] entries expanding to JSON Schema + implementation
  3. Engine primitives — concrete executors registered in nativePrimitiveTools (backend/internal/engine/primitive_tools.go)

Only (2)+(3) are pack-controlled.

Tool policy shape

version.tool_policy JSON eventually hydrates sandbox.ToolPolicy (backend/internal/sandbox/sandbox.go):

  • allowed_tool_kinds — list controlling capability groups
  • allow_shell — separate bool gating the exec primitive

Recognized kind strings

Validated set (supportedToolKinds in challengepack/validation.go):

browser, build, data, file, network

Shell is not a kind—enable it with allow_shell: true.

Empty allowlist semantics

allowsToolKind treats an empty allowed_tool_kinds as “allow everything” (per primitive_helpers.go). In practice, prefer explicit lists so validation errors catch typos early.

Mode guardrails

prompt_eval packs must omit tool_policy entirely—see Bundle YAML reference.

Built-in primitive names

Declared in executor_builders.go, registered in nativePrimitiveTools:

| Primitive | Gated by | | --- | --- | | submit | Always available (final answer) | | read_file, write_file, list_files, search_files, search_text | file kind | | query_json, query_sql | data kind | | http_request | network kind (+ runtime network flags) | | run_tests, build | build kind | | exec | allow_shell |

Browser tooling exists in policy (toolKindBrowser)—ensure your template + worker build includes whatever browser bridge your pack expects before relying on it in production.

Composed tools (tools.custom[])

Each item:

tools:
  custom:
    - name: call_support_api
      description: Fetch ticket JSON
      parameters:
        type: object
        properties:
          ticket_id: { type: string }
        required: [ticket_id]
        additionalProperties: false
      implementation:
        primitive: http_request
        args:
          method: GET
          url: https://api.example.com/tickets/${ticket_id}
          headers:
            Authorization: Bearer ${secrets.SUPPORT_TOKEN}

Validation highlights (validateComposedToolConfig):

  • Non-mock tools require implementation.primitive not equal to the composed name (prevents self-delegation footgun)
  • implementation.args object required; templates validated for placeholder safety
  • Parameters must be JSON Schema passing templateutil.ValidateToolParameterSchema
  • Custom graph cannot contain cycles or depth > 8 delegation jumps

Mock implementations

Set implementation.type: mock to skip primitive resolution—useful for dry-run packs or policy-only testing. Mocks bypass cycle detection.

Workspace tools vs pack tools

Pack tools are not the same records as API tools resources—they are bundle-local contracts interpreted entirely inside the worker.

Secret placeholders

Composed args may reference ${secrets.NAME} which resolve through workspace secret stores—never place secret material inline. Sandbox env_vars explicitly reject secret placeholders (see native executor sandbox guard) because environment leaks are too easy; prefer header injection on http_request.

Provider visibility

buildToolRegistry lifts final OpenAI/Anthropic/etc. tool definitions from the registry’s visible map—only tools allowed by policy + manifest appear to the model.

See also

  • Sandbox & E2B for network pairing with http_request
  • backend/internal/challengepack/tools_validation_test.go for edge-case fixtures