Skip to content

Allow AgentDefinitions to disable all or specific default tools on the Copilot SDK chat client #1263

Closed
Closed
Allow AgentDefinitions to disable all or specific default tools on the Copilot SDK chat client#1263
Assignees
JoshuaRowePhantom
Labels
bugSomething isn't workingdiagnosedRoot cause identifiedenhancementNew feature or requestnext-upqueuedIn the active work queue (tracked in work-queue.md)verified-locallyImplementation has been verified locally

Description

@JoshuaRowePhantom
JoshuaRowePhantom
opened on Aug 6, 2026

Summary

The GitHub Copilot CLI SDK chat agent path (CopilotSdkChatClient — the Copilot-SDK-backed chat client wired by the github-copilot and github-copilot-subagent providers in AgentFactory) currently has no mechanism to control which Copilot CLI built-in tools an agent gets. Every session created through BuildSessionConfig / BuildResumeSessionConfig inherits the SDK's default tool set (all built-ins). Restricted or safety-sensitive agents (sandboxed sub-agents, review-only agents, agents that should route exclusively to user-supplied MCP tools) cannot opt out, and there is no way to whitelist just the isolated set or to exclude specific tools.

This bug adds a github-cli-builtin-tools CustomTool entry on the AgentDefinition whose options object exposes:

  • One reusable tool-set selector applied to two independent slots — available-tools and excluded-tools — mapping natively onto SessionConfig.AvailableTools and SessionConfig.ExcludedTools on the pinned GitHub.Copilot.SDK 1.0.8. Selector entries accept bare built-in names (auto-prefixed to builtin:<name>) as well as explicit source-qualified patterns such as mcp:*, custom:*, builtin:*, or mcp:<wire-name> — which flow through to the SDK verbatim.
  • A client-mode field opting an agent into the SDK's CopilotClientMode.Empty (a client-construction-time property, NOT a session field). This is the canonical way to build "locked-down" agents that expose nothing by default and route only to what available-tools explicitly allows — see the worked "MCP-only" recipe below.

The reusable selector — three forms (post-simplification)

A BuiltinToolSet selector is a small JSON object with one of these shapes:

# Form Meaning
1 { "tools": ["*"] } The entire built-in tool set — the wildcard "*" is a normal member of tools
2 { "tools": ["tool1","tool2"] } Exactly the named tools
3 { "isolated": true } The isolated built-in tool set (GitHub.Copilot.BuiltInTools.Isolated)
4 { "tools": [] } The empty set (no tools)

The selector is essentially { tools: string[] } OR { isolated: true }. There is no longer a dedicated all boolean — the wildcard "*" collapses into the tools list and covers form #1 as an ordinary token. Nullability of the deserialised Tools property (see BuiltinToolSet record below) distinguishes form #4 (Tools = []) from form-#3-with-Tools-absent (Tools = null, Isolated = true).

Two slots — same selector shape applied to both

The github-cli-builtin-tools tool's options object has two optional slots, each of which — when present — is a BuiltinToolSet selector:

{
  "kind": "github-cli-builtin-tools",
  "available-tools": { /* selector */ },   // optional
  "excluded-tools":  { /* selector */ }    // optional
}

Owner-supplied examples showing the wildcard:

{ "kind": "github-cli-builtin-tools", "available-tools": { "tools": ["*"] } }
{ "kind": "github-cli-builtin-tools", "excluded-tools":  { "tools": ["*"] } }
{ "kind": "github-cli-builtin-tools", "excluded-tools":  { "isolated": true } }

Both slots may be present simultaneously. Absent slots leave the corresponding SDK field untouched (SDK default). Omitting the github-cli-builtin-tools entry entirely leaves both fields untouched, i.e. SDK default = all built-ins.

Precedence within a selector

Highest wins:

  1. tools (any list, including []) — most specific.
  2. isolated: true.
  3. Empty selector object / neither key present ⇒ rejected by validation (see "Validation" section below). Under the earlier design this was a logged warning.

The factory logs a warning if it detects both tools and isolated on the same selector, and applies tools > isolated.

Combined-slot semantics

Both slots are independent SDK fields on the underlying SessionConfig. The SDK's own filter applies AvailableTools (allow-list) first, then removes anything named in ExcludedTools. Documented equivalent behaviour: final built-in tool set = AvailableTools \ ExcludedTools (AND semantics).

Direct answers to owner questions

  • "Is there no such capability in the C# SDK?" — Yes there is. The pinned GitHub.Copilot.SDK 1.0.8 exposes:
    • SessionConfig.AvailableTools : IList<string> — allow-list of source-qualified tool names. null (unset) ⇒ SDK default (no filter). Empty list ⇒ no tools included.
    • SessionConfig.ExcludedTools : IList<string> — deny-list of source-qualified tool names.
    • GitHub.Copilot.BuiltInTools.Isolated : IReadOnlyList<string> — the canonical "isolated" built-in tool set. Observed values in 1.0.8: ask_user, task_complete, exit_plan_mode, task, read_agent, write_agent, list_agents, send_inbox, context_board, skill (10 entries).
    • GitHub.Copilot.ToolSet — a List<string>-like builder with AddBuiltIn(string), AddBuiltIn(IEnumerable<string>), AddMcp(string), AddCustom(string). Each method prepends a source prefix to the tool name: AddBuiltIn("shell") → "builtin:shell", AddBuiltIn("*") → "builtin:*", AddMcp("*") → "mcp:*".
    • ResumeSessionConfig exposes the identical AvailableTools / ExcludedTools / ExcludedBuiltInAgents properties.
  • "Is it possible we need a more recent version?" — No. Directory.Packages.props pins GitHub.Copilot.SDK 1.0.8, which already contains all of the above. No package bump required.
  • Does the SDK special-case the "*" wildcard? — Yes, but only in source-qualified form, i.e. "builtin:*" / "mcp:*" / "custom:*". A bare "*" in AvailableTools/ExcludedTools is explicitly rejected by the SDK: verified from a string literal embedded in GitHub.Copilot.SDK.dll 1.0.8:

    entry '*': there is no bare wildcard. Use \new ToolSet().AddBuiltIn("")`, `.AddMcp("")`, or `.AddCustom("")` to target a specific source.Confidence: **high** (message extracted from the shipping assembly;ToolSet.AddBuiltIn("")was executed and observed to produce"builtin:"). The owner-facing JSON schema still uses the ergonomic bare ""; **AgentFactorytransparently rewrites""→"builtin:"and every other name →"builtin:"when populating the SDK fields** (this tool kind's entire purpose is built-in tool control, so thebuiltin:prefix is applied uniformly). Equivalently,AgentFactorymay build the list vianew ToolSet().AddBuiltIn(name)` per name — same result.

Root Cause / Current State

1. Session configuration only adds user tools; it never gates SDK built-in defaults.

Phantom.Workspaces.Llm.Core/CopilotSdkChatClient.cs:

  • BuildSessionConfig (~L260–306): 
    var sessionConfig = new SessionConfig
{
    Model = effectiveModelId,
    Streaming = true,
    OnPermissionRequest = PermissionHandler.ApproveAll,   // ~L274
};
...
var tools = options?.Tools?.OfType<AIFunctionDeclaration>().ToList();   // ~L299
if (tools is { Count: > 0 })
{
    sessionConfig.Tools = tools;                                        // ~L302
}
  • BuildResumeSessionConfig (~L314–360): same shape.

Neither method ever sets AvailableTools / ExcludedTools, so the SDK falls back to its full default built-in set.

2. AgentFactory never reads a tool-policy entry from the AgentDefinition.

Phantom.Workspaces.Llm.Core/AgentFactory.cs:

  • CreateGitHubCopilotClient (~L798–841) constructs new CopilotSdkChatClient(modelId, displayName, gitHubToken, ...) at ~L829 with no tool-policy parameter.
  • CreateGitHubCopilotByokClient (~L847–895) — constructor call at ~L883.
  • Additional new CopilotSdkChatClient(...) sites at ~L933 and ~L989 are identical.
  • ExtractTools (~L148–151) returns (agent as PromptAgent)?.Tools ?? [] with no type filtering; entries flow into ToolsetFactory for MCP/agent-definition handling and are not forwarded to CopilotSdkChatClient.

3. AgentDefinition JSON schema — the generic tool subschema already accepts arbitrary properties.

Phantom.Workspaces.Llm.Core/JsonSchemas/AgentDefinition.json:

  • tools[] (~L129–143) is an anyOf union of mcpTool / tool / agentDefinitionTool.
  • The generic $defs/tool subschema (~L151–172) sets additionalProperties: true (~L153), which is why an unrecognised kind currently validates trivially.
  • allowedTools (~L269–274) lives inside mcpTool and gates MCP-server tools; it does not affect Copilot CLI built-in tools.
  • alwaysRequireApprovalTools / neverRequireApprovalTools only tune approval prompts for tools already exposed.

This design adds a dedicated $defs/githubCliBuiltinToolsTool subschema with kind: { const: "github-cli-builtin-tools" } and fully-typed available-tools, excluded-tools, and client-mode properties, and inserts it into tools[]'s anyOf before the generic $defs/tool. To keep the shape strict (so a malformed github-cli-builtin-tools entry cannot silently fall through to the permissive generic tool), the generic $defs/tool is amended to exclude this specific kind — see "Schema-level enforcement" below. This is the owner-preferred approach: the schema is the authoritative shape gate, so an incorrectly-generated definition fails at load/validate time rather than at first session build.

4. Copilot SDK 1.0.8 surface — verified by reflection and DLL string extraction.

Verified by loading %USERPROFILE%\.nuget\packages\github.copilot.sdk\1.0.8\lib\net8.0\GitHub.Copilot.SDK.dll. On both GitHub.Copilot.SessionConfig and GitHub.Copilot.ResumeSessionConfig:

Member Type Role
Tools ICollection<Microsoft.Extensions.AI.AIFunctionDeclaration> User-supplied custom tools (already used today)
AvailableTools IList<string> Allow-list of source-qualified tool names. null (unset) ⇒ no filter (SDK default). Empty list ⇒ no tools. Bare "*" is rejected.
ExcludedTools IList<string> Deny-list of source-qualified tool names. null (unset) ⇒ no exclusion.
ExcludedBuiltInAgents IList<string> Deny-list of built-in sub-agent names (background only)
OnPermissionRequest Func<PermissionRequest, ..., PermissionDecision> Per-invocation permission gate

Public GitHub.Copilot.BuiltInTools (static class): sole public static member is Isolated : IReadOnlyList<string> (10 entries: ask_user, task_complete, exit_plan_mode, task, read_agent, write_agent, list_agents, send_inbox, context_board, skill). There is no All / Default / Names enumeration — this does not matter for the current design, because the wildcard "builtin:*" covers the "everything" case natively via ExcludedTools.

Public GitHub.Copilot.ToolSet (implements IList<string>): constructor .ctor(), methods AddBuiltIn(string) → ToolSet, AddBuiltIn(IEnumerable<string>) → ToolSet, AddMcp(string) → ToolSet, AddCustom(string) → ToolSet. Runtime observation: new ToolSet().AddBuiltIn("*") yields the single element "builtin:*"; AddBuiltIn(BuiltInTools.Isolated) yields builtin:ask_user, builtin:task_complete, …. String literal embedded in the DLL:

entry '*': there is no bare wildcard. Use \new ToolSet().AddBuiltIn("")`, `.AddMcp("")`, or `.AddCustom("*")` to target a specific source.`

Also embedded (Empty-mode diagnostic):

Empty mode requires every session to explicitly opt into the tools it wants — e.g. \AvailableTools = new ToolSet().AddBuiltIn(BuiltInTools.Isolated)`.`

Together these confirm: (a) the SDK does natively understand "*" as a wildcard, but only source-qualified; (b) ToolSet is the recommended builder for the source-prefixed strings that AvailableTools/ExcludedTools accept.

5. AgentSchema.CustomTool deserialisation — typed subclass, not the open Options bag.

AgentSchema.CustomTool currently exposes Kind, Name, Description, ShorthandProperty, Connection, IList<Binding> Bindings, and an open IDictionary<string, object> Options bag (precedent: the chat-history CustomTool at AgentFactory.cs:581–586 reads Options["connection"] is IDictionary<string, object> connectionDict).

For this feature we prefer typed JSON deserialisation over the open Options bag (owner directive — the policy shape must be a first-class deserialisable type, not object fished out of a dictionary). Concretely: introduce a GitHubCliBuiltinToolsTool subclass of CustomTool with typed properties AvailableTools : BuiltinToolSet?, ExcludedTools : BuiltinToolSet?, ClientMode : CopilotClientMode, deserialised via a kind-discriminated System.Text.Json polymorphic converter on CustomTool (or, equivalently, an AgentSchema.ToolConverter that dispatches on kind to the right subclass and falls through to the base CustomTool for unknown kinds). BuiltinToolSet is itself a typed record — see the model section below — with [JsonPropertyName("tools")] IReadOnlyList<string>? Tools and [JsonPropertyName("isolated")] bool Isolated.

This has several concrete benefits:

  • The policy record is JSON-deserialisable end-to-end — no runtime unboxing of IDictionary<string, object> / IEnumerable<object>, no is IDictionary<string, object> checks, no .OfType<string>().ToArray() gymnastics. AgentFactory receives a strongly-typed GitHubCliBuiltinToolsTool and passes it (or a projected CopilotBuiltinToolPolicy derived from it) directly to CopilotSdkChatClient.
  • The typed subclass co-composes with the schema-level enforcement described above: the JSON schema constrains the wire format, System.Text.Json deserialisation constrains the CLR shape, and both agree on the same property names / types.
  • Discoverability — git grep GitHubCliBuiltinToolsTool finds every producer and consumer, unlike an untyped string key "github-cli-builtin-tools" in an Options bag.

The Options bag on the base CustomTool remains for genuinely open-ended kinds; new provider-specific kinds should follow the GitHubCliBuiltinToolsTool pattern (typed subclass + schema subschema).

Affected Files

File Change
Directory.Packages.props No version bump required. Pins GitHub.Copilot.SDK 1.0.8, which already exposes AvailableTools/ExcludedTools, BuiltInTools.Isolated, and ToolSet with source-qualified wildcards.
Phantom.Workspaces.Llm.Core/JsonSchemas/AgentDefinition.json Add a $defs/githubCliBuiltinToolsTool subschema with kind: { const: "github-cli-builtin-tools" }, typed available-tools / excluded-tools selectors ({ oneOf: [ {tools: string[]}, {isolated: bool} ] }), and a typed client-mode: { enum: ["empty","copilot-cli"] }. Insert it into tools[] anyOf before $defs/tool. Amend the generic $defs/tool to exclude kind == "github-cli-builtin-tools" (see "Schema-level enforcement") so a malformed entry cannot silently fall through to the permissive generic. Update the kind field's description to list the recognised kinds.
AgentSchema package Model change required. Introduce GitHubCliBuiltinToolsTool : CustomTool with typed properties AvailableTools : BuiltinToolSet?, ExcludedTools : BuiltinToolSet?, ClientMode : CopilotClientMode = CopilotCli. Introduce the typed record BuiltinToolSet { IReadOnlyList<string>? Tools; bool Isolated }. Wire a kind-discriminated System.Text.Json polymorphic converter (or extend the existing CustomTool converter) so "kind": "github-cli-builtin-tools" deserialises into GitHubCliBuiltinToolsTool and unknown kinds continue to deserialise to the base CustomTool (backwards-compatible).
Phantom.Workspaces.Llm.Core/AgentFactory.cs In CreateGitHubCopilotClient (~L798–841) and CreateGitHubCopilotByokClient (~L847–895) — and the sibling sites at ~L933 and ~L989 — find the single GitHubCliBuiltinToolsTool entry via promptAgent.Tools.OfType<GitHubCliBuiltinToolsTool>().SingleOrDefault(), project it to a CopilotBuiltinToolPolicy (source-qualification and auto-append rules applied here), and pass it to the CopilotSdkChatClient constructor. Skip / filter this subtype out of ExtractTools (~L148–151) so it isn't handed to ToolsetFactory.
Phantom.Workspaces.Llm.Core/CopilotSdkChatClient.cs Accept a CopilotBuiltinToolPolicy? (default null = SDK defaults) via constructor; apply it in BuildSessionConfig (~L260–306) and BuildResumeSessionConfig (~L314–360) as a native passthrough onto SessionConfig.AvailableTools and SessionConfig.ExcludedTools. Custom tools (SessionConfig.Tools) and MCP tools continue to flow through unchanged.
Phantom.Workspaces.Llm.Core.Tests/AgentFactoryTests.cs New tests (see below).
Phantom.Workspaces.Llm.Core.Tests/CopilotSdkChatClientTests.cs New tests asserting SessionConfig.AvailableTools / SessionConfig.ExcludedTools shape for each form and slot.
Phantom.Workspaces.Llm.Core.Tests/AgentDefinitionJsonSchema*Tests.cs Schema round-trip tests for the new $defs/githubCliBuiltinToolsTool subschema, plus negative tests confirming that malformed github-cli-builtin-tools entries do NOT fall through to the generic $defs/tool (the not amendment on $defs/tool is what closes that hole).
docs/design/github-copilot-provider-support.md Document the two slots, the shared selector, the three forms, the precedence rule, and the source-prefix rewrite.

Design / Fix (Canonical) — shared selector + two slots + native passthrough

Represent the built-in-tool policy as a CustomTool entry with kind: "github-cli-builtin-tools" whose options bag carries two independent selector slots — available-tools and excluded-tools — each a BuiltinToolSet selector. Map both onto the pinned SDK's SessionConfig.AvailableTools and SessionConfig.ExcludedTools, with the SDK's own wildcard "builtin:*" used natively when the selector says "*".

BuiltinToolSet selector — resolution table

A selector object is resolved to a ResolvedToolSet — either Absent ("selector not present"), or Concrete(IReadOnlyList<string>) (a possibly-empty list of bare built-in tool names; "*" is a permitted member meaning "all built-ins"):

Selector shape Resolved meaning
{ "tools": ["*"] } Concrete(["*"]) — the wildcard is a normal list member
{ "tools": ["a","b"] } Concrete(["a","b"])
{ "tools": ["*","x"] } Concrete(["*","x"]) — permitted; the SDK sees builtin:* and builtin:x
{ "isolated": true } Concrete(BuiltInTools.Isolated.ToArray())
{ "tools": [] } Concrete([]) — explicit empty
{ } / neither key set Absent (with warning)
(selector absent from slot) Absent

Presence check on Tools uses null-vs-empty (selector.Tools is { } tools succeeds for [] and fails for absent) so [] is meaningfully distinct from an absent tools key. Precedence when more than one key is set: tools > isolated.

Selector-to-SDK mapping — per slot

Applied identically to SessionConfig and ResumeSessionConfig. Bare names (no :) are rewritten to their source-qualified form by prepending builtin: (equivalent to calling new ToolSet().AddBuiltIn(name) per element). Source-qualified entries (containing : — e.g. mcp:*, custom:*, builtin:*, mcp:github-list_issues) flow through verbatim and suppress the custom:*/mcp:* auto-append (see rule above).

available-tools selector → SessionConfig.AvailableTools:

Selector AvailableTools
{ "tools": ["*"] } Leave unset (null) — see note below on why we do NOT populate ["builtin:*"] on the allow-list.
{ "tools": ["a","b"] } ["builtin:a", "builtin:b", "custom:*", "mcp:*"] — bare-only ⇒ custom:* and mcp:* auto-appended.
{ "tools": ["a","*"] } ["builtin:a", "builtin:*", "custom:*", "mcp:*"] — bare-only ⇒ auto-append still applies.
{ "tools": ["mcp:*"] } ["mcp:*"] — MCP-only (built-ins AND custom off). Source-qualified ⇒ no auto-append. See the "MCP-only agent" recipe.
{ "tools": ["mcp:*","custom:*"] } ["mcp:*","custom:*"] — MCP + custom only (built-ins off). Source-qualified ⇒ no auto-append.
{ "tools": ["mcp:github-list_issues"] } ["mcp:github-list_issues"] — a single MCP tool by its runtime canonical wire name. mcp:* matches ALL MCP servers; there is no server-scoped wildcard, so to restrict to one server list each tool individually.
{ "tools": ["a","mcp:*"] } ["builtin:a","mcp:*"] — mixed bare + source-qualified: bare "a" is prefixed; presence of the source-qualified mcp:* suppresses auto-append.
{ "isolated": true } new ToolSet().AddBuiltIn(BuiltInTools.Isolated).AddCustom("*").AddMcp("*") → ["builtin:ask_user", …, "custom:*", "mcp:*"] — isolated is a bare-form shorthand, so auto-append applies.
{ "tools": [] } ["custom:*", "mcp:*"] — no built-ins, but custom and MCP tools remain reachable (empty list is treated as the bare-only case).
(slot absent) Leave unset

CRITICAL — allow-list hides all other sources. SessionConfig.AvailableTools is a global allow-list across builtin:, custom:, and mcp: (SDK Types.cs:265-269, 3011-3015, 331-337; session.create always sets toolFilterPrecedence: "excluded" so ExcludedTools wins over AvailableTools). If a caller writes available-tools: { "tools": ["bash","edit"] } intending "restrict built-ins to bash/edit", a naïve mapping to AvailableTools = ["builtin:bash","builtin:edit"] would ALSO silently deny every custom tool and every MCP tool — including the agent's own MCP-server tools declared elsewhere in the same AgentDefinition. This is almost never the author's intent — this tool kit is called github-cli-builtin-tools and its purpose is to govern built-ins only.

Chosen policy — bare-name-triggered auto-append. The rule is:

  • Every element in available-tools.tools is a bare built-in name (matches ^[a-zA-Z0-9_-]+$ or is "*" — NO colon) ⇒ auto-append custom:* and mcp:* to the resulting AvailableTools. The caller clearly meant "govern built-ins only, leave the rest untouched".
  • ANY element is a source-qualified pattern (contains : — e.g. mcp:*, custom:*, builtin:*, mcp:github-list_issues) ⇒ honor exactly what the caller wrote; do NOT auto-append anything. Presence of an explicit source-qualified pattern is treated as the caller taking full responsibility for the cross-source allow-list.

This is what makes { "available-tools": { "tools": ["mcp:*"] } } mean strictly "MCP-only" (built-ins AND custom off) — see the worked "MCP-only agent" recipe below. The excluded-tools slot is unaffected by this rule (it is a deny-list — appending custom:*/mcp:* to an exclusion would break the agent).

Rejected alternatives: (b) strict pass-through as a global allow-list — technically faithful to the SDK, but almost certainly the wrong behaviour for this kit and produces silent, hard-to-debug tool-disappearance bugs; (c) refuse specific available-tools lists entirely and steer users to excluded-tools — too restrictive.

Ergonomic guidance (documented in the schema description and docs): for the common "disable a few built-ins" case, prefer excluded-tools (a deny-list). Because session.create sets toolFilterPrecedence: "excluded", ExcludedTools wins over AvailableTools, and a deny-list never accidentally hides other sources.

Note (unset-vs-explicit-["*"] decision on available-tools). For the sole-wildcard case { "tools": ["*"] }, leave AvailableTools unset (null) — SDK default = all sources permitted. This is why removing the dedicated all boolean is safe: { "tools": ["*"] } on available-tools and the previous { "all": true } both mean "SDK default", and both map to "leave unset". On excluded-tools, the wildcard flows through natively — see next table.

excluded-tools selector → SessionConfig.ExcludedTools:

Selector ExcludedTools
{ "tools": ["*"] } ["builtin:*"] — native SDK wildcard meaning "exclude all built-ins". No alias hack needed.
{ "tools": ["a","b"] } ["builtin:a", "builtin:b"]
{ "isolated": true } new ToolSet().AddBuiltIn(BuiltInTools.Isolated) → ["builtin:ask_user", …]
{ "tools": [] } new List<string>() (exclude nothing — no-op)
(slot absent) Leave unset

Combined slots: both slots may coexist. Final effective built-in tool set = AvailableTools \ ExcludedTools. Absent slots do not touch the corresponding SDK field.

Absent github-cli-builtin-tools entry: neither field touched ⇒ SDK default (all built-ins). Semantically equivalent to { "available-tools": { "tools": ["*"] } } with no excluded-tools slot.

Precedence within a selector (multiple keys set)

Highest wins:

  1. tools (any list, including []) — most specific.
  2. isolated: true.
  3. None set (empty selector object) ⇒ rejected by validation (see "Validation" section below).

JSON examples

// Include ALL built-in tools (= omitting the entry entirely)
{ "kind": "github-cli-builtin-tools",
  "available-tools": { "tools": ["*"] } }
// Include ONLY the named tools
{ "kind": "github-cli-builtin-tools",
  "available-tools": { "tools": ["read_agent", "list_agents"] } }
// Include the ISOLATED set
{ "kind": "github-cli-builtin-tools",
  "available-tools": { "isolated": true } }
// Include NO built-in tools (MCP-only agent)
{ "kind": "github-cli-builtin-tools",
  "available-tools": { "tools": [] } }
// Exclude ALL built-in tools (native wildcard — MCP+custom untouched)
{ "kind": "github-cli-builtin-tools",
  "excluded-tools": { "tools": ["*"] } }
// Exclude the ISOLATED set (keep everything else)
{ "kind": "github-cli-builtin-tools",
  "excluded-tools": { "isolated": true } }
// Exclude a specific named tool
{ "kind": "github-cli-builtin-tools",
  "excluded-tools": { "tools": ["shell"] } }
// COMBINED — start from the isolated set, then remove one tool
{ "kind": "github-cli-builtin-tools",
  "available-tools": { "isolated": true },
  "excluded-tools":  { "tools": ["exit_plan_mode"] } }

Worked recipe — "MCP-only agent" (disable all built-ins, keep only MCP server tools)

Users frequently want an agent that has no Copilot built-ins available and routes exclusively to the MCP servers the agent declares. There are two idiomatic expressions of this, and the semantic difference matters — both must be documented as first-class in the schema examples.

1. Allowlist form — strictly MCP-only (also drops custom/SDK tools)

{ "kind": "github-cli-builtin-tools",
  "available-tools": { "tools": ["mcp:*"] } }
  • Because SessionConfig.AvailableTools is a global allow-list across every source, setting it to ["mcp:*"] means only tools whose source-qualified name matches mcp:* are available. All builtin:* are off; all custom:* are off; only MCP tools remain.
  • Under the "bare-name-triggered auto-append" rule above, the presence of the source-qualified pattern mcp:* suppresses the custom:* auto-append — this is what makes the recipe actually "MCP-only" instead of "MCP + custom".
  • Use this form when the requirement is "the model can call only our MCP tools; nothing else exists".

2. Denylist form — turn off built-ins, keep MCP + any custom/SDK tools

{ "kind": "github-cli-builtin-tools",
  "excluded-tools": { "tools": ["builtin:*"] } }
  • ExcludedTools = ["builtin:*"] starts from the SDK default (everything available across all sources) and removes every builtin:* entry. MCP tools and custom tools survive. Because session.create always sets toolFilterPrecedence: "excluded", this is guaranteed to win over AvailableTools.
  • Use this form when the requirement is "keep everything we register (MCP + custom + user tools), just drop the SDK built-ins".

Guidance

  • "Only our MCP tools, nothing else" → allowlist form (available-tools: { "tools": ["mcp:*"] }).
  • "Keep everything we register except the built-ins" → denylist form (excluded-tools: { "tools": ["builtin:*"] }).
  • mcp:* matches all MCP servers configured for the agent. There is no server-scoped wildcard in the SDK. To restrict to one specific MCP server, enumerate each tool by its runtime canonical wire name (mcp:<wire-name>, e.g. mcp:github-list_issues) — this is derived from SDK sources ToolSet.cs (AddBuiltIn→"builtin:{name}", AddCustom→"custom:{name}", AddMcp→"mcp:{toolName}"; ValidateName allows * or ^[a-zA-Z0-9_-]+$) and Types.cs:265-269,331-337.
  • The canonical "locked-down" configuration pairs the allowlist form with client-mode: "empty" (next section):
{ "kind": "github-cli-builtin-tools",
  "client-mode": "empty",
  "available-tools": { "tools": ["mcp:*"] } }

client-mode — opt into CopilotClientMode.Empty (client-construction property)

A new option on the github-cli-builtin-tools CustomTool options bag opts an agent into the SDK's alternative client mode:

{ "kind": "github-cli-builtin-tools",
  "client-mode": "empty" | "copilot-cli",
  "available-tools": { /* required if client-mode == "empty" */ },
  "excluded-tools":  { /* optional */ } }

Default: "copilot-cli" (identical to today's behaviour).

SDK facts (verified in C:\dev\microsoft\copilot-sdk\dotnet\src\Types.cs:249-345):

  • enum CopilotClientMode { Empty, CopilotCli }. Default CopilotCli.
  • CRITICAL: Mode lives on CopilotClientOptions (CLIENT-level, per CopilotClient), NOT on SessionConfig / ResumeSessionConfig. Selecting Empty is therefore a property of how the CopilotClient backing this agent is constructed, not a per-session tweak. AgentFactory (and the client-construction / client-caching path — the four new CopilotSdkChatClient(...) sites at ~L829, ~L883, ~L933, ~L989) must build (or select from a pool) a CopilotClient whose CopilotClientOptions.Mode = Empty for such definitions. If the current architecture reuses one CopilotClient across multiple agents, that reuse must be keyed on Mode (and on BaseDirectory/SessionFs) so an Empty-mode agent never accidentally shares a CopilotCli-mode client.
  • In Empty mode the SDK requires:
    1. CopilotClientOptions.BaseDirectory or CopilotClientOptions.SessionFs set on the client ctor;
    2. AvailableTools supplied on every SessionConfig (Empty exposes no tools by default — see the DLL diagnostic string quoted above under §4);
    3. session.create always sets toolFilterPrecedence: "excluded" (so ExcludedTools still wins over AvailableTools);
    4. Safe ambient defaults are injected and COPILOT_DISABLE_KEYTAR=1 is set.

Placement rationale. Placing client-mode on the github-cli-builtin-tools CustomTool options (rather than as a top-level AgentDefinition field or a separate github-copilot-client tool kind) is consistent with the rest of this design: this tool kit is already the single Copilot-SDK-provider-specific configuration surface on the agent, and client-mode and available-tools are inextricably linked (Empty mode requires available-tools). Keeping them in the same JSON object keeps the interlock local, validatable in one pass, and discoverable by anyone reading the tool entry. A top-level AgentDefinition field would split the interlock across two schema locations and cross the "tool policy lives in tools[]" invariant (see Alternative D in Considered / Background).

Fail-fast validation for client-mode (in addition to the general validation table below):

Rule Detail On violation
client-mode is a string of "empty" or "copilot-cli" Case-insensitive, but the canonical serialised form is lowercase-kebab. false/null/other strings rejected. Reject with 'client-mode' must be one of "empty" or "copilot-cli"; got <actual>.
Empty mode ⇒ available-tools is present and non-empty Mirrors the SDK's own requirement. available-tools: { "tools": [] } counts as empty for this rule (Empty mode with zero tools would be an unusable agent). Reject with 'client-mode: empty' requires 'available-tools' to be present and non-empty; Empty mode exposes no tools by default so every session must opt in.
Empty mode ⇒ BaseDirectory or SessionFs is configured AgentFactory's client-construction path is responsible for ensuring one of these is set on the CopilotClientOptions used to build the CopilotClient. If neither can be resolved from the agent's provider/account context, reject at agent-load time. Reject with 'client-mode: empty' requires the underlying CopilotClient to be constructed with BaseDirectory or SessionFs; neither could be resolved for this AgentDefinition.

Natural pairing. Empty mode + available-tools: { "tools": ["mcp:*"] } (see recipe above) is the canonical "locked-down, MCP-only" configuration: the SDK opens the client in Empty mode (nothing exposed by default), and each session opts into exactly mcp:*. Both requirements — non-empty available-tools and MCP-only surface — are satisfied simultaneously.

AgentSchema types — JSON-deserialisable

// AgentSchema — deserialisable via System.Text.Json. Wire names use kebab-case
// via [JsonPropertyName]; the schema's typed shape (see "Schema-level enforcement")
// is the authoritative gate on which combinations are legal.
public sealed record BuiltinToolSet(
    [property: JsonPropertyName("tools")]    IReadOnlyList<string>? Tools,
    [property: JsonPropertyName("isolated")] bool Isolated);

// Typed subclass of AgentSchema.CustomTool. Deserialised by a kind-discriminated
// polymorphic converter — an entry with "kind": "github-cli-builtin-tools" lands here;
// unknown kinds still deserialise to the base CustomTool for backwards compatibility.
public sealed class GitHubCliBuiltinToolsTool : CustomTool
{
    public const string KindName = "github-cli-builtin-tools";

    [JsonPropertyName("available-tools")] public BuiltinToolSet? AvailableTools { get; init; }
    [JsonPropertyName("excluded-tools")]  public BuiltinToolSet? ExcludedTools  { get; init; }
    [JsonPropertyName("client-mode")]     public CopilotClientMode ClientMode   { get; init; } = CopilotClientMode.CopilotCli;
}

CopilotBuiltinToolPolicy model — projected view for CopilotSdkChatClient

/// <summary>
/// Projected view of a GitHubCliBuiltinToolsTool after applying source-qualification
/// and the auto-append rule. Each list is applied verbatim to its corresponding SDK
/// field. A null list means "leave that SDK field unset". List elements are already
/// source-qualified (e.g. "builtin:shell", "builtin:*", "mcp:*", "mcp:github-list_issues").
/// ClientMode is applied at CopilotClient construction time (CopilotClientOptions.Mode),
/// NOT at session-config build time.
/// </summary>
public sealed record CopilotBuiltinToolPolicy(
    IReadOnlyList<string>? AvailableTools,
    IReadOnlyList<string>? ExcludedTools,
    GitHub.Copilot.CopilotClientMode ClientMode);   // default CopilotCli

AgentFactory wiring — typed projection to CopilotBuiltinToolPolicy

Because GitHubCliBuiltinToolsTool is a strongly-typed subclass of CustomTool, AgentFactory retrieves it directly with no dictionary unboxing:

// A selector resolves to one of two outcomes.
private abstract record ResolvedToolSet
{
    public sealed record Absent : ResolvedToolSet;
    /// <summary>Bare tool names as authored (may include the wildcard "*").</summary>
    public sealed record Concrete(IReadOnlyList<string> Tools) : ResolvedToolSet;
}

// Resolve a typed BuiltinToolSet into a ResolvedToolSet. Precedence: tools > isolated.
// Empty-selector / unknown-key / wrong-type cases are already rejected at schema-validate
// time (see "Schema-level enforcement"), so this method only handles well-formed input.
static ResolvedToolSet ResolveBuiltinToolSet(BuiltinToolSet? selector, string slotName, ILogger? log)
{
    if (selector is null) return new ResolvedToolSet.Absent();

    if (selector.Tools is { } tools && selector.Isolated)
        log?.LogWarning(
            "github-cli-builtin-tools {Slot}: both 'tools' and 'isolated' set; " +
            "applying precedence tools > isolated.", slotName);

    if (selector.Tools is { } t) return new ResolvedToolSet.Concrete(t);
    if (selector.Isolated)       return new ResolvedToolSet.Concrete(GitHub.Copilot.BuiltInTools.Isolated.ToArray());
    return new ResolvedToolSet.Absent();   // defensive; schema rejects {}
}

// Source-qualify: bare names and the wildcard become "builtin:<name>" / "builtin:*".
// Source-qualified names (containing ':') are passed through verbatim.
static IReadOnlyList<string> ToSdkBuiltinList(IReadOnlyList<string> names)
{
    var set = new GitHub.Copilot.ToolSet();
    foreach (var n in names)
    {
        if (n.Contains(':')) set.Add(n);      // already source-qualified — verbatim
        else                 set.AddBuiltIn(n);
    }
    return set.ToList();
}

// Build the AvailableTools list from a resolved built-in selector, applying the
// bare-name-triggered auto-append rule. When ANY element is source-qualified
// (contains ':'), the caller has taken explicit responsibility for the cross-source
// allow-list, so no auto-append happens.
static IReadOnlyList<string> ToSdkAvailableList(IReadOnlyList<string> names)
{
    var anySourceQualified = names.Any(n => n.Contains(':'));
    var set = new GitHub.Copilot.ToolSet();
    foreach (var n in names)
    {
        if (n.Contains(':')) set.Add(n);
        else                 set.AddBuiltIn(n);
    }
    if (!anySourceQualified)
    {
        set.AddCustom("*");
        set.AddMcp("*");
    }
    return set.ToList();
}

static CopilotBuiltinToolPolicy? ExtractBuiltinToolPolicy(AgentDefinition agent, ILogger? log)
{
    if (agent is not PromptAgent promptAgent || promptAgent.Tools is null) return null;

    // Direct, typed lookup — no dictionary spelunking.
    var tool = promptAgent.Tools.OfType<GitHubCliBuiltinToolsTool>().SingleOrDefault();
    if (tool is null) return null;

    // Cross-field validation (e.g. client-mode: empty ⇒ available-tools required) is
    // performed here — the schema constrains individual field shapes but does not
    // express inter-field interlocks. See "Cross-field runtime checks" below.
    ValidateClientModeInterlock(tool);

    var available = ResolveBuiltinToolSet(tool.AvailableTools, "available-tools", log);
    var excluded  = ResolveBuiltinToolSet(tool.ExcludedTools,  "excluded-tools",  log);

    IReadOnlyList<string>? availableList = null;
    if (available is ResolvedToolSet.Concrete cAvail)
    {
        // Sole-wildcard optimisation: {"tools":["*"]} means "SDK default, all sources".
        if (cAvail.Tools.Count == 1 && cAvail.Tools[0] == "*")
            availableList = null;
        else
            availableList = ToSdkAvailableList(cAvail.Tools);
    }

    IReadOnlyList<string>? excludedList = excluded is ResolvedToolSet.Concrete cExcl
        ? ToSdkBuiltinList(cExcl.Tools)                       // may be empty; "*" → "builtin:*"
        : null;

    return new CopilotBuiltinToolPolicy(availableList, excludedList, tool.ClientMode);
}

// At each CopilotSdkChatClient construction site (~L829 / ~L883 / ~L933 / ~L989):
var builtinPolicy = ExtractBuiltinToolPolicy(agent, services?.LoggerFactory?.CreateLogger(...));

var client = new CopilotSdkChatClient(
    modelId, displayName, gitHubToken, services?.LoggerFactory,
    queueManager: queueManager,
    modelOptions: model.Options,
    subAgentChatRegistry: subAgentChatRegistry,
    accountUpsertService: services?.AccountUpsertService,
    slashCommandRegistry: services?.SlashCommandRegistry as SlashCommands.ISlashCommandRegistry,
    builtinToolPolicy: builtinPolicy);   // null ⇒ SDK defaults

ExtractTools (~L148–151) is updated to skip entries of concrete type GitHubCliBuiltinToolsTool so ToolsetFactory never sees them.

Schema-level enforcement (authoritative gate)

The AgentDefinition.json schema is the authoritative shape gate for github-cli-builtin-tools entries. This is the owner-preferred approach: an incorrectly-generated definition must fail JSON-schema validation at agent load, not silently mis-apply, and other tool kinds must keep their own permissive schemata.

How it composes with tools[] anyOf — no fragility. The trick that avoids the previously-feared anyOf / if/then ordering problem is a two-part change that leaves other tool kinds' schemata untouched:

  1. Add $defs/githubCliBuiltinToolsTool — a fully-typed subschema:

    "githubCliBuiltinToolsTool": {
  "type": "object",
  "required": ["kind"],
  "additionalProperties": false,
  "properties": {
    "name":        { "type": "string" },
    "kind":        { "const": "github-cli-builtin-tools" },
    "description": { "type": "string" },
    "bindings":    { "type": "array", "items": { "type": "object", "additionalProperties": true } },
    "available-tools": { "$ref": "#/$defs/builtinToolSet" },
    "excluded-tools":  { "$ref": "#/$defs/builtinToolSet" },
    "client-mode":     { "enum": ["empty", "copilot-cli"] }
  },
  "allOf": [
    {
      "if":   { "properties": { "client-mode": { "const": "empty" } }, "required": ["client-mode"] },
      "then": { "required": ["available-tools"],
                "properties": { "available-tools": { "not": { "properties": { "tools": { "maxItems": 0 } } } } } }
    }
  ]
},
"builtinToolSet": {
  "type": "object",
  "additionalProperties": false,
  "oneOf": [
    { "required": ["tools"],    "properties": { "tools":    { "type": "array", "items": {
          "type": "string",
          "pattern": "^(\\*|[a-zA-Z0-9_-]+|(builtin|custom|mcp):(\\*|[a-zA-Z0-9_-]+))$"
      } } } },
    { "required": ["isolated"], "properties": { "isolated": { "const": true } } }
  ]
}

  2. Insert githubCliBuiltinToolsTool first in tools[]'s anyOf, and amend the generic $defs/tool to exclude this specific kind. The generic tool's own definition is unchanged for every other kind; it merely gains a not constraint saying "I am not a github-cli-builtin-tools":

    // tools[] anyOf:
{ "anyOf": [
    { "$ref": "#/$defs/mcpTool" },
    { "$ref": "#/$defs/githubCliBuiltinToolsTool" },
    { "$ref": "#/$defs/tool" },                       // now excludes the discriminated kind
    { "$ref": "#/$defs/agentDefinitionTool" }
] }

// $defs/tool — additive amendment:
"tool": {
  "type": "object",
  "additionalProperties": true,
  "properties": { /* unchanged */ },
  "not": {
    "properties": { "kind": { "const": "github-cli-builtin-tools" } },
    "required":   ["kind"]
  }
}

    This closes the "silent fall-through to the permissive generic" hole: a malformed github-cli-builtin-tools entry cannot satisfy $defs/tool (blocked by the not) and cannot satisfy $defs/githubCliBuiltinToolsTool (blocked by additionalProperties: false and the typed oneOf on the selector), so anyOf fails and validation loudly rejects the definition. Other kinds keep the permissive additionalProperties: true behaviour of the generic tool exactly as today.

What the schema enforces (no runtime duplication):

  • Only available-tools, excluded-tools, and client-mode are permitted top-level options (additionalProperties: false).
  • Each selector is a builtinToolSet — exactly one of tools or isolated (oneOf), no other keys (additionalProperties: false), isolated must be the literal true (false is meaningless).
  • tools is a string array; each element matches the wire-name pattern ("*" or ^[a-zA-Z0-9_-]+$ or a source-qualified ^(builtin|custom|mcp):(\*|[a-zA-Z0-9_-]+)$).
  • client-mode is one of "empty" / "copilot-cli".
  • Cross-field interlock: client-mode: "empty" requires available-tools present and non-empty. Expressed via the if/then inside the subschema's allOf.

Cross-field runtime checks (belt-and-suspenders). A small ValidateClientModeInterlock(GitHubCliBuiltinToolsTool) in AgentFactory re-checks the client-mode: empty ⇒ available-tools non-empty interlock (redundant with the schema, kept for defence-in-depth against callers that construct AgentDefinition in-memory and bypass schema validation) and additionally enforces the environmental precondition that the SDK does not express: Empty mode requires BaseDirectory or SessionFs to be resolvable at CopilotClient construction time. These are the only rules that live in code — every wire-shape rule lives in the schema.

Known-name allow-list — deliberately NOT enforced (schema or runtime). Neither layer hard-codes a known set of built-in tool names beyond the wire-name pattern above. Rationale: the SDK classifies built-in tool names at runtime (in session.create) and the set of built-ins can change between SDK versions (GitHub.Copilot.SDK may add new tools in future 1.0.x releases). Hard-coding a name allow-list would cause valid definitions to be rejected after an SDK bump. If a user misspells a tool name, the SDK itself will refuse to route calls to it — a benign failure mode that surfaces on first call.

Validation error mapping. Because the schema does the shape enforcement, callers get standard JSON-schema errors with JSON-pointer paths (e.g. /tools/2/available-tools/tools/0). AgentDefinitionValidationException should surface those verbatim (as elsewhere in the loader) so LLM-authored definitions receive the same actionable feedback humans do.

CopilotSdkChatClient enforcement — native passthrough

Both BuildSessionConfig (~L260–306) and BuildResumeSessionConfig (~L314–360) apply the same block after constructing sessionConfig. ResumeSessionConfig shares the identical property surface, so the code is symmetric:

if (_builtinToolPolicy is { } policy)
{
    if (policy.AvailableTools is { } available)
        sessionConfig.AvailableTools = available.ToList();   // [] or populated (source-qualified)
    if (policy.ExcludedTools  is { } excluded)
        sessionConfig.ExcludedTools  = excluded.ToList();
}
// Absent policy or absent slot ⇒ leave SDK field unset ⇒ SDK default applies.
// OnPermissionRequest remains PermissionHandler.ApproveAll — the SDK's own
// AvailableTools/ExcludedTools filter runs before any built-in reaches the model.
// SessionConfig.Tools (user custom tools) and MCP tools are unaffected.

Considered / Background (NOT chosen)

Superseded — earlier four-form selector with a dedicated all boolean

Before the SDK's source-qualified wildcard "builtin:*" was verified, the canonical selector had four forms with a dedicated boolean:

# Form Meaning
1 { "all": true } The entire built-in tool set
2 { "tools": [...] } Exactly the named tools
3 { "isolated": true } The isolated built-in tool set
4 { "tools": [] } The empty set

Precedence was tools > isolated > all. The all boolean was collapsed into the tools list once the wildcard "*" was confirmed: form #1 { "all": true } is now expressed as { "tools": ["*"] }, and form #1 on available-tools continues to map to "leave unset" (the wildcard's canonical mapping — see design note).

Superseded — excluded-tools: { "all": true } alias hack

Under the four-form design, excluded-tools: { "all": true } had no clean SDK translation because BuiltInTools exposes no enumeration of all built-in tool names. The workaround was to treat it as a documented alias for available-tools: { "tools": [] }, setting SessionConfig.AvailableTools = new List<string>() on the available slot and leaving ExcludedTools untouched — with a warning about the aliasing and an edge-case rule for what to do if both slots were present.

This hack is now retired. With the source-qualified wildcard confirmed, { "excluded-tools": { "tools": ["*"] } } maps natively to ExcludedTools = ["builtin:*"] — no cross-slot rewrite, no warning, no lost expressive power (MCP and custom tools remain reachable when only built-ins are excluded, which the old alias did not preserve). This is a strict semantic improvement over the alias.

Alternative A — earlier include-only shape (include-all / include / include-isolated)

An even earlier canonical design expressed the policy as an include-only allow-list on the CustomTool options bag:

  • options.include-all: bool ⇒ AvailableTools = null (SDK default, all built-ins).
  • options.include: string[] ⇒ AvailableTools = [...] (empty list = no built-ins).
  • options.include-isolated: bool ⇒ AvailableTools = BuiltInTools.Isolated.ToList().

Rejected in favour of the reusable-selector-plus-two-slots shape because it could not express "exclude a specific tool from the SDK default set" without listing every remaining built-in by name (fragile as the SDK's default set evolves), used three parallel keys whose semantics were not reusable in an excluded-tools slot, and did not compose across allow + deny.

Alternative B — earlier disableAll / disabled (exclude-based) schema

An even earlier design expressed the policy as two exclusion-shaped properties in the options bag: options.disableAll: bool and options.disabled: string[]. Rejected because it only expressed the deny-list side; there was no natural way to express the isolated set, no way to allow-list a small set of tools, and no way to compose allow + deny.

Alternative C — wrapper-layer OnPermissionRequest clamp + SessionConfig.Tools withholding

An early proposal (from when the SDK surface was believed to lack native disable knobs) enforced the policy at the wrapper layer by composing OnPermissionRequest to deny disabled built-in tool names. Rejected because pinned SDK 1.0.8 exposes AvailableTools / ExcludedTools (with source-qualified wildcards) directly (see §4), which the SDK applies before any tool reaches the model. An OnPermissionRequest deny still lets the model attempt the call and only refuses at invocation time. Preserved as the documented fallback should a future SDK regress or rename these fields.

Alternative D — top-level AgentDefinition fields

An earlier proposal expressed the policy as top-level AgentDefinition fields (disableDefaultTools: bool, disabledDefaultTools: string[]). Rejected because tool policy is naturally scoped to tools[]; expressing it as a tool entry keeps AgentDefinition's top-level surface small and consistent with how MCP allow-lists and agent-definition tools are modelled. A kind-discriminated tool entry also composes cleanly with future provider-specific tool policies (e.g. an analogous openai-builtin-tools kind).

Alternative E — dedicated $defs/githubCliBuiltinToolsTool discriminated subschema (now the chosen approach)

An intermediate proposal added a new $defs/githubCliBuiltinToolsTool subschema (with kind: { const: "github-cli-builtin-tools" } and typed properties) and inserted it into the tools[] anyOf before the generic tool. It was briefly rejected out of concern that JSON Schema anyOf / if/then composition would be fragile in downstream validators and would silently fall through to the permissive generic $defs/tool.

Reinstated (owner directive — see "Schema-level enforcement" above). The fall-through concern is closed by amending the generic $defs/tool with a not constraint that excludes this specific kind, so anyOf cannot silently accept a malformed github-cli-builtin-tools entry via the generic branch. Every other kind's schema is unchanged. The typed subschema plus the not amendment together make the JSON schema the authoritative shape gate; runtime validation is reduced to cross-field interlocks that the schema cannot express (see "Cross-field runtime checks").

The considered-but-not-chosen designs are preserved so the alternatives aren't lost.

Expected Tests

Follow the Subject_Scenario_ExpectedOutcome naming already used in Phantom.Workspaces.Llm.Core.Tests (see AgentFactoryTests.ConfigureChatOptions_MapsThinkingToReasoningEffort and AgentFactoryTests.CreateChatClient_GitHubCopilotProvider_ReturnsCopilotSdkClient).

Test Name Class What It Verifies
AgentFactory_AvailableToolsWildcard_LeavesAvailableToolsUnset AgentFactoryTests { "available-tools": { "tools": ["*"] } } produces a policy whose AvailableTools is null, so SessionConfig.AvailableTools is not touched (SDK default = all sources). Regression guard for the "do not populate ["builtin:*"] alone" rule.
AgentFactory_AvailableToolsList_SetsAvailableTools AgentFactoryTests { "available-tools": { "tools": ["read_agent","list_agents"] } } sets SessionConfig.AvailableTools == ["builtin:read_agent","builtin:list_agents","custom:*","mcp:*"] in exact order — the trailing custom:* and mcp:* are auto-appended to preserve non-built-in tools.
AgentFactory_AvailableToolsSpecificList_PreservesCustomAndMcp AgentFactoryTests Regression guard for the allow-list-hides-all-sources caveat: with available-tools: { "tools": ["bash","edit"] } and an agent that also declares an MCP tool and a custom tool elsewhere in its AgentDefinition.tools[], both custom:* and mcp:* are present in SessionConfig.AvailableTools alongside builtin:bash / builtin:edit, so the agent's own custom/MCP tools remain reachable.
AgentFactory_AvailableToolsWildcardMixed_PassesThrough AgentFactoryTests { "available-tools": { "tools": ["*","read_agent"] } } sets SessionConfig.AvailableTools == ["builtin:*","builtin:read_agent","custom:*","mcp:*"] — mixed lists containing "*" are passed through verbatim (author-explicit intent) with custom:* / mcp:* still appended.
AgentFactory_AvailableToolsIsolated_SetsAvailableToolsToIsolatedSet AgentFactoryTests { "available-tools": { "isolated": true } } sets SessionConfig.AvailableTools equal to new ToolSet().AddBuiltIn(BuiltInTools.Isolated).AddCustom("*").AddMcp("*") (i.e. ["builtin:ask_user", …, "custom:*", "mcp:*"]), element-wise.
AgentFactory_AvailableToolsEmpty_SetsAvailableToolsToCustomAndMcpOnly AgentFactoryTests { "available-tools": { "tools": [] } } sets SessionConfig.AvailableTools == ["custom:*","mcp:*"] — no built-ins, but custom and MCP tools remain reachable.
AgentFactory_ExcludedToolsWildcard_SetsExcludedToolsToBuiltinStar AgentFactoryTests { "excluded-tools": { "tools": ["*"] } } sets SessionConfig.ExcludedTools == ["builtin:*"] — native SDK wildcard, no alias, AvailableTools untouched.
AgentFactory_ExcludedToolsList_SetsExcludedTools AgentFactoryTests { "excluded-tools": { "tools": ["shell"] } } sets SessionConfig.ExcludedTools == ["builtin:shell"].
AgentFactory_ExcludedToolsIsolated_SetsExcludedToolsToIsolatedSet AgentFactoryTests { "excluded-tools": { "isolated": true } } sets SessionConfig.ExcludedTools equal to new ToolSet().AddBuiltIn(BuiltInTools.Isolated).
AgentFactory_ExcludedToolsEmpty_LeavesExcludedToolsEmptyList AgentFactoryTests { "excluded-tools": { "tools": [] } } sets SessionConfig.ExcludedTools to a non-null, empty list (no-op deny).
AgentFactory_AvailableAndExcludedCombined_AppliesBoth AgentFactoryTests { "available-tools": { "isolated": true }, "excluded-tools": { "tools": ["exit_plan_mode"] } } sets AvailableTools = ["builtin:ask_user", …, "custom:*", "mcp:*"] and ExcludedTools = ["builtin:exit_plan_mode"] on SessionConfig.
AgentFactory_NoBuiltinToolsEntry_LeavesDefaults AgentFactoryTests Omitting the github-cli-builtin-tools entry yields a null policy so SessionConfig.AvailableTools and SessionConfig.ExcludedTools remain unset (regression guard).
AgentFactory_SelectorBothToolsAndIsolated_AppliesPrecedence AgentFactoryTests When both tools and isolated are set on the same selector, precedence tools > isolated is applied and a warning is logged (this remains a warn, not a reject).
AgentFactory_BuiltinToolsUnknownOptionKey_SchemaRejects AgentDefinitionJsonSchemaTests Loading an AgentDefinition whose github-cli-builtin-tools entry contains an unrecognised key (e.g. { "kind": "github-cli-builtin-tools", "disabledTools": [...] }) fails JSON-schema validation with a message naming the offending path. Enforced by additionalProperties: false on $defs/githubCliBuiltinToolsTool plus the not amendment on $defs/tool that prevents fall-through. Fail-fast LLM-safety guard.
AgentFactory_BuiltinToolsSelectorNeitherToolsNorIsolated_Throws AgentFactoryTests An empty selector object (e.g. available-tools: {}) throws with a message stating that the selector must set either tools or isolated. Replaces the earlier "empty selector is ignored with warning" behaviour — reject is safer for LLM-authored definitions.
AgentFactory_BuiltinToolsSelectorUnknownKey_Throws AgentFactoryTests A selector containing an unrecognised key (e.g. available-tools: { "include": ["x"] }) throws naming the offending key.
AgentFactory_BuiltinToolsSelectorWithBadToolName_Throws AgentFactoryTests A tools list containing a name with disallowed characters (e.g. "builtin:shell" — already-prefixed — or "foo bar") throws with a message quoting the offending name and the required regex. Enforces ToolSet.ValidateName semantics up-front.
AgentFactory_BuiltinToolsSelectorToolsNotArray_Throws AgentFactoryTests If tools is present but not a JSON array (e.g. a string), throws with a message stating tools must be an array of strings.
AgentFactory_BuiltinToolsSelectorToolsElementNotString_Throws AgentFactoryTests If tools contains a non-string element (e.g. ["shell", 42]), throws naming the offending index and its actual type.
AgentFactory_BuiltinToolsIsolatedNonBool_Throws AgentFactoryTests If isolated is present but not a boolean (e.g. "true", 1), throws with a message stating isolated must be a boolean.
AgentFactory_BuiltinToolsSlotNotObject_Throws AgentFactoryTests If a slot's value is not a JSON object (e.g. available-tools: ["shell"]), throws with a message stating the slot must be a selector object.
AgentFactory_BuiltinToolsEntry_IsNotForwardedToToolsetFactory AgentFactoryTests The github-cli-builtin-tools entry is filtered out of ExtractTools (~L148–151) and never reaches ToolsetFactory.
AgentFactory_BuiltinToolsEntry_OnNonCopilotProvider_IsIgnoredOrRejected AgentFactoryTests Non-Copilot providers ignore the entry (or the loader/factory rejects it), per the chosen semantics.
CopilotSdkChatClient_BuildSessionConfig_NoPolicy_LeavesFieldsUnset CopilotSdkChatClientTests With builtinToolPolicy == null, neither AvailableTools nor ExcludedTools is touched.
CopilotSdkChatClient_BuildSessionConfig_AvailablePolicyList_SetsAvailableTools CopilotSdkChatClientTests With policy AvailableTools = ["builtin:a","builtin:b","custom:*","mcp:*"], sessionConfig.AvailableTools equals exactly that list.
CopilotSdkChatClient_BuildSessionConfig_AvailablePolicyCustomMcpOnly_KeepsNonBuiltinTools CopilotSdkChatClientTests With policy AvailableTools = ["custom:*","mcp:*"] (the empty-built-in-tools case), sessionConfig.AvailableTools equals exactly that list — asserts the built-in-empty path still passes custom/MCP through.
CopilotSdkChatClient_BuildSessionConfig_ExcludedPolicyStar_SetsExcludedToolsToBuiltinStar CopilotSdkChatClientTests With policy ExcludedTools = ["builtin:*"], sessionConfig.ExcludedTools equals ["builtin:*"].
CopilotSdkChatClient_BuildSessionConfig_BothSlotsSet_AppliesBoth CopilotSdkChatClientTests With both AvailableTools and ExcludedTools non-null on the policy, both are applied to sessionConfig.
CopilotSdkChatClient_BuildResumeSessionConfig_AppliesSamePolicy CopilotSdkChatClientTests The resume path (~L314–360) applies the same AvailableTools / ExcludedTools mapping as the fresh-session path across all forms.
AgentDefinitionJsonSchema_GithubCliBuiltinToolsEntry_RoundTrips AgentDefinitionJsonSchemaTests The schema accepts and round-trips a tools[] entry with kind: "github-cli-builtin-tools" and any combination of available-tools / excluded-tools selector objects — validating against the new $defs/githubCliBuiltinToolsTool subschema and deserialising into a strongly-typed GitHubCliBuiltinToolsTool instance whose AvailableTools / ExcludedTools are BuiltinToolSet records (not IDictionary<string, object>).
AgentDefinitionToolExtractor_GithubCliBuiltinToolsEntry_IsExtractedAsTypedSubclass AgentDefinitionToolExtractorTests The extractor surfaces the entry as a GitHubCliBuiltinToolsTool (subclass of AgentSchema.CustomTool), exposing typed AvailableTools, ExcludedTools, and ClientMode properties for downstream consumption by AgentFactory.
AgentFactory_AvailableToolsMcpStar_MapsToMcpStarWithNoAutoAppend AgentFactoryTests The MCP-only allowlist recipe: { "available-tools": { "tools": ["mcp:*"] } } produces SessionConfig.AvailableTools == ["mcp:*"] — no custom:* auto-appended. Regression guard for the source-qualified-suppresses-auto-append rule.
AgentFactory_AvailableToolsMcpStarAndCustomStar_PassesThroughVerbatim AgentFactoryTests { "available-tools": { "tools": ["mcp:*","custom:*"] } } produces SessionConfig.AvailableTools == ["mcp:*","custom:*"] — verbatim; no auto-append; no builtin:* injected.
AgentFactory_AvailableToolsMixedBareAndSourceQualified_PrefixesBareOnly AgentFactoryTests { "available-tools": { "tools": ["a","mcp:*"] } } produces SessionConfig.AvailableTools == ["builtin:a","mcp:*"] — bare "a" is builtin:-prefixed; presence of a source-qualified pattern suppresses the custom:*/mcp:* auto-append.
AgentFactory_AvailableToolsBareNamesOnly_AutoAppendsCustomAndMcp AgentFactoryTests Existing rule preserved: bare-only { "available-tools": { "tools": ["a","b"] } } produces ["builtin:a","builtin:b","custom:*","mcp:*"]. Regression guard against accidental suppression when no source-qualified pattern is present.
AgentFactory_ExcludedToolsBuiltinStar_MapsToBuiltinStarVerbatim AgentFactoryTests The denylist recipe: { "excluded-tools": { "tools": ["builtin:*"] } } produces SessionConfig.ExcludedTools == ["builtin:*"] (source-qualified passes through the excluded slot unchanged; no builtin: re-prefixing).
AgentFactory_ExcludedToolsMcpSpecificWireName_PassesThroughVerbatim AgentFactoryTests { "excluded-tools": { "tools": ["mcp:github-list_issues"] } } produces SessionConfig.ExcludedTools == ["mcp:github-list_issues"].
AgentFactory_ClientModeEmpty_WithAvailableTools_ConstructsClientWithEmptyMode AgentFactoryTests { "client-mode": "empty", "available-tools": { "tools": ["mcp:*"] } } yields a CopilotBuiltinToolPolicy with ClientMode = CopilotClientMode.Empty, and the CopilotClient backing the resulting CopilotSdkChatClient is constructed with CopilotClientOptions.Mode = Empty (and BaseDirectory or SessionFs set).
AgentFactory_ClientModeCopilotCli_IsDefault AgentFactoryTests Omitting client-mode or setting it to "copilot-cli" yields ClientMode = CopilotClientMode.CopilotCli. Regression guard for the default.
AgentFactory_ClientModeEmpty_WithoutAvailableTools_Throws AgentFactoryTests { "client-mode": "empty" } with no available-tools (or with available-tools: { "tools": [] }) throws with a message stating that Empty mode requires a non-empty available-tools.
AgentFactory_ClientModeEmpty_WithoutBaseDirectoryOrSessionFs_Throws AgentFactoryTests Empty mode requires BaseDirectory or SessionFs; if neither can be resolved from the agent's provider/account context, agent-load throws with a message naming the missing prerequisite.
AgentFactory_ClientMode_InvalidValue_Throws AgentFactoryTests { "client-mode": "off" } (or any string other than empty/copilot-cli, or a non-string type) throws with a message enumerating the accepted values.
AgentFactory_ClientMode_ClientCachingKeyedOnMode AgentFactoryTests Two agents differing only in client-mode do NOT share the same CopilotClient — the client-construction/caching path is keyed on Mode (and on BaseDirectory/SessionFs). Regression guard for the architectural implication called out in the design.
CopilotSdkChatClient_BuildSessionConfig_EmptyMode_RequiresAvailableTools CopilotSdkChatClientTests Under ClientMode = Empty, BuildSessionConfig (and BuildResumeSessionConfig) always sets SessionConfig.AvailableTools (never leaves it null) — mirrors the SDK's Empty-mode requirement that every session opts in.

Expected Documentation

The implementation is expected to update the following markdown files in features\Phantom.Workspaces.Data.Core\JsonEntities\documentation\ (and, for research history, features\docs\research\AgentSchema-tool-kinds.md). This section captures the expected outputs of the implementation — do not edit these files as part of the design; they are updated when the code lands.

File What it should describe
agent-options-tools.md Add a section for the github-cli-builtin-tools CustomTool kind: the two selector slots (available-tools, excluded-tools), the three selector forms ({ tools: [...] }, { isolated: true }, and the wildcard "*"), the bare-vs-source-qualified naming rules and the resulting auto-append behaviour, the "MCP-only agent" recipe in both its allowlist and denylist forms, and the client-mode field with the Empty-mode preconditions. Include a JSON example table matching the "JSON examples" section of this issue.
agent-options-overview.md Add a bullet cross-referencing github-cli-builtin-tools as the provider-specific configuration surface for github-copilot / github-copilot-subagent agents, and note that client-mode: empty selects the SDK's Empty client mode (a client-level property, not a session tweak).
agent-configuration.md Document the canonical locked-down configuration recipe (client-mode: empty + available-tools: { "tools": ["mcp:*"] }), calling out that Empty mode + MCP-only allowlist is the recommended pattern for safety-sensitive agents. Note the interlock between client-mode: empty and the required non-empty available-tools.
features\docs\research\AgentSchema-tool-kinds.md Add the github-cli-builtin-tools kind to the research inventory of recognised tool kinds. Record the SDK surface it maps to (SessionConfig.AvailableTools / SessionConfig.ExcludedTools / CopilotClientOptions.Mode) and the SDK version pin (GitHub.Copilot.SDK 1.0.8).

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdiagnosedRoot cause identifiedenhancementNew feature or requestnext-upqueuedIn the active work queue (tracked in work-queue.md)verified-locallyImplementation has been verified locally

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions