docs: fix inaccurate SDK/runtime claims found in docs audit - #2064
Conversation
Cross-checked doc claims against the SDK and copilot-agent-runtime repos and corrected five verified inaccuracies: - setup/local-cli.md, setup/bundled-cli.md: the Go SDK is not bundle-less — it embeds a CLI via `go tool bundler` and reads COPILOT_CLI_PATH; corrected the "must always provide Connection" guidance. - setup/bundled-cli.md: Java configures the CLI via setCliPath / setCliUrl (or `copilot` on PATH), not `Connection` / COPILOT_CLI_PATH. - hooks/user-prompt-submitted.md: the hook output has no reject/rejectReason; replaced the rate-limit example and best-practice bullet with the real additionalContext behavior. - auth/byok.md: omitted Azure apiVersion uses the GA versionless v1 route, not a 2024-10-21 default. - features/custom-agents.md: the runtime now inherits parent reasoning effort for same-model subagents (#13388); corrected the "parent effort is not inherited" statement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: af5056fb-e73c-4bc2-bc11-753fd633cb98
There was a problem hiding this comment.
Pull request overview
Corrects verified documentation inaccuracies across CLI setup, hooks, BYOK configuration, and custom-agent behavior.
Changes:
- Clarifies Go and Java CLI resolution.
- Corrects hook and Azure API behavior.
- Documents reasoning-effort inheritance.
Show a summary per file
| File | Description |
|---|---|
docs/setup/local-cli.md |
Updates Go CLI setup guidance. |
docs/setup/bundled-cli.md |
Corrects Go and Java CLI behavior. |
docs/hooks/user-prompt-submitted.md |
Replaces unsupported prompt rejection fields. |
docs/features/custom-agents.md |
Clarifies reasoning-effort resolution. |
docs/auth/byok.md |
Corrects Azure API-version routing. |
Review details
Comments suppressed due to low confidence (1)
docs/hooks/user-prompt-submitted.md:495
additionalContextcannot gate a request; it only influences the model after the request has already proceeded. Use advisory wording such as “redirect” rather than retaining another implication of enforcement. The replacement also uses the colon required for labeled list items indocs/.
1. **Prefer `additionalContext` for soft controls** - This hook cannot hard-reject a prompt; to discourage or gate a request, add context explaining the constraint rather than expecting a block.
- Files reviewed: 5/5 changed files
- Comments generated: 5
- Review effort level: Medium
Document the Go PATH fallback, reframe the hook example as advisory, and correct Azure versionless routing references across SDK docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (3)
docs/hooks/user-prompt-submitted.md:439
- Record the current submission before testing the threshold. As written, the tenth prompt produces no notice, and prompts that receive a notice are never added to
promptTimestamps, so the counter stops representing actual usage. Pushnowfirst and report the actual count.
if (promptTimestamps.length >= NOTICE_THRESHOLD) {
// This is advisory context for the model, not an enforced rate limit.
// Enforce hard limits before calling session.send().
return {
additionalContext: `The user has sent ${NOTICE_THRESHOLD} prompts in the last minute. Suggest waiting before sending more.`,
docs/features/custom-agents.md:262
- This correction leaves all public SDK API documentation stating the opposite:
nodejs/src/types.ts:1646-1647,python/copilot/session.py:1083-1084,go/types.go:952-954,dotnet/src/Types.cs:2661-2664,rust/src/types.rs:645-648, andjava/src/main/java/com/github/copilot/rpc/CustomAgentConfig.java:298-302still say the parent effort is not inherited. Update those comments too so users relying on IDE/API docs receive the corrected same-model behavior.
Set `model` and `reasoningEffort` to override the parent session's model settings while a custom agent runs. When `reasoningEffort` is omitted, the SDK sends no per-agent override and the runtime resolves the effort from its own precedence: a per-call client option, the resolved model's default, or the agent definition all take priority; otherwise the runtime inherits the parent session's effort only when the subagent runs the same model as the parent. When the subagent resolves to a different model, it falls back to that model's default instead of inheriting the parent's effort. Python uses `reasoning_effort`, .NET uses `ReasoningEffort`, Go uses `ReasoningEffort`, Java uses `setReasoningEffort`, and Rust uses `with_reasoning_effort`.
docs/hooks/user-prompt-submitted.md:494
- Use a colon between the bold label and its description, as required by
.github/instructions/docs-style.instructions.md:79-82.
1. **Use `additionalContext` for advisory guidance** - This hook cannot reject a prompt or enforce policy. Enforce hard limits before calling `session.send()`.
- Files reviewed: 12/12 changed files
- Comments generated: 0 new
- Review effort level: Medium
This comment has been minimized.
This comment has been minimized.
Correct the advisory threshold example and align Azure routing and reasoning-effort API documentation across all six SDKs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SteveSandersonMS
left a comment
There was a problem hiding this comment.
Verified the documentation corrections against the SDK and runtime implementations. All review feedback is addressed consistently across the six SDKs.
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (4)
docs/setup/local-cli.md:5
- This still implies that using the Go SDK is itself a reason to supply a local CLI explicitly.
NewClient(nil)can use a bundled CLI or resolvecopilotfromPATH, so Go users do not need this setup unless they want to override that resolution. Please narrow this guidance to the override/pinning case.
**Use when:** You need to pin a specific CLI version, or work with the Go SDK (which does not include a CLI automatically).
docs/setup/bundled-cli.md:148
- This note is prose inside a
<details>language tab. The docs pipeline requires consecutive language tabs to contain only a code fence (plus an optional validation marker); prose here prevents the group from normalizing correctly. Move this note outside the tab group and label it**(Java)**.
> [!NOTE]
> The Java SDK does not bundle or embed the Copilot CLI. Install the CLI separately and either make `copilot` available on your `PATH` or set its location with `setCliPath(...)` (or connect to a running CLI server with `setCliUrl(...)`).
docs/setup/local-cli.md:81
- This note is prose inside a
<details>language tab. The docs pipeline requires consecutive language tabs to contain only a code fence (plus an optional validation marker); prose here prevents the group from normalizing correctly. Move this note outside the tab group and label it**(Go)**.
> [!NOTE]
> The Go SDK does not ship a CLI automatically. Install `copilot` on `PATH`, set the `COPILOT_CLI_PATH` environment variable, embed a CLI with the [bundler tool](../../go/README.md#distributing-your-application-with-an-embedded-github-copilot-cli), or point `StdioConnection.Path` at an installed binary.
docs/setup/bundled-cli.md:82
- This note is prose inside a
<details>language tab. The docs pipeline requires consecutive language tabs to contain only a code fence (plus an optional validation marker); prose here prevents the group from normalizing correctly. Move this note outside the tab group and label it**(Go)**.
This issue also appears on line 147 of the same file.
> [!NOTE]
> Unlike Node.js, Python, and .NET, the Go SDK does not include a CLI as an automatic dependency. With no explicit path, `NewClient(nil)` uses an embedded CLI when available, then falls back to `copilot` on `PATH`. To embed a CLI, run the [bundler tool](../../go/README.md#distributing-your-application-with-an-embedded-github-copilot-cli) at build time. You can also set `COPILOT_CLI_PATH` or point a `Connection` at an existing binary. See [Local CLI Setup](./local-cli.md) for details.
- Files reviewed: 15/15 changed files
- Comments generated: 0 new
- Review effort level: Medium
This comment has been minimized.
This comment has been minimized.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cross-SDK Consistency Review ✅This PR updates documentation and comments across all six SDK implementations (Node.js/TypeScript, Python, Go, .NET, Java, Rust) for two related changes:
All language implementations are updated in parallel. API surface and semantics are consistent across languages, accounting for appropriate naming conventions (camelCase for TS/Java, snake_case for Python/Rust, PascalCase for Go/.NET). No cross-SDK inconsistencies found. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Summary
Ran a documentation audit that cross-checked concrete claims in
docs/against both the SDK repo and thecopilot-agent-runtimerepo (source of truth for protocol/CLI behavior). This PR fixes the five high-confidence, verified inaccuracies. Each was re-verified against source before editing.Fixes
setup/local-cli.md,setup/bundled-cli.mdConnection"go tool bundlerand readsCOPILOT_CLI_PATH(go/README.md:99-134,go/types.go:43)setup/bundled-cli.md(Java)ConnectionorCOPILOT_CLI_PATHsetCliPath/setCliUrl, orcopilotonPATH(CopilotClientOptions.java:172,202); noConnection/env varhooks/user-prompt-submitted.mdreject/rejectReasonmodifiedPrompt/additionalContext/suppressOutput(go/types.go:672-676) — example rewrittenauth/byok.mdapiVersion"default2024-10-21"v1route (runtime/generated/api.schema.json:22541-22543)features/custom-agents.mdValidation
just validate-docs-extract(only changed TS sample is a partial block;.validation/is git-ignored).just validate-docs-ts-> 178 files pass.Not included (follow-ups for separate PRs)
The audit also surfaced larger gaps that need net-new docs rather than corrections:
canvasProvider, in-process FFI transport,enableManagedSettings, toolmetadatapassthrough.allowedMcpServers/deniedMcpServers), sidekick events, sandbox auth injection, plugin Rust port / Open Plugin Spec v1.COPILOT_OTEL_*env vars, protocol version / bundled CLI version introubleshooting/compatibility.md, compaction events in session docs.Two audit findings were investigated and intentionally not changed (false positives): the
error-handling.mderrorTypeclaim (doc is already correct) and themcp-debugging.mdMCP_DEBUG/NODE_DEBUGenv vars (those configure the user's own MCP server, not a runtime toggle).