docs: correct fabricated, contradicted, and implementation-mismatched claims (source-verified audit)#155
Merged
TeoSlayer merged 2 commits intoJul 24, 2026
Conversation
… claims (source-verified audit) Fixes 32 documentation defects found in a source-level audit of the docs against the shipped implementation (daemon, pilotctl, registry, services, SDKs). Each human docs page edit is mirrored in its /plain/ machine-UI twin, with the plain-source-sha256 stamps refreshed to keep check:plain green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD
The audit finding behind fix E was wrong: handshake.received and message.received ARE emitted, by the handshake and dataexchange plugin modules (handshake@v0.2.2 handshake.go:661,800; dataexchange@v0.2.1-beta service.go:299,369 — published via the daemon runtime into the webhook pipeline, proven end-to-end by tests/zz_webhook_test.go). The original grep covered only the pilotprotocol repo and missed the plugin modules. The integration page was correct as written; this restores it and its plain twin to main's version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These fixes come from a full source-verified audit of the docs pages against the shipped implementation (pilotprotocol daemon/CLI/registry, service repos, and the published SDK artifacts). Each item below lists the defect, the fix, and the evidence (file:line in the relevant repo). Every human page edit is mirrored in its
src/pages/plain/machine-UI twin; the twins'plain-source-sha256stamps were refreshed by hand sonpm run check:plainstays green (theregen-plain.mjsLLM path needsGEMINI_API_KEY, which fork contributors don't have — the twin edits are verbatim mirrors of the human edits). Bothnpm run build(364 pages) andnpm run check:plainpass locally on this branch. Per maintainer guidance,/docs/comparison,/docs/comparison-networking, broadcasts, and specialist-count numbers were not touched.A. Fabricated auto-hostname (getting-started, configuration, cli-reference)
Claimed that omitting
--hostnameyields an auto-generatedpilot-XXXXXXXXname derived from SHA-256(public_key). No such generation exists anywhere:pkg/daemon/daemon.godocumentsHostname string // hostname for discovery (empty = none)and the registry'ssetNodeHostnameearly-returns on empty;/docs/conceptsalready said "the node simply has no hostname (empty)". Now both pages say: no hostname, reachable by address only. Also fixed the same fabrication in cli-reference'sclear-hostnameentry ("keeps a registry-assigned internal hostname (of the formpilot-XXXXXXXX)").B. Auto-update falsehood (getting-started)
"Future releases are applied automatically in the background" — false.
cmd/updater/main.goautoUpdateEnabled()returns false when~/.pilot/auto-update.jsonis absent ("OFF by default");install.shnever writes that file;/docs/configurationalready states OFF by default. Now: updater service is set up, but automatic updates are OFF by default — enable withpilotctl update enable.C. Nonexistent
--channel edge(configuration)The PILOT_RC row called it a "legacy alias for
--channel edge". No--channelflag or "edge" channel exists in install.sh, cmd/pilotctl, or cmd/updater. Now uses install.sh's own wording: "Set to1to install the latest pre-release (RC build)."D. Fabricated error string (troubleshooting)
Quoted error "free networks are limited to 3 agents" exists in no repo; the real string is
network membership limit reached(rendezvous/membership/membership.go:408-409). Remediation now matches/docs/error-codes(raise the network policy's member limit via--max-members, or kick a member) instead of "Upgrade to a Private Network plan", and no specific free-tier cap number is asserted.E.
Fabricated webhook events (integration)— WITHDRAWN, reverted in e56348aThe original audit claimed
handshake.received/message.receiveddon't exist (grep of the pilotprotocol repo's emitters). That grep missed the plugin modules:handshake@v0.2.2publisheshandshake.received/handshake.pending(handshake.go:661,800) anddataexchange@v0.2.1-betapublishesmessage.received/file.received(service.go:299,369), delivered through the webhook pipeline (proven end-to-end bytests/zz_webhook_test.go). The integration page was correct as written; commit e56348a restores it and its plain twin byte-for-byte to main.F. Dead provenance stamp (sdk-parity)
"Updated against
sdk-node@d02bd00,sdk-python@93584ea,sdk-swift@0d49f87(audited 2026-05-28)" — all three short SHAs 404 on GitHub (control short-SHA lookups work). Stamp removed.G. Nonce prefix (concepts)
"Random nonce prefix … to prevent replay" — wrong on both counts.
common/secure/secure.gouses fixed role constants{0,0,0,1}/{0,0,0,2}(comment: prevent nonce collision between directions); actual anti-replay is the per-peer sliding-window check inpkg/daemon/keyexchange(ReplayWindowSize,ErrReplay) and tunnel.go. Text now describes role-based prefixes plus the separate replay window.H. Ping-vs-trust diagnostic (troubleshooting)
"If ping works but connections fail, the target may be dropping SYN packets because trust is not established" — impossible: the SYN trust gate (pkg/daemon/daemon.go:3038-3060) covers every port including echo (port 7, per /docs/diagnostics), so a successful ping proves trust and path. Now: a failure on another port means no listener there or a port policy blocks it.
I. Email persistence + missing files (getting-started, configuration)
"--email is persisted to config" → corrected to
~/.pilot/account.json(pkg/daemon/daemon.go:781-788 → account.Save; confirmed on a live machine; cli-reference already said account.json). Addedaccount.jsonandauto-update.jsonto configuration's directory-structure listing.J. Log file (configuration)
"pilot.log # Daemon log file" / "Logs are written to ~/.pilot/pilot.log" → corrected: per-run
pilot-<pid>.logfiles withpilot.loga symlink to the current one (cmd/pilotctl/main.go:2944; getting-started's own sample output already showedpilot-<pid>.log).K. Socket default (configuration)
Page was self-inconsistent (example said XDG_RUNTIME_DIR-else-/tmp; the env and flag tables said /tmp flatly). Unified everywhere to:
$XDG_RUNTIME_DIR/pilot.sockon Linux when set; otherwise/tmp/pilot.sock(macOS always/tmp/pilot.sock) — percommon/driverDefaultSocketPath(), whose XDG branch is Linux-only.L. AWS Lambda in compat-mode list (firewalls)
Lambda removed from the compat-appropriate platform list: compat mode requires a long-lived daemon holding a persistent WSS connection (the page's own description), which Lambda's teardown model cannot provide; getting-started correctly says serverless is not supported.
M. "One outbound TCP/443 connection" (getting-started)
"one outbound TCP/443 connection to the beacon… Every Pilot frame rides that one socket" → replaced with the firewalls page's accurate version: outbound on TCP/443 only — one long-lived WSS connection to the beacon plus a small TLS connection pool to the registry (cmd/daemon/main.go:161-177 sets up the separate registry TLS channel).
N.
-transport=compatattribution (getting-started)The firewall callout implied
pilotctl daemon starttakes the transport flag. It doesn't —buildDaemonArgsforwards a fixed flag set. Now: enable compat on the standalonepilot-daemonbinary (-transport=compat) or via"transport": "compat"in the daemon config JSON.O. "Nearest beacon region" (firewalls)
There are no regions:
beacon.pilotprotocol.networkandregistry.pilotprotocol.networkboth resolve to the single A record 34.71.57.205. Now "~50-200 ms one-way to the beacon".P. Quickstart description (cli-reference)
The banner was described as "(start the daemon, discover agents, handshake + query)" with a /help→/data step. The actual banner (
cmdQuickstart, a staticfmt.Print) is 1 DISCOVER (list-agents query), 2 TRUST (handshake), 3 TALK (send-message --wait), with first-time daemon setup only in a footer. Rewritten to match; the (correct) statement that it is static and does not detect daemon state is kept.Q.
--jsonuniversality (cli-reference)"All commands support
--json" → "Nearly all commands support--json", referencing the page's own documented exceptions (quickstart always prints its text banner;gateway startreturns no JSON envelope).R. Trust-page absolutes (trust)
"No other agent can discover your address, resolve your hostname, or open a connection to you until you explicitly establish mutual trust" → scoped: untrusted peers cannot obtain your endpoint (IP:port) and cannot connect; directory metadata (hostname, tags, network membership) remains visible in lookups; connectivity is granted by mutual trust OR shared network membership. Evidence: registry tests
TestPrivateNodeResolveBlockedvsTestLookupHidesRealAddr(onlyreal_addrwithheld; tags returned); the daemon SYN gate falls through to registryCheckTrust, which passes shared-network peers; /docs/messaging and /docs/networks already state this model.S.
-dataexchange-b64semantics (services)"adds a lossless
data_b64field" → "replaces thedatafield withdata_b64", matching /docs/messaging ("instead of data"). dataexchange service.go L351-355 is an if/else — one field or the other, never both.T. Webhook delivery semantics (webhooks)
"retried up to 3× with exponential backoff … dropped only after retries are exhausted" → corrected per webhook@v0.2.0/webhook.go: 3 delivery attempts total (
for attempt := 0; attempt < MaxRetries, MaxRetries=3) with 1s doubling backoff; events are dropped immediately with zero attempts when the 1024-event buffer is full (Emit'sdefault:branch); while the circuit breaker is open (5 consecutive failures, 30s cooldown) events are skipped.U. Missing pub/sub rate limit (pubsub)
The Limits section omitted a real limit. Added: 100 events/s per publisher, burst budget 200, excess publishes emit
pubsub.rate_limited(eventstream@v0.2.2/service.gopublishRatePerSecond=100,publishBurstBudget=200; the webhooks page already documents the event).V. "No cache or propagation delay" (networks)
Scoped: the accept gate is live (a kicked peer's next SYN re-checks the registry), but membership metadata reconciles on a 5-minute loop (
DefaultNetworkSyncInterval = 5m) and resolve/policy caches exist (resolve-cache regression test; the site's own webhooks page documents "last-known policy is retained").W. Tags maximum (tags)
"Maximum 3 tags per node" presented as a tag-format rule → corrected:
pilotctlcaps tags at 3 per node (cmdSetTagsrejects >3); the registry itself accepts up to 10 (validation test rejects only >10), so SDK clients may set more than 3. The page's workflow-example contradiction was not touched (pending maintainer decision).X. Python quick-start dial (python-sdk)
d.dial("other-agent:1000")cannot work:dial()(client.py:675) passes the string toPilotDial→ParseSocketAddr, which hard-fails on anything butN:XXXX.YYYY.YYYY:PORT(per the SDK's own docstring; every high-level helper resolves first; the Node page's quick start already resolves). Both the Quick Start and thedial()API example now callresolve_hostname()first.Y.
subscribe_eventexamples (python-sdk)The API reference correctly calls it a generator, but the Pub/Sub example used callback style
d.subscribe_event(peer, topic, on_event)— which cannot work (the function body contains an unconditional yield; calling it just returns a never-iterated generator, so the callback never fires) — and the reference example fed a single tuple to a two-parameteron_event(topic, data). Both examples now use generator style:for topic, data in d.subscribe_event(...).Z. Parity: waitForTrust (sdk-parity)
"Currently Swift-only. Planned for Node and Python" → Swift + Python shipped, Node planned. PyPI 1.13.2 client.py:612 exposes
wait_for_trust(docstring cites the parity ticket PILOT-202) and the shipped libpilot exports_PilotWaitForTrust; the npm dist has nothing.AA. Parity: managed score/rankings (sdk-parity)
The single "score/status/rankings/cycle/reconcile — Node ✅ Python ✅" row was split: Status/ForceCycle/Reconcile remain supported; Score/Rankings were removed in v1.10 — the symbols are absent from shipped libpilot (verified via
nmon both the PyPI wheel's dylib and the npm tarball's .so; libpilot bindings.go documents the v1.10 removal).BB. Node version claim (node-sdk)
"The published 1.12.4 build…" → 1.13.2. npm
dist-tags.latestis 1.13.2, and the published version list jumps 1.12.3→1.12.5 — 1.12.4 was never published.CC. Swift install snippet (swift-sdk)
.package(url: …, from: "0.1.0")→from: "1.13.0". SwiftPM'sfrom:is up-to-next-major, so 0.1.0 resolves the tag list (v0.1.0, v0.2.0, v1.12.5…v1.13.2) to v0.2.0 — a stale wire-protocol build — while sdk-parity promises coordinated 1.13.x releases.DD. Consent:
skills checkin disabled mode (consent)"works in every mode, including disabled" → corrected: in disabled mode
skills checkreports disabled and writes nothing — disabled is a hard opt-out (skillinjecttick()/ForceTickreturnReport{Disabled:true}without touching disk or network).EE. Consent: injected toolchains (consent; also security, getting-started, cli-reference)
The docs listed five toolchains (Claude Code, OpenClaw, OpenHands, PicoClaw, Hermes); the live inject manifest (TeoSlayer/pilot-skills inject-manifest.json) targets seven — added Goose and OpenCode (install.sh ~L1038-1048 also prints Goose paths). The identical stale list also appeared on /docs/security, /docs/getting-started, and cli-reference's
skillsentry; all four instances updated for consistency.FF. Consent: stale side-effect note (consent)
Deleted "(note: status runs a reconcile pass as a side effect — a strictly read-only preview is tracked as a product fix)" — the fix has landed:
cmdSkillsStatusis annotated "// Read-only: status must not write" and calls the dry-runplanTick.Skipped — already fixed on main / text not present
None — every audited claim was still present on
mainand was fixed. (Two related-but-out-of-scope items were deliberately left: the/docs/webhooksevent-type table rows forhandshake.received/message.received(see E), and/docs/concepts' shorter "No other agent can discover, resolve, or communicate" sentence (the scoped rewrite in R targeted the trust page).)🤖 Generated with Claude Code
https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD