Skip to content

docs: correct fabricated, contradicted, and implementation-mismatched claims (source-verified audit)#155

Merged
TeoSlayer merged 2 commits into
pilot-protocol:mainfrom
pstayets:fix/docs-audit-corrections
Jul 24, 2026
Merged

docs: correct fabricated, contradicted, and implementation-mismatched claims (source-verified audit)#155
TeoSlayer merged 2 commits into
pilot-protocol:mainfrom
pstayets:fix/docs-audit-corrections

Conversation

@pstayets

@pstayets pstayets commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

These fixes come from a full source-verified audit of the docs pages against the shipped implementation (pilotprotocol daemon/CLI/registry, service repos, and the published SDK artifacts). Each item below lists the defect, the fix, and the evidence (file:line in the relevant repo). Every human page edit is mirrored in its src/pages/plain/ machine-UI twin; the twins' plain-source-sha256 stamps were refreshed by hand so npm run check:plain stays green (the regen-plain.mjs LLM path needs GEMINI_API_KEY, which fork contributors don't have — the twin edits are verbatim mirrors of the human edits). Both npm run build (364 pages) and npm run check:plain pass locally on this branch. Per maintainer guidance, /docs/comparison, /docs/comparison-networking, broadcasts, and specialist-count numbers were not touched.

A. Fabricated auto-hostname (getting-started, configuration, cli-reference)

Claimed that omitting --hostname yields an auto-generated pilot-XXXXXXXX name derived from SHA-256(public_key). No such generation exists anywhere: pkg/daemon/daemon.go documents Hostname string // hostname for discovery (empty = none) and the registry's setNodeHostname early-returns on empty; /docs/concepts already said "the node simply has no hostname (empty)". Now both pages say: no hostname, reachable by address only. Also fixed the same fabrication in cli-reference's clear-hostname entry ("keeps a registry-assigned internal hostname (of the form pilot-XXXXXXXX)").

B. Auto-update falsehood (getting-started)

"Future releases are applied automatically in the background" — false. cmd/updater/main.go autoUpdateEnabled() returns false when ~/.pilot/auto-update.json is absent ("OFF by default"); install.sh never writes that file; /docs/configuration already states OFF by default. Now: updater service is set up, but automatic updates are OFF by default — enable with pilotctl update enable.

C. Nonexistent --channel edge (configuration)

The PILOT_RC row called it a "legacy alias for --channel edge". No --channel flag or "edge" channel exists in install.sh, cmd/pilotctl, or cmd/updater. Now uses install.sh's own wording: "Set to 1 to install the latest pre-release (RC build)."

D. Fabricated error string (troubleshooting)

Quoted error "free networks are limited to 3 agents" exists in no repo; the real string is network membership limit reached (rendezvous/membership/membership.go:408-409). Remediation now matches /docs/error-codes (raise the network policy's member limit via --max-members, or kick a member) instead of "Upgrade to a Private Network plan", and no specific free-tier cap number is asserted.

E. Fabricated webhook events (integration) — WITHDRAWN, reverted in e56348a

The original audit claimed handshake.received / message.received don't exist (grep of the pilotprotocol repo's emitters). That grep missed the plugin modules: handshake@v0.2.2 publishes handshake.received/handshake.pending (handshake.go:661,800) and dataexchange@v0.2.1-beta publishes message.received/file.received (service.go:299,369), delivered through the webhook pipeline (proven end-to-end by tests/zz_webhook_test.go). The integration page was correct as written; commit e56348a restores it and its plain twin byte-for-byte to main.

F. Dead provenance stamp (sdk-parity)

"Updated against sdk-node@d02bd00, sdk-python@93584ea, sdk-swift@0d49f87 (audited 2026-05-28)" — all three short SHAs 404 on GitHub (control short-SHA lookups work). Stamp removed.

G. Nonce prefix (concepts)

"Random nonce prefix … to prevent replay" — wrong on both counts. common/secure/secure.go uses fixed role constants {0,0,0,1}/{0,0,0,2} (comment: prevent nonce collision between directions); actual anti-replay is the per-peer sliding-window check in pkg/daemon/keyexchange (ReplayWindowSize, ErrReplay) and tunnel.go. Text now describes role-based prefixes plus the separate replay window.

H. Ping-vs-trust diagnostic (troubleshooting)

"If ping works but connections fail, the target may be dropping SYN packets because trust is not established" — impossible: the SYN trust gate (pkg/daemon/daemon.go:3038-3060) covers every port including echo (port 7, per /docs/diagnostics), so a successful ping proves trust and path. Now: a failure on another port means no listener there or a port policy blocks it.

I. Email persistence + missing files (getting-started, configuration)

"--email is persisted to config" → corrected to ~/.pilot/account.json (pkg/daemon/daemon.go:781-788 → account.Save; confirmed on a live machine; cli-reference already said account.json). Added account.json and auto-update.json to configuration's directory-structure listing.

J. Log file (configuration)

"pilot.log # Daemon log file" / "Logs are written to ~/.pilot/pilot.log" → corrected: per-run pilot-<pid>.log files with pilot.log a symlink to the current one (cmd/pilotctl/main.go:2944; getting-started's own sample output already showed pilot-<pid>.log).

K. Socket default (configuration)

Page was self-inconsistent (example said XDG_RUNTIME_DIR-else-/tmp; the env and flag tables said /tmp flatly). Unified everywhere to: $XDG_RUNTIME_DIR/pilot.sock on Linux when set; otherwise /tmp/pilot.sock (macOS always /tmp/pilot.sock) — per common/driver DefaultSocketPath(), whose XDG branch is Linux-only.

L. AWS Lambda in compat-mode list (firewalls)

Lambda removed from the compat-appropriate platform list: compat mode requires a long-lived daemon holding a persistent WSS connection (the page's own description), which Lambda's teardown model cannot provide; getting-started correctly says serverless is not supported.

M. "One outbound TCP/443 connection" (getting-started)

"one outbound TCP/443 connection to the beacon… Every Pilot frame rides that one socket" → replaced with the firewalls page's accurate version: outbound on TCP/443 only — one long-lived WSS connection to the beacon plus a small TLS connection pool to the registry (cmd/daemon/main.go:161-177 sets up the separate registry TLS channel).

N. -transport=compat attribution (getting-started)

The firewall callout implied pilotctl daemon start takes the transport flag. It doesn't — buildDaemonArgs forwards a fixed flag set. Now: enable compat on the standalone pilot-daemon binary (-transport=compat) or via "transport": "compat" in the daemon config JSON.

O. "Nearest beacon region" (firewalls)

There are no regions: beacon.pilotprotocol.network and registry.pilotprotocol.network both resolve to the single A record 34.71.57.205. Now "~50-200 ms one-way to the beacon".

P. Quickstart description (cli-reference)

The banner was described as "(start the daemon, discover agents, handshake + query)" with a /help→/data step. The actual banner (cmdQuickstart, a static fmt.Print) is 1 DISCOVER (list-agents query), 2 TRUST (handshake), 3 TALK (send-message --wait), with first-time daemon setup only in a footer. Rewritten to match; the (correct) statement that it is static and does not detect daemon state is kept.

Q. --json universality (cli-reference)

"All commands support --json" → "Nearly all commands support --json", referencing the page's own documented exceptions (quickstart always prints its text banner; gateway start returns no JSON envelope).

R. Trust-page absolutes (trust)

"No other agent can discover your address, resolve your hostname, or open a connection to you until you explicitly establish mutual trust" → scoped: untrusted peers cannot obtain your endpoint (IP:port) and cannot connect; directory metadata (hostname, tags, network membership) remains visible in lookups; connectivity is granted by mutual trust OR shared network membership. Evidence: registry tests TestPrivateNodeResolveBlocked vs TestLookupHidesRealAddr (only real_addr withheld; tags returned); the daemon SYN gate falls through to registry CheckTrust, which passes shared-network peers; /docs/messaging and /docs/networks already state this model.

S. -dataexchange-b64 semantics (services)

"adds a lossless data_b64 field" → "replaces the data field with data_b64", matching /docs/messaging ("instead of data"). dataexchange service.go L351-355 is an if/else — one field or the other, never both.

T. Webhook delivery semantics (webhooks)

"retried up to 3× with exponential backoff … dropped only after retries are exhausted" → corrected per webhook@v0.2.0/webhook.go: 3 delivery attempts total (for attempt := 0; attempt < MaxRetries, MaxRetries=3) with 1s doubling backoff; events are dropped immediately with zero attempts when the 1024-event buffer is full (Emit's default: branch); while the circuit breaker is open (5 consecutive failures, 30s cooldown) events are skipped.

U. Missing pub/sub rate limit (pubsub)

The Limits section omitted a real limit. Added: 100 events/s per publisher, burst budget 200, excess publishes emit pubsub.rate_limited (eventstream@v0.2.2/service.go publishRatePerSecond=100, publishBurstBudget=200; the webhooks page already documents the event).

V. "No cache or propagation delay" (networks)

Scoped: the accept gate is live (a kicked peer's next SYN re-checks the registry), but membership metadata reconciles on a 5-minute loop (DefaultNetworkSyncInterval = 5m) and resolve/policy caches exist (resolve-cache regression test; the site's own webhooks page documents "last-known policy is retained").

W. Tags maximum (tags)

"Maximum 3 tags per node" presented as a tag-format rule → corrected: pilotctl caps tags at 3 per node (cmdSetTags rejects >3); the registry itself accepts up to 10 (validation test rejects only >10), so SDK clients may set more than 3. The page's workflow-example contradiction was not touched (pending maintainer decision).

X. Python quick-start dial (python-sdk)

d.dial("other-agent:1000") cannot work: dial() (client.py:675) passes the string to PilotDialParseSocketAddr, which hard-fails on anything but N:XXXX.YYYY.YYYY:PORT (per the SDK's own docstring; every high-level helper resolves first; the Node page's quick start already resolves). Both the Quick Start and the dial() API example now call resolve_hostname() first.

Y. subscribe_event examples (python-sdk)

The API reference correctly calls it a generator, but the Pub/Sub example used callback style d.subscribe_event(peer, topic, on_event) — which cannot work (the function body contains an unconditional yield; calling it just returns a never-iterated generator, so the callback never fires) — and the reference example fed a single tuple to a two-parameter on_event(topic, data). Both examples now use generator style: for topic, data in d.subscribe_event(...).

Z. Parity: waitForTrust (sdk-parity)

"Currently Swift-only. Planned for Node and Python" → Swift + Python shipped, Node planned. PyPI 1.13.2 client.py:612 exposes wait_for_trust (docstring cites the parity ticket PILOT-202) and the shipped libpilot exports _PilotWaitForTrust; the npm dist has nothing.

AA. Parity: managed score/rankings (sdk-parity)

The single "score/status/rankings/cycle/reconcile — Node ✅ Python ✅" row was split: Status/ForceCycle/Reconcile remain supported; Score/Rankings were removed in v1.10 — the symbols are absent from shipped libpilot (verified via nm on both the PyPI wheel's dylib and the npm tarball's .so; libpilot bindings.go documents the v1.10 removal).

BB. Node version claim (node-sdk)

"The published 1.12.4 build…" → 1.13.2. npm dist-tags.latest is 1.13.2, and the published version list jumps 1.12.3→1.12.5 — 1.12.4 was never published.

CC. Swift install snippet (swift-sdk)

.package(url: …, from: "0.1.0")from: "1.13.0". SwiftPM's from: is up-to-next-major, so 0.1.0 resolves the tag list (v0.1.0, v0.2.0, v1.12.5…v1.13.2) to v0.2.0 — a stale wire-protocol build — while sdk-parity promises coordinated 1.13.x releases.

DD. Consent: skills check in disabled mode (consent)

"works in every mode, including disabled" → corrected: in disabled mode skills check reports disabled and writes nothing — disabled is a hard opt-out (skillinject tick()/ForceTick return Report{Disabled:true} without touching disk or network).

EE. Consent: injected toolchains (consent; also security, getting-started, cli-reference)

The docs listed five toolchains (Claude Code, OpenClaw, OpenHands, PicoClaw, Hermes); the live inject manifest (TeoSlayer/pilot-skills inject-manifest.json) targets seven — added Goose and OpenCode (install.sh ~L1038-1048 also prints Goose paths). The identical stale list also appeared on /docs/security, /docs/getting-started, and cli-reference's skills entry; all four instances updated for consistency.

FF. Consent: stale side-effect note (consent)

Deleted "(note: status runs a reconcile pass as a side effect — a strictly read-only preview is tracked as a product fix)" — the fix has landed: cmdSkillsStatus is annotated "// Read-only: status must not write" and calls the dry-run planTick.

Skipped — already fixed on main / text not present

None — every audited claim was still present on main and was fixed. (Two related-but-out-of-scope items were deliberately left: the /docs/webhooks event-type table rows for handshake.received/message.received (see E), and /docs/concepts' shorter "No other agent can discover, resolve, or communicate" sentence (the scoped rewrite in R targeted the trust page).)

🤖 Generated with Claude Code

https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD

pstayet and others added 2 commits July 23, 2026 15:07
… claims (source-verified audit)

Fixes 32 documentation defects found in a source-level audit of the docs
against the shipped implementation (daemon, pilotctl, registry, services,
SDKs). Each human docs page edit is mirrored in its /plain/ machine-UI twin,
with the plain-source-sha256 stamps refreshed to keep check:plain green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD
The audit finding behind fix E was wrong: handshake.received and
message.received ARE emitted, by the handshake and dataexchange plugin
modules (handshake@v0.2.2 handshake.go:661,800; dataexchange@v0.2.1-beta
service.go:299,369 — published via the daemon runtime into the webhook
pipeline, proven end-to-end by tests/zz_webhook_test.go). The original
grep covered only the pilotprotocol repo and missed the plugin modules.
The integration page was correct as written; this restores it and its
plain twin to main's version.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142ryqVGEmJN66VZtCC7wFD
@TeoSlayer
TeoSlayer merged commit 3df102f into pilot-protocol:main Jul 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants