Make retrieval receipts reference, not embed, the complete visibility snapshot

[drewstone]

Problem

PR #153 correctly separates visibility, retrieval, and downstream use, but its first receipt shape binds the complete ordered visibility snapshot inside every retrieval receipt. That is exact, yet it makes each retrieval receipt O(number of visible pages). A long-running Discovery campaign with a large inherited/shared knowledge view would repeatedly serialize and hash the same page inventory for every query.

This is evidence duplication, not useful redundancy.

Required invariant

Keep the three distinct facts:

1. which exact ordered page versions were visible;
2. which exact ranked results the retriever returned;
3. which returned rank a downstream consumer selected.

Do not weaken the result-to-visibility join or page-version identity. Change only where the complete snapshot bytes live.

Proposed shape

Visibility snapshot

KnowledgeVisibilitySnapshot remains the canonical ordered inventory and retains:

• position;
• stable page id;
• origin;
• path;
• page digest;
• source ids;
• invalidation state;
• snapshot digest.

Store it once as a durable, content-addressed artifact/record for a stable run knowledge view.

Retrieval receipt

The retrieval receipt should bind a compact reference:

visibility: {
  snapshotDigest: Sha256Digest
  pageCount: number
  artifact?: {
    uri: string
    digest: Sha256Digest
    byteLength: number
  }
}

The artifact field may remain optional at the pure-contract layer when the caller retains the snapshot in another durable record; production adapters should require a durable locator.

Returned results continue to copy exact rank, page id, origin, path, page digest, score, snippet, and reasons.

Verification split

• verifyKnowledgeRetrievalReceipt(receipt) verifies canonical receipt shape/digest, rank continuity, finite scores, and exact result identities. It must not claim a result-to-snapshot join without snapshot evidence.
• assertKnowledgeRetrievalMatchesVisibility(receipt, visiblePages | snapshot) recomputes the canonical snapshot digest/count and proves every returned result occurs in that exact snapshot.
• A durable verifier loads the referenced snapshot artifact, verifies its digest, then calls the same join assertion.

The documentation must state the proof distinction clearly.

Do not add

• no second page registry;
• no Merkle tree unless real scale measurements show snapshot artifacts are themselves too expensive to load;
• no generic Interface artifact type yet—the current cross-package artifact semantics have not converged;
• no SDK copy of either receipt;
• no implicit trust that a matching page id means matching page bytes.

DX

Normal callers should not manage the artifact manually. A Runtime/SDK adapter should:

1. create or reuse the snapshot for the current knowledge-view digest;
2. persist it once;
3. create compact retrieval receipts referencing it;
4. emit trace attributes for snapshot and receipt digests;
5. create use receipts from selected ranks.

A cache keyed by snapshotDigest should make repeated retrievals over an unchanged view O(results), not O(all visible pages).

Migration and tests

• compatibility reader for the embedded-snapshot v1 draft if any artifacts escaped the branch;
• repeated retrievals over one snapshot serialize the full page inventory once;
• changed order/origin/path/page bytes/source/invalidation changes the snapshot digest;
• compact receipt cannot verify against another snapshot with the same page ids but different bytes;
• missing snapshot artifact is explicit unverifiable evidence, not an empty snapshot;
• receipt/use mutation tests remain;
• benchmark receipt bytes and hashing time across 10², 10⁴, and 10⁶ visible pages.

Merge gate

PR #153 should not be described as production-scale retrieval provenance until this duplication is resolved or bounded by measured deployment constraints.