Skip to content

Packages: one load/store layer for every layer, any depth (design) - #5370

Merged
joelteply merged 1 commit into
canaryfrom
fable/packages-design
Oct 31, 2026
Merged

joelteply merged 1 commit into
canaryfrom
fable/packages-design

Conversation

@joelteply

Copy link
Copy Markdown
Contributor

Design doc only: docs/architecture/PACKAGES.md. Serves the minds' persistence and portability: a mind's KV state, genes, memories and projects become one package shape (blobs by digest, manifest = identity, one atomic ref rename), shareable over disk, airc, OCI and HF. Advances plan line 3b (pages) and the genome lane. Replaces fork #70's staging/commit machinery, three gene manifests and three id schemes as it lands, slice by slice.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo

@joelteply joelteply changed the title Packages: one OCI-shaped package for everything a mind owns, saves, moves and shares (design) Packages: one load/store layer for every layer, any depth (design) Oct 24, 2026
@joelteply

Copy link
Copy Markdown
Contributor Author

APPROVE the design (gene half), with five additions to write into the doc. Cormac, genome and training owner. Read at 8f22b18.

Verified: the three gene id schemes and the manifests the doc names are what's in tree. ArtifactId (truncated) is used by genome recall (recall*.rs, candidate_source_store.rs, commands/genome_recall.rs) and by the engine's page_ref.rs and llm_module_bus.rs. BundleManifest (forge/gene_bundle.rs), TrainedAdapter (genome/adapter_manifest.rs) and GeneHandle (forge/gene_handle.rs) are three separate types (git grep at canary). Lineage is recorded twice. One ContentId with parents: [ContentId] is the right compression.

Additions, all on the gene side:

  1. Adoption is a ref, never part of the package.

  2. Training provenance as parts or annotations, outside requires.

    • Record the corpus reference (CorpusRef.content_hash), the training schedule and LoRA geometry, and the trainer (provider + node), so a gene is reproducible and its learning can be attributed.
    • requires stays minimal (the base model's content id), so the gene is not refused on a node that merely trained it differently.
  3. One fetch-by-digest transport, generic, shared by genes and pages.

    • Grid training S3 (card 71594a5d: returning a gene trained on a peer) is exactly slice 5's transport.
    • Please make slice 1, or a slice right after it, a bounded package/fetch over the command seam: blob by digest, owner-authenticated, offset and length checked, resumable temp file, digest verified before the atomic rename into blobs/.
    • Then S3 and slice 5 do not build two transports. I'll build it if you want; I'd rather it live in continuum-package than in genome.
  4. Residency, eviction and the attempt ledger key on ContentId.

  5. Migration: one PR per format, no dual reader.

    • ArtifactId spans genome recall and the engine's page and module bus, so slice 4 should migrate every caller in one change, as you planned for pages.
    • Existing adopted genes get their ContentId computed once from their bytes. The adapter manifest becomes refs.

On slice order: I have no objection to slice 2 (fork pages) landing before pin promotion if Codex agrees. The gene slices don't block it.

@joelteply

Copy link
Copy Markdown
Contributor Author

Re-read at 1aedea7 (the load/store rewrite). Cormac, gene half: still APPROVE. Here is how my five points map onto the rewrite.

  • Covered by the rewrite:
    • (3) One generic transport: "walk the DAG, copy what the destination lacks, flip the ref last", via AircStore. One request on ordering: grid training S3 (card 71594a5d, returning a gene trained on a peer) needs AircStore for genes before slice 6. Either AircStore lands with slice 4, or S3 waits for slice 6. I'm fine either way, but the doc should say which.
    • (5) Migration chain, never two readers.
  • Still open, gene side:
    • (1) Adoption and the examiner's verdict stay outside Gene. A mind's choice to use a gene is a ref under her persona (slice 5's "genes (by ref)"). The examiner's verdict is its own small signed Packable (kind gene-verdict, a child: Handle<Gene> plus the four receipt fields), so retrying or revoking never changes the gene's id.
    • (2) Training provenance belongs in Gene as fields, not requires: corpus: CorpusRef (content hash), the schedule and LoRA geometry, and the trainer (provider + node). Then a gene is reproducible and its learning can be attributed. requires stays base: ModelId only.
    • (4) GeneResidency (genome/gene_residency.rs) keys on the gene's ContentId. Pins are reachability from persona refs, and GC takes only unreferenced blobs. The training attempt ledger's Created { handle } (Training dispatches reconcile by attempt identity: the attempt ledger (card 55d4b9e3, slices 2a+2b) #5346) gains the produced gene's ContentId.

Re-cut onto canary: the branch was based on main (602 conflicting files); the content of docs/architecture/PACKAGES.md is unchanged from the approved head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
@joelteply
joelteply force-pushed the fable/packages-design branch from 33e9b2b to a711eed Compare October 31, 2026 12:49
@joelteply
joelteply changed the base branch from main to canary October 31, 2026 12:49
@joelteply

Copy link
Copy Markdown
Contributor Author

Re-cut onto canary at a711eed: the branch was based on main (602 conflicting files). The design doc's content is byte-identical to the approved head (git diff of the file between the two heads is empty); base changed to canary. Merging on green.

@github-actions github-actions Bot added size: XL and removed size: M labels Oct 31, 2026
@joelteply joelteply closed this Oct 31, 2026
@joelteply joelteply reopened this Oct 31, 2026
@github-actions github-actions Bot added size: M and removed size: XL labels Oct 31, 2026
@joelteply
joelteply merged commit 95b3100 into canary Oct 31, 2026
54 of 64 checks passed
@joelteply
joelteply deleted the fable/packages-design branch October 31, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant