refactor(metadata): split the stored envelope from the authored body at the api parse boundary (#5309)

[baozhoutao]

Fixes #5309

What this is

The metadata layer's storage envelope is now peeled off a stored row before
its body is handed to a spec schema. Purely internal: no authored format
changes, no spec change, STILL_STRIP untouched — that flip is #5384's step.

ApiEndpointSchema has two parse sites and both of them are fed STORED ROWS,
not authored declarations:

• buildEndpointIndex — packages/metadata/src/endpoint-matcher.ts, the
  load-time backstop;
• gateApiItemsForPublish — packages/metadata/src/metadata-manager.ts, the
  ADR-0121 publish gate.

A stored row carries the metadata layer's own bookkeeping — packageId,
package, state, version, publishedDefinition, publishedAt,
publishedBy — written by MetadataManager.register / publishPackage and
read back by publishPackage's package filter. None of it is endpoint
vocabulary, and the only thing that kept it from being judged as vocabulary was
the schema's unknown-key stripping. That is exactly why api could not
leave #4001's STILL_STRIP list.

The split

New module packages/metadata/src/stored-envelope.ts:

• a row carrying a metadata value is an envelope around it — the body is
  that value, everything beside it is bookkeeping, by construction;
• otherwise the body is the row minus the declared STORED_ENVELOPE_KEYS.

Neither half is invented. metadata ?? row is already the layer's
body-selection rule in three places (publishedDefinition's snapshot,
getPublished's fallback, gateApiItemsForPublish); the key list is exactly
what MetadataManager writes onto a row or filters rows by. The peel returns
views and never mutates the row, so every existing envelope reader keeps
reading the same object, and publishedDefinition still snapshots
data.metadata ?? data verbatim (revertPackage restores from it).

The prerequisite, demonstrated

Local probe only — ApiEndpointSchema flipped to strictObject and never
committed (that is #5384's step, spec seat). pnpm --filter @objectstack/metadata test:

tree	probe	result
origin/main @ 6f657f4	none	26 files / 519 tests, all pass
origin/main @ 6f657f4	strictObject	2 files failed, 11 tests failed — every one via unrecognized_keys on envelope keys
this branch	none	27 files / 541 tests, all pass
this branch	strictObject	1 file failed, 1 test failed

The 11 reds under the probe on main were: endpoint-matcher.test.ts's
storage-annotation case, and ten of publish-endpoint-gate.test.ts (the whole
D6 group, the other gates, both namespace cases, the publish-envelope case).
They are gone because the envelope no longer reaches the schema.

The one remaining red, explained — it is #5384's business, not this card's.
publish-endpoint-gate.test.ts › does NOT infer the namespace from the item being judged. Its fixture plants an authored namespace: 'acme' key precisely
to prove the gate does not believe it. Under a strict schema that key is
refused by name before the gate ever runs:

api item 'list_things' does not satisfy ApiEndpointSchema and cannot be
published: Unrecognized key: "namespace" (at < root >). An endpoint that does not
parse cannot be gated and would be excluded from endpoint matching at load
anyway.

(The angle brackets around root are spaced only so GitHub's body sanitizer
does not eat them as a tag; the real message has no spaces.)

Publish still refuses the item (success: false); what changes is which
refusal speaks — the schema's named rejection instead of the gate's namespace
precondition, so the assertion message.includes('manifest.namespace') no
longer holds. That is the tightening working, and re-spelling the fixture is
part of the flip, not of the split. namespace is deliberately NOT added to
STORED_ENVELOPE_KEYS: it is an authored key here, not layer bookkeeping, and
peeling it would hide from the schema exactly the typo class #5384 exists to
catch.

One behavioural consequence, stated rather than buried

buildEndpointIndex was the last reader in the layer that did not follow
the metadata ?? row body-selection rule — it parsed the outer object. So a
publish envelope ({ name, packageId, state, metadata: {…} }) passed the
publish gate and was then excluded from the endpoint index: its route
answered 404 while publish had just accepted it. The two doors now read the same
document, which is what "the matcher and the publish gate parse the body" in the
issue's own direction says. Nothing in the repo pinned the old verdict, and it
only affects rows that today fail; every flat row — what artifact ingest,
register() and publishPackage actually produce — indexes byte-identically.
Pinned by indexes a WRAPPED publish envelope off its 'metadata' body.

Tests

• new stored-envelope.test.ts — 15 cases over the peel itself: every
  declared key off the body and on the envelope, the same-object return for a
  row with no bookkeeping, no mutation of the input, metadata: null falling
  through to the flat branch, a non-object metadata handed on for the schema
  to refuse, degenerate inputs, and storedItemName's envelope-first order.
• endpoint-matcher.test.ts — a flat row carrying all seven bookkeeping keys
  indexes with none of them on the answer; a wrapped envelope indexes off its
  body; an unparseable wrapped row is still named by its envelope name; an
  authored declaration is untouched.
• publish-endpoint-gate.test.ts — a fully-published row yields the D6 verdict
  and not a schema error; register → publishPackage → matchEndpoint
  round-trips through both doors. The apiItem fixture is split into
  endpointBody (authored) + apiItem (stored), so the publish-envelope case
  stops nesting a stored row inside metadata — a fixture accident only the
  stripping made invisible.

No rejection case here asserts through toThrow(); the publish refusal surface
is PackagePublishResult.validationErrors[] ({type, name, message}), which
carries no ADR-0112 code/status envelope, so the assertions are on the
verdict entries themselves.

Verification

pnpm --filter @objectstack/metadata test        27 files / 541 tests passed
pnpm --filter @objectstack/rest test            64 files / 885 tests passed
pnpm --filter @objectstack/runtime test        110 files / 1606 tests passed

@objectstack/metadata has no typecheck script (it carries a measured DEBT
entry, 92); tsc --noEmit over the package reports 88, below the ledger, and
none of the errors are in the files this PR touches. Every check:* step
enumerated from .github/workflows/lint.yml was run one by one — 31 in the
ESLint job and 27 in the TypeScript Type Check job, all green, including the full
turbo build + turbo typecheck and check:type-check-debt.

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 8, 2026 5:40am

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata.

8 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

• content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata)
• content/docs/kernel/cluster.mdx (via packages/metadata)
• content/docs/kernel/services-checklist.mdx (via @objectstack/metadata)
• content/docs/plugins/packages.mdx (via @objectstack/metadata)
• content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/metadata)
• content/docs/protocol/kernel/metadata-service.mdx (via @objectstack/metadata)
• content/docs/releases/v12.mdx (via @objectstack/metadata)
• content/docs/releases/v9.mdx (via @objectstack/metadata)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.