feat(metadata-protocol,objectql,rest,spec)!: every metadata read is environment → code; legacy organization rows and sealed overlays are reported at boot, not served (#15206 S5)

.changeset/15206-reads-environment-only.md

@@ -0,0 +1,45 @@
+---
+'@objectstack/metadata-protocol': minor
+'@objectstack/metadata-core': minor
+'@objectstack/objectql': minor
+'@objectstack/rest': minor
+'@objectstack/plugin-security': minor
+'@objectstack/spec': minor
+---
+
+feat(metadata-protocol,metadata-core,objectql,rest,plugin-security,spec)!: every metadata read is environment → code; legacy organization rows and overlays of sealed managed content are reported at boot, not served (ADR-0131 D6)
+
+Clause-②: no (narrowing)
+
+<!-- adr-0087: registered metadata-read-organization-scope-retired -->
+
+**BREAKING (narrowing).** No read that is refused today is admitted. Graded `minor` on the v18 prerelease line: Changesets is in pre mode with the tag `next`, and the fixed group is already majored by the line's opening marker, so this ships in an `18.0.0-next.N`.
+
+ADR-0131 D6 retires the per-organization overlay axis. The doors carry no organization and the protocol refuses every organization-scoped write; now every protocol read is environment → code as well.
+
+**What changes.**
+
+- The item, list, layered, cached (ETag), history, diff, audit, drafts and commit-timeline reads, the `_lock` gate's overlay layer, search's page sweep, diagnostics and references serve the environment's stored row, else the code packages' definition — whatever organization a caller names. A row stored organization-scoped by an earlier release is served by no read and loaded by no boot.
+- `overlayScope` on the layered read is `'env'` or `null`; `'org'` is gone.
+- `listDrafts` lists the environment's drafts only, and its rows carry no `organizationId`.
+- `listCommits` lists the environment's commits only, for every caller: a legacy organization commit is not shown, so an operator on the packages door no longer sees every organization's legacy commits, and a rollback plan never contains one.
+- [Triage ruling Q1 → C] An environment row that overlays an item a managed package ships, on a type sealed against overlays (any type whose registry entry admits no overlay — a managed flow, action, hook, object or datasource, for example — written through the `OS_METADATA_WRITABLE` hatch before the seal), is not served: the package's definition is. An `object` such row is not loaded at boot either, so the engine keeps the packaged schema. A stored fork of a code-declared permission set keeps its own ruling (detection reading and Discard Overlay) and is still served.
+- `@objectstack/objectql` binds the environment's authored hook and action rows only; a legacy organization-scoped hook or action row is no longer bound.
+- Boot names both populations, once each: `[metadata_org_scoped_unserved]` lists every legacy organization row per type (active and draft), plus the legacy rows of `sys_metadata_commit`, `sys_metadata_history` and `sys_metadata_audit`, and names the v18 migration ceremony (`os migrate`, ADR-0131 D10) that carries them; `[metadata_sealed_overlay_unserved]` lists the sealed overlays per type with the two remedies. Nothing is deleted or rewritten.
+- The anonymous public-form doors keep reading the Default Organization's legacy `view` layer, fail-closed, until that ceremony (triage ruling Q3 A) — through the protocol-internal `legacyFormOrganizationId` key, which no spec request declares and which the protocol honours for `view` alone.
+- The seed loader's refusal of an unowned record now prescribes `organization_id` on the record first: a seed published through a package carries no caller organization.
+
+**What moves for consumers.**
+
+| From | To |
+|:--|:--|
+| `organizationId` on a `GetMetaItems` / `GetMetaItem` / `GetMetaItemLayered` / `AuditMetaItem` / `HistoryMetaItem` / `GetMetaItemCached` request (`@objectstack/spec`) | drop it: the read is the environment's. The key is stripped at a spec parse (the schemas are not strict) and no longer type-checks |
+| `organizationId` on a `listDrafts` / `listCommits` / `diffMetaItem` / `getMetaDiagnostics` / `findReferencesToMeta` request (`@objectstack/metadata-protocol`) | drop it |
+| `ListDraftsResponse` draft `organizationId` | gone: every listed draft is the environment's |
+| `overlayScope: 'org'` (`GetMetaItemLayeredResponse`) | only `'env'` or `null` |
+| `declaresOrgOverride`, `organizationIdForMetaRead` (`@objectstack/metadata-core`) | removed — no read takes an organization. Read `allowOrgOverride` off `DEFAULT_METADATA_TYPE_REGISTRY` where a type's overlay channel is the question |
+| `MetadataAuthoringGateContext.organizationId` (`@objectstack/metadata-protocol`) | removed — no write is organization-scoped, and no gate read it |
+| a legacy organization-scoped `sys_metadata` row a deployment relied on | re-save the item in Studio to make it environment-wide now, or wait for the v18 migration ceremony, which promotes it |
+| an environment overlay of a managed item on a sealed type (flow, action, hook, object, datasource …) | re-express the change as a new item under a new name (a linkage-free clone), then delete the stored row; or delete the row |
+
+**What does not change.** No stored row is deleted, rewritten or migrated. The anonymous form doors' read of organization-layer withdrawals is unchanged.

content/docs/concepts/metadata-lifecycle.mdx

@@ -45,8 +45,8 @@ Reads walk top-to-bottom: the first non-null layer wins. Writes always route to
 <Callout type="warn">
 Today this precedence is implemented **inside the protocol layer**, not by
 `LayeredRepository`. `ObjectStackProtocolImplementation` lazily builds one
-`SysMetadataRepository` per org (`getOverlayRepo()`) and merges it against the
-in-memory code registry — `getMetaItemLayered()` returns `code` / `overlay` /
+`SysMetadataRepository` for the environment (`getOverlayRepo()`) and merges it
+against the in-memory code registry — `getMetaItemLayered()` returns `code` / `overlay` /
 `effective` separately. `LayeredRepository` ships in `@objectstack/metadata-core`
 and implements exactly the semantics above, but it is **not yet composed** into
 any production path: every `new LayeredRepository(...)` call site is a test
@@ -106,7 +106,7 @@ later ships. See [ADR-0070](https://github.com/objectstack-ai/objectstack/blob/m
 
 In shared-database multi-tenancy, **most metadata types must not be per-org customizable** — overriding them would break the physical schema. The whitelist lives in **one** place: `MetadataTypeRegistryEntry.allowOrgOverride` in `packages/spec/src/kernel/metadata-plugin.zod.ts`.
 
-> **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (`organization_id` NULL) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is not served by any `/meta` read until the promotion ceremony (ADR-0131 C7) carries it to the environment layer; re-save the item in Studio to make the edit live on the `/meta` doors now. Public forms are the exception: until that ceremony the anonymous form doors read a form `view` in the Default Organization and prefer its overlay for the form's body, while a withdrawal in either layer closes the form, fail-closed. So a legacy organization overlay of a public form keeps serving its body there: a Studio re-save of that body (an environment row) does not change the body the public form serves, and a Studio withdrawal (an environment row) still closes it. The table still records which types take an overlay of a shipped item.
+> **ADR-0131 D6 — the per-organization overlay axis is retired.** The `/meta` doors (REST and the runtime dispatcher) carry no organization into a metadata write or read: a Studio or `PUT /api/v1/meta/...` write of the five ✅ types below lands **environment-wide** (`organization_id` NULL) and is served to every organization, and an organization admin holding only the retired `manage_org_presentation` capability is refused (`403`). The protocol itself refuses every organization-scoped write, and every read — item, list, layers, cached, history, diff, audit, drafts and the commit timeline — is environment → code, whatever organization a caller names. An overlay row an earlier release stored under an organization — under the `single` posture, every Studio save of these types, filed under the Default Organization — is served by no read: boot names each such row, per type (`[metadata_org_scoped_unserved]`), and the v18 migration ceremony (ADR-0131 C7, `os migrate`) carries it to the environment layer; nothing deletes or rewrites it before then. Re-save the item in Studio to make the edit live now. Public forms are the exception: until that ceremony the anonymous form doors read a form `view` in the Default Organization and prefer its overlay for the form's body, while a withdrawal in either layer closes the form, fail-closed. So a legacy organization overlay of a public form keeps serving its body there: a Studio re-save of that body (an environment row) does not change the body the public form serves, and a Studio withdrawal (an environment row) still closes it. The table still records which types take an overlay of a shipped item.
 
 | Type | `allowOrgOverride` | Rationale |
 | :--- | :---: | :--- |
@@ -142,6 +142,8 @@ See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/
 
 The switches are operator-only where one install serves several organizations. The refusal names the same path whichever layer answers it: `403 NOT_OVERRIDABLE` for a write that names no base, and `403 ITEM_LOCKED` for one that names the read-only package. Which writes are refused, and with which code, is unchanged.
 
+An environment row stored over a managed item of a type sealed against overlays (a flow, action, hook, object or datasource, for example — any type whose registry entry admits no overlay) before the seal — written through the `OS_METADATA_WRITABLE` hatch — is **not served** (ADR-0131 D6): every read, and boot, takes the package's definition, and the row stays at rest, untouched. Boot names each such row, per type (`[metadata_sealed_overlay_unserved]`), with the two remedies: re-express the change as a new item under a new name, or delete the stored row. A stored fork of a code-declared permission set is the one exception: it keeps the detection reading and the operator's Discard Overlay action it already had.
+
 ---
 
 ## Conflict handling
@@ -176,13 +178,13 @@ The hash is `sha256:` + 64-hex of a canonical JSON serialization of the body, wi
 | :--- | :--- |
 | `InMemoryRepository`, `LayeredRepository` | ✅ Shipped (`@objectstack/metadata-core`) |
 | `FileSystemRepository` | ✅ Shipped (`@objectstack/metadata-fs`) |
-| Change log + `seq` (per-org, monotonic) | ✅ Shipped |
+| Change log + `seq` (per environment, monotonic) | ✅ Shipped |
 | SSE bridge (`/api/v1/dev/metadata-events`, `event: metadata-change`) | ✅ Shipped |
 | Granular client-side HMR hook + seq status badge | ❌ Not implemented — no such hook or badge exists in the console today |
 | Console dev-mode HMR reloader (`MetadataHmrReloader`) | ✅ Shipped (console app, mounted app-wide, dev-only) |
 | `SysMetadataRepository` (overlay over `sys_metadata`) | ✅ Shipped (`@objectstack/metadata-protocol`) |
 | `LayeredRepository(SysMeta + artifact)` composition | ⏳ Not wired — the class ships, but nothing composes it outside tests |
-| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (via the per-org `getOverlayRepo()` cache) |
+| `protocol.ts:saveMetaItem` routed through `SysMetadataRepository.put` | ✅ Shipped (via the environment's `getOverlayRepo()` repository) |
 | `sys_metadata_history` table (durable, org-keyed change log) | ✅ Shipped (object definition in `@objectstack/metadata-core`; written by `SysMetadataRepository`) |
 | Cache + Registry refactor against `MetadataRepository` | ⏳ Post-M0 |
 
@@ -236,9 +238,9 @@ The hash is `sha256:` + 64-hex of a canonical JSON serialization of the body, wi
 ### Why artifact never enters the database
 
 1. **Single source of truth.** Code → artifact → memory. If artifact also wrote to `sys_metadata`, every deploy would need a reconciliation step ("the row says v3.0.0 but the code says v3.0.1") that is impossible to get right under concurrent edits.
-2. **Overlay stays small and auditable.** Per-org overlay = "what this organization deliberately changed at runtime". A clean table that's safe to dump, diff, or reset to factory.
+2. **Overlay stays small and auditable.** The overlay = "what this environment deliberately changed at runtime" (ADR-0131 D6: one environment layer, no per-organization one). A clean table that's safe to dump, diff, or reset to factory.
 3. **Immutable infrastructure.** Rolling back a deploy = switching image tags. No DB migration, no reverse `put`, no history replay required.
-4. **Per-org multi-tenant economics.** If artifacts were materialised into `sys_metadata`, every org would carry tens of thousands of baseline rows. Overlay-only keeps each org's row count proportional to its actual customisation.
+4. **Storage economics.** If artifacts were materialised into `sys_metadata`, every environment would carry tens of thousands of baseline rows. Overlay-only keeps its row count proportional to its actual customisation.
 
 ### What if I want auditable history of deployed artifact changes?
 

content/docs/deployment/environment-variables.mdx

@@ -306,7 +306,7 @@ that bypassed write hooks, `rebuildSearchCompanion` (from
 | `OS_MARKETPLACE_CACHE` | enum | `on` | `off` disables the in-memory marketplace listing cache. |
 | `OS_MARKETPLACE_PUBLIC_BASE_URL` | url | — | Public base URL of the marketplace registry (proxied from this runtime when set). |
 | `OS_ALLOW_UNMASKED_OBJECT_METADATA` | boolean | `false` | Escape hatch for the metadata-plane field-level security mask ([ADR-0106](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0106-metadata-plane-fls-object-schema-masking.md) D8). By default every object schema served by `/meta` and `/metadata` is projected onto the fields the **calling user** may read, so a field they cannot read does not appear at all — not its name, label, type, picklist options, formula, `visibleWhen` predicate, `defaultValue`, or the `requiredPermissions` capability guarding it. Set to `1` to serve the full schema to every authenticated caller, as releases before this one did. This changes **disclosure only**: the data plane still masks values and refuses forbidden writes either way, and the console reads field affordances from `/auth/me/permissions`, so toggling it never changes UI correctness. The REST layer also honours a per-server `metadata.maskObjectFields: false`; this variable is the deployment-wide knob and covers the runtime `/metadata` dispatcher, which has no REST config to read. |
-| `OS_METADATA_WRITABLE` | csv | — (none) | Comma-separated metadata type names (e.g. `hook,job`) granted a runtime escape hatch that treats them as `allowOrgOverride: true` for items **no managed package ships**: it opens runtime creation of a type whose registry entry allows none. It never opens an organization-scoped write: since [ADR-0131](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0131-total-organization-ownership-no-null-organization-id.md) D6 the metadata protocol refuses every organization-scoped write with `403 NOT_OVERRIDABLE`. It **never opens an item a managed package ships**: managed content is sealed ([ADR-0131](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0131-total-organization-ownership-no-null-organization-id.md) D6), so an overlay or a removal of a shipped flow, object, field, permission set, position or any other type without an environment overlay is refused with `403 NOT_OVERRIDABLE` whether the hatch is set or not. Customize managed content through its type's own route instead: an environment overlay for `view`, `dashboard`, `report`, `translation` and `email_template`; disable, or clone under a new name, for a flow; clone for a permission set. Overlay rows this hatch wrote before are still served, and a row of a type that merges overlays at read (`permission`, `position`, `page`, `app`, `dataset`, `book`, `tool`, `skill`) can still be removed to restore the package's definition. See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md). |
+| `OS_METADATA_WRITABLE` | csv | — (none) | Comma-separated metadata type names (e.g. `hook,job`) granted a runtime escape hatch that treats them as `allowOrgOverride: true` for items **no managed package ships**: it opens runtime creation of a type whose registry entry allows none. It never opens an organization-scoped write: since [ADR-0131](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0131-total-organization-ownership-no-null-organization-id.md) D6 the metadata protocol refuses every organization-scoped write with `403 NOT_OVERRIDABLE`. It **never opens an item a managed package ships**: managed content is sealed ([ADR-0131](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0131-total-organization-ownership-no-null-organization-id.md) D6), so an overlay or a removal of a shipped flow, object, field, permission set, position or any other type without an environment overlay is refused with `403 NOT_OVERRIDABLE` whether the hatch is set or not. Customize managed content through its type's own route instead: an environment overlay for `view`, `dashboard`, `report`, `translation` and `email_template`; disable, or clone under a new name, for a flow; clone for a permission set. An overlay row this hatch wrote before over a managed item is kept at rest but **not served**: every read takes the package's definition, and boot names each such row, per type (`[metadata_sealed_overlay_unserved]`), with the two remedies — re-express the change as a new item under a new name, or delete the row. A stored fork of a packaged permission set is the one exception: it is still served, with its detection reading and Discard Overlay. Removing that permission-set row restores the package's definition. A stored row over a managed item of the other types that merge overlays at read (`position`, `page`, `app`, `dataset`, `book`, `tool`, `skill`) can still be removed, but removing it changes nothing that is served: no read serves it, so the package's definition is already served. See [ADR-0005](https://github.com/objectstack-ai/objectstack/blob/main/docs/adr/0005-metadata-customization-overlay.md). |
 
 ---