From d8dee94891193033783d97da118eccba93dcb703 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 7 Aug 2026 17:31:28 -0400 Subject: [PATCH 01/10] fix(metadata): write constraint-merge's NUL join delimiter as an escape, not a raw byte MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit childRuleId() joins its composite-key parts on NUL — sound as a delimiter (cannot collide with a real name) — but the delimiter was written as a raw 0x00 byte inside the string literal, which made the whole file test as binary: file(1) reported it as "data", and binary-skipping text-search tools ignored it silently, returning no output rather than an error. A sibling instance of this same defect in the Java port caused a cross-port code audit to mis-classify that file. This was the last NUL-bearing tracked source file in the repo. The literal is now the escape sequence backslash-u0000 (unambiguous in TS; avoids the deprecated-octal reading backslash-0 can take before a digit). The runtime string is byte-identical, confirmed by the constraint-merge tests, and the file now tests as UTF-8 text. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE --- .../packages/metadata/src/constraint-merge.ts | Bin 7532 -> 7537 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/server/typescript/packages/metadata/src/constraint-merge.ts b/server/typescript/packages/metadata/src/constraint-merge.ts index b1ce607b338d571ef55a30bc7833ae0ff0f36c3b..430869501857adc7c73c288ef24ee0873b78d81c 100644 GIT binary patch delta 19 ZcmaE3_0ej>LLRo5QUf5^yol$N002`%2Y~

<{xj|K1m From 93955c3526609252fee11c13666fb2801bdec3d8 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 7 Aug 2026 17:31:49 -0400 Subject: [PATCH 02/10] feat(metamodel): shrink source.rdb @role to primary|replica (#212) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #212's actionable content. @role registered six members (primary, replica, index, cache, publish, mirror), but cross-port verification established that every read of @role in all five ports is an equality test against primary — no consumer ever dispatched on the other five, so the consumed information content is one bit and the four unused members were indistinguishable from replica. Per the 2026-08-05 ruling, allowedValues shrinks to ["primary", "replica"]; index/cache/publish/mirror become reserved-not-registered (the ADR-0040 treatment) with the re-entry bar recorded in the registered description: a role member enters the registry only when a shipping consumer dispatches on it. An adopter scan over this repo, the public reference app and downstream consumer models found zero uses of the four retired members. Mechanics, all five ports (Kotlin composes from the JVM registry): - spec/metamodel/db.json + the regenerated TS embedded definition + the committed Python/C# spec copies carry the two-member list. - Per-port constants for the four retired members are deleted outright (TS source-constants.ts, Java MetaSource/RdbSource, C# SourceConstants, Python source_constants); the roles arrays shrink to the two survivors. C# SourceSchema's in-code description byte-matches db.json. - expected-registry.json and the metamodel docs regenerate; both diffs are confined to the @role attribute block/row. - New conformance fixture error-source-role-reserved: a retired member now fails load with ERR_BAD_ATTR_VALUE via the existing generic allowedValues check — no new error code, no new validation pass. - spec/roadmap.md marks #212 shipped (the ADR-0007/0028/FR-024 doc amendments landed with the ruling). No behavior dispatch changes anywhere (none existed); codegen golden output is byte-identical. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE --- docs/CONFORMANCE.md | 4 ++-- .../expected-errors.json | 15 +++++++++++++++ .../input/meta.demo.json | 19 +++++++++++++++++++ .../error-source-role-reserved/providers.json | 1 + .../metamodel-docs/expected/types/source.md | 2 +- .../expected-registry.json | 8 ++------ .../Persistence/Source/SourceConstants.cs | 8 -------- .../Persistence/Source/SourceSchema.cs | 2 +- .../csharp/MetaObjects/SpecMetamodel/db.json | 2 +- .../com/metaobjects/source/MetaSource.java | 8 ++------ .../com/metaobjects/source/RdbSource.java | 3 +-- .../metaobjects/source/MetaSourceTest.java | 4 ---- .../persistence/source/source_constants.py | 8 -------- .../src/metaobjects/spec_metamodel/db.json | 2 +- server/python/tests/unit/test_meta_source.py | 5 +++-- .../tests/unit/test_one_primary_source.py | 4 ++-- .../persistence/db/db-definition.embedded.ts | 8 ++------ .../persistence/source/source-constants.ts | 8 -------- spec/metamodel/db.json | 2 +- spec/roadmap.md | 2 +- 20 files changed, 55 insertions(+), 60 deletions(-) create mode 100644 fixtures/conformance/error-source-role-reserved/expected-errors.json create mode 100644 fixtures/conformance/error-source-role-reserved/input/meta.demo.json create mode 100644 fixtures/conformance/error-source-role-reserved/providers.json diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 8e5e01e94..5ee052ab7 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -25,7 +25,7 @@ regenerate with `ls -d fixtures//*/ | wc -l`. | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 262 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 263 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 15 | 15 / 15 | 14 / 15 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 14 / 15 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 15 / 15 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -69,7 +69,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (262) +### `fixtures/conformance/` — metamodel loader + canonical serializer (263) | Fixture prefix | Feature doc | |---|---| diff --git a/fixtures/conformance/error-source-role-reserved/expected-errors.json b/fixtures/conformance/error-source-role-reserved/expected-errors.json new file mode 100644 index 000000000..3bc26d689 --- /dev/null +++ b/fixtures/conformance/error-source-role-reserved/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[1]['source.rdb']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-source-role-reserved/input/meta.demo.json b/fixtures/conformance/error-source-role-reserved/input/meta.demo.json new file mode 100644 index 000000000..2ead870eb --- /dev/null +++ b/fixtures/conformance/error-source-role-reserved/input/meta.demo.json @@ -0,0 +1,19 @@ +{ + "metadata.root": { + "package": "demo", + "children": [ + { + "object.entity": { + "name": "Country", + "children": [ + { "source.rdb": { "@table": "countries" } }, + { "source.rdb": { "@kind": "view", "@view": "v_countries", "@role": "publish" } }, + { "field.uuid": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-source-role-reserved/providers.json b/fixtures/conformance/error-source-role-reserved/providers.json new file mode 100644 index 000000000..aefa027b6 --- /dev/null +++ b/fixtures/conformance/error-source-role-reserved/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types", "metaobjects-db"] diff --git a/fixtures/metamodel-docs/expected/types/source.md b/fixtures/metamodel-docs/expected/types/source.md index 4fb46a3c6..ae2cb51fd 100644 --- a/fixtures/metamodel-docs/expected/types/source.md +++ b/fixtures/metamodel-docs/expected/types/source.md @@ -43,7 +43,7 @@ The relational-database paradigm source (ADR-0007): binds an object to a physica | `@materializedView` | string | no | | | metaobjects-db | Physical SQL materialized-view name for source.rdb @kind: "materializedView". Same internal slot as @table. | | `@parameterRef` | string | no | | | metaobjects-db | FR-015: name or FQN of an object.value describing the input shape of this source's callable interface. Permitted on @kind: "storedProc" / "tableFunction"; rejected on non-callable kinds (table / view / materializedView). Field children of the referenced object.value become the call-site parameter list in declaration order. Symmetric with template.@payloadRef in FR-004 — the typed-input pattern reuses object.value rather than minting a new parameter.* node type. | | `@proc` | string | no | | | metaobjects-db | Physical SQL stored-procedure name for source.rdb @kind: "storedProc". Same internal slot as @table. | -| `@role` | string | no | | `primary`, `replica`, `index`, `cache`, `publish`, `mirror` | metaobjects-db | Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror. | +| `@role` | string | no | | `primary`, `replica` | metaobjects-db | Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it. | | `@schema` | string | no | | | metaobjects-db | Optional database schema name (e.g. 'catalog', 'public'). Postgres defaults to 'public'; SQLite rejects any non-default value. | | `@sql` | string | no | | | metaobjects-db | FR-024/#208 escape valve — a hand-written SQL body the tool REGISTERS + fingerprints + drift-checks but never authors or parses. The body goes INSIDE `CREATE AS …` (never the CREATE wrapper, never the object name). Legal only on a read-only kind (not @kind: table); migrate lowers it on @kind: view (matview/proc/tableFunction: registered but not yet migrate-managed). Mutually exclusive with @unmanaged; forbids origin.* children (two sources of truth). | | `@table` | string | no | | | metaobjects-db | Physical SQL table name for source.rdb @kind: "table" (default). FR-016: Defaults from the source's bare structural `name` via the project's columnNamingStrategy when omitted, then from the owning entity's name. Pre-1.0 legacy spelling for view/materializedView/storedProc/tableFunction kinds during the transition; canonical-serializer rewrites to the kind-matching alias. | diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index b67d1fa7d..5da7846c4 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -3740,13 +3740,9 @@ "required": false, "allowedValues": [ "primary", - "replica", - "index", - "cache", - "publish", - "mirror" + "replica" ], - "description": "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror." + "description": "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it." }, { "name": "schema", diff --git a/server/csharp/MetaObjects/Persistence/Source/SourceConstants.cs b/server/csharp/MetaObjects/Persistence/Source/SourceConstants.cs index a2683bee8..8b2e71c99 100644 --- a/server/csharp/MetaObjects/Persistence/Source/SourceConstants.cs +++ b/server/csharp/MetaObjects/Persistence/Source/SourceConstants.cs @@ -167,19 +167,11 @@ public static class SourceConstants public const string SOURCE_ROLE_PRIMARY = "primary"; public const string SOURCE_ROLE_REPLICA = "replica"; - public const string SOURCE_ROLE_INDEX = "index"; - public const string SOURCE_ROLE_CACHE = "cache"; - public const string SOURCE_ROLE_PUBLISH = "publish"; - public const string SOURCE_ROLE_MIRROR = "mirror"; public static readonly string[] SOURCE_ROLES = [ SOURCE_ROLE_PRIMARY, SOURCE_ROLE_REPLICA, - SOURCE_ROLE_INDEX, - SOURCE_ROLE_CACHE, - SOURCE_ROLE_PUBLISH, - SOURCE_ROLE_MIRROR, ]; ///

Default @role when omitted (system of record). diff --git a/server/csharp/MetaObjects/Persistence/Source/SourceSchema.cs b/server/csharp/MetaObjects/Persistence/Source/SourceSchema.cs index de99abce3..d8ef8a94d 100644 --- a/server/csharp/MetaObjects/Persistence/Source/SourceSchema.cs +++ b/server/csharp/MetaObjects/Persistence/Source/SourceSchema.cs @@ -67,7 +67,7 @@ public static class SourceSchema ValueType: AttrConstants.ATTR_SUBTYPE_STRING, Required: false, AllowedValues: [.. SourceConstants.SOURCE_ROLES], - Description: "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror."), + Description: "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it."), new AttrSchema( Name: SourceConstants.SOURCE_ATTR_SCHEMA, diff --git a/server/csharp/MetaObjects/SpecMetamodel/db.json b/server/csharp/MetaObjects/SpecMetamodel/db.json index 4e609f358..6b188b9e9 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/db.json +++ b/server/csharp/MetaObjects/SpecMetamodel/db.json @@ -41,7 +41,7 @@ { "type": "attr", "subType": "string", "name": "proc", "min": 0, "max": 1, "description": "Physical SQL stored-procedure name for source.rdb @kind: \"storedProc\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "function", "min": 0, "max": 1, "description": "Physical SQL table-function name for source.rdb @kind: \"tableFunction\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "kind", "min": 0, "max": 1, "allowedValues": ["table", "view", "materializedView", "storedProc", "tableFunction"], "description": "The kind of database object this source represents: table (default, writable), view, materializedView, storedProc, or tableFunction. Non-table kinds are read-only." }, - { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica", "index", "cache", "publish", "mirror"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror." }, + { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it." }, { "type": "attr", "subType": "string", "name": "schema", "min": 0, "max": 1, "description": "Optional database schema name (e.g. 'catalog', 'public'). Postgres defaults to 'public'; SQLite rejects any non-default value." }, { "type": "attr", "subType": "string", "name": "sql", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — a hand-written SQL body the tool REGISTERS + fingerprints + drift-checks but never authors or parses. The body goes INSIDE `CREATE AS …` (never the CREATE wrapper, never the object name). Legal only on a read-only kind (not @kind: table); migrate lowers it on @kind: view (matview/proc/tableFunction: registered but not yet migrate-managed). Mutually exclusive with @unmanaged; forbids origin.* children (two sources of truth)." }, { "type": "attr", "subType": "boolean", "name": "unmanaged", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — this DB object is managed elsewhere (Flyway / a hand-migration owns its DDL). meta migrate does NOT create, drop, or drift-check it; verify --db reports it as external (declared). Legal on any @kind including table (the externally-managed-entity case). Mutually exclusive with @sql." }, diff --git a/server/java/metadata/src/main/java/com/metaobjects/source/MetaSource.java b/server/java/metadata/src/main/java/com/metaobjects/source/MetaSource.java index ee1a3915b..196764f44 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/source/MetaSource.java +++ b/server/java/metadata/src/main/java/com/metaobjects/source/MetaSource.java @@ -55,7 +55,7 @@ public abstract class MetaSource extends MetaData { /** Object kind: table / view / materializedView / storedProc / tableFunction. */ public static final String ATTR_KIND = "kind"; - /** Multi-source role: primary / replica / index / cache / publish / mirror. */ + /** Multi-source role: primary / replica. */ public static final String ATTR_ROLE = "role"; /** DB schema / namespace (Postgres default "public"; SQLite rejects non-default values). */ @@ -129,17 +129,13 @@ public abstract class MetaSource extends MetaData { public static final String ROLE_PRIMARY = "primary"; public static final String ROLE_REPLICA = "replica"; - public static final String ROLE_INDEX = "index"; - public static final String ROLE_CACHE = "cache"; - public static final String ROLE_PUBLISH = "publish"; - public static final String ROLE_MIRROR = "mirror"; /** Default role when {@code @role} is absent. */ public static final String DEFAULT_ROLE = ROLE_PRIMARY; /** All valid {@code @role} values. Used by {@code ValidationPhase} for enum-membership checks. */ public static final Set VALID_ROLES = Set.of( - ROLE_PRIMARY, ROLE_REPLICA, ROLE_INDEX, ROLE_CACHE, ROLE_PUBLISH, ROLE_MIRROR + ROLE_PRIMARY, ROLE_REPLICA ); // ----------------------------------------------------------------------- diff --git a/server/java/metadata/src/main/java/com/metaobjects/source/RdbSource.java b/server/java/metadata/src/main/java/com/metaobjects/source/RdbSource.java index 17b402b8d..18a6b66ca 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/source/RdbSource.java +++ b/server/java/metadata/src/main/java/com/metaobjects/source/RdbSource.java @@ -103,8 +103,7 @@ public static void registerTypes(MetaDataRegistry registry) { // @role — enum-constrained; withEnum also marks it as single. def.optionalAttributeWithConstraints(ATTR_ROLE) .ofType(StringAttribute.SUBTYPE_STRING) - .withEnum(ROLE_PRIMARY, ROLE_REPLICA, ROLE_INDEX, - ROLE_CACHE, ROLE_PUBLISH, ROLE_MIRROR); + .withEnum(ROLE_PRIMARY, ROLE_REPLICA); // @schema — optional, string, single value. def.optionalAttributeWithConstraints(ATTR_SCHEMA) diff --git a/server/java/metadata/src/test/java/com/metaobjects/source/MetaSourceTest.java b/server/java/metadata/src/test/java/com/metaobjects/source/MetaSourceTest.java index f25f9c86c..52aa475a5 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/source/MetaSourceTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/source/MetaSourceTest.java @@ -349,10 +349,6 @@ public void constantsAreCorrect() { assertEquals("ROLE_PRIMARY", "primary", MetaSource.ROLE_PRIMARY); assertEquals("ROLE_REPLICA", "replica", MetaSource.ROLE_REPLICA); - assertEquals("ROLE_INDEX", "index", MetaSource.ROLE_INDEX); - assertEquals("ROLE_CACHE", "cache", MetaSource.ROLE_CACHE); - assertEquals("ROLE_PUBLISH", "publish", MetaSource.ROLE_PUBLISH); - assertEquals("ROLE_MIRROR", "mirror", MetaSource.ROLE_MIRROR); assertEquals("DEFAULT_ROLE", "primary", MetaSource.DEFAULT_ROLE); } diff --git a/server/python/src/metaobjects/meta/persistence/source/source_constants.py b/server/python/src/metaobjects/meta/persistence/source/source_constants.py index 9a0bebdd7..166990df4 100644 --- a/server/python/src/metaobjects/meta/persistence/source/source_constants.py +++ b/server/python/src/metaobjects/meta/persistence/source/source_constants.py @@ -107,18 +107,10 @@ # --- @role values + default ------------------------------------------------- SOURCE_ROLE_PRIMARY = "primary" SOURCE_ROLE_REPLICA = "replica" -SOURCE_ROLE_INDEX = "index" -SOURCE_ROLE_CACHE = "cache" -SOURCE_ROLE_PUBLISH = "publish" -SOURCE_ROLE_MIRROR = "mirror" SOURCE_ROLES = ( SOURCE_ROLE_PRIMARY, SOURCE_ROLE_REPLICA, - SOURCE_ROLE_INDEX, - SOURCE_ROLE_CACHE, - SOURCE_ROLE_PUBLISH, - SOURCE_ROLE_MIRROR, ) # @role default when omitted (system of record). diff --git a/server/python/src/metaobjects/spec_metamodel/db.json b/server/python/src/metaobjects/spec_metamodel/db.json index 4e609f358..6b188b9e9 100644 --- a/server/python/src/metaobjects/spec_metamodel/db.json +++ b/server/python/src/metaobjects/spec_metamodel/db.json @@ -41,7 +41,7 @@ { "type": "attr", "subType": "string", "name": "proc", "min": 0, "max": 1, "description": "Physical SQL stored-procedure name for source.rdb @kind: \"storedProc\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "function", "min": 0, "max": 1, "description": "Physical SQL table-function name for source.rdb @kind: \"tableFunction\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "kind", "min": 0, "max": 1, "allowedValues": ["table", "view", "materializedView", "storedProc", "tableFunction"], "description": "The kind of database object this source represents: table (default, writable), view, materializedView, storedProc, or tableFunction. Non-table kinds are read-only." }, - { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica", "index", "cache", "publish", "mirror"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror." }, + { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it." }, { "type": "attr", "subType": "string", "name": "schema", "min": 0, "max": 1, "description": "Optional database schema name (e.g. 'catalog', 'public'). Postgres defaults to 'public'; SQLite rejects any non-default value." }, { "type": "attr", "subType": "string", "name": "sql", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — a hand-written SQL body the tool REGISTERS + fingerprints + drift-checks but never authors or parses. The body goes INSIDE `CREATE AS …` (never the CREATE wrapper, never the object name). Legal only on a read-only kind (not @kind: table); migrate lowers it on @kind: view (matview/proc/tableFunction: registered but not yet migrate-managed). Mutually exclusive with @unmanaged; forbids origin.* children (two sources of truth)." }, { "type": "attr", "subType": "boolean", "name": "unmanaged", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — this DB object is managed elsewhere (Flyway / a hand-migration owns its DDL). meta migrate does NOT create, drop, or drift-check it; verify --db reports it as external (declared). Legal on any @kind including table (the externally-managed-entity case). Mutually exclusive with @sql." }, diff --git a/server/python/tests/unit/test_meta_source.py b/server/python/tests/unit/test_meta_source.py index 67a3d2c34..80d534a39 100644 --- a/server/python/tests/unit/test_meta_source.py +++ b/server/python/tests/unit/test_meta_source.py @@ -76,8 +76,9 @@ def test_source_role_value_set() -> None: """@role value set — Tier-1 contract.""" assert SOURCE_ROLE_PRIMARY == "primary" assert SOURCE_ROLE_REPLICA == "replica" - # Cross-language vocabulary fixed at six members; assert content not order - assert set(SOURCE_ROLES) == {"primary", "replica", "index", "cache", "publish", "mirror"} + # Cross-language vocabulary is two members (#212); index/cache/publish/mirror + # are reserved-not-registered. Assert content not order. + assert set(SOURCE_ROLES) == {"primary", "replica"} def test_source_defaults() -> None: diff --git a/server/python/tests/unit/test_one_primary_source.py b/server/python/tests/unit/test_one_primary_source.py index 1c7276324..63c5e00a8 100644 --- a/server/python/tests/unit/test_one_primary_source.py +++ b/server/python/tests/unit/test_one_primary_source.py @@ -94,11 +94,11 @@ def test_write_through_one_primary_one_replica_ok() -> None: def test_no_primary_raises_err_source_no_primary() -> None: - """All sources are non-primary (replica/index/cache) → ERR_SOURCE_NO_PRIMARY.""" + """All sources are non-primary (replica) → ERR_SOURCE_NO_PRIMARY.""" codes = _load( _entity( {"source.rdb": {"@table": "v_products_a", "@kind": "view", "@role": "replica"}}, - {"source.rdb": {"@table": "v_products_b", "@kind": "view", "@role": "cache"}}, + {"source.rdb": {"@table": "v_products_b", "@kind": "view", "@role": "replica"}}, ) ) assert "ERR_SOURCE_NO_PRIMARY" in codes, f"Expected ERR_SOURCE_NO_PRIMARY; got {codes}" diff --git a/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts b/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts index 08e920b4e..e7bb4c76e 100644 --- a/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/persistence/db/db-definition.embedded.ts @@ -165,13 +165,9 @@ export const DB_DEFINITION: ProviderDefinition = { "max": 1, "allowedValues": [ "primary", - "replica", - "index", - "cache", - "publish", - "mirror" + "replica" ], - "description": "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror." + "description": "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it." }, { "type": "attr", diff --git a/server/typescript/packages/metadata/src/persistence/source/source-constants.ts b/server/typescript/packages/metadata/src/persistence/source/source-constants.ts index 151e021f9..44317484d 100644 --- a/server/typescript/packages/metadata/src/persistence/source/source-constants.ts +++ b/server/typescript/packages/metadata/src/persistence/source/source-constants.ts @@ -114,18 +114,10 @@ export const SOURCE_READ_ONLY_KINDS: ReadonlySet = new Set([ export const SOURCE_ROLE_PRIMARY = "primary"; export const SOURCE_ROLE_REPLICA = "replica"; -export const SOURCE_ROLE_INDEX = "index"; -export const SOURCE_ROLE_CACHE = "cache"; -export const SOURCE_ROLE_PUBLISH = "publish"; -export const SOURCE_ROLE_MIRROR = "mirror"; export const SOURCE_ROLES = [ SOURCE_ROLE_PRIMARY, SOURCE_ROLE_REPLICA, - SOURCE_ROLE_INDEX, - SOURCE_ROLE_CACHE, - SOURCE_ROLE_PUBLISH, - SOURCE_ROLE_MIRROR, ] as const; export type SourceRole = (typeof SOURCE_ROLES)[number]; diff --git a/spec/metamodel/db.json b/spec/metamodel/db.json index 4e609f358..6b188b9e9 100644 --- a/spec/metamodel/db.json +++ b/spec/metamodel/db.json @@ -41,7 +41,7 @@ { "type": "attr", "subType": "string", "name": "proc", "min": 0, "max": 1, "description": "Physical SQL stored-procedure name for source.rdb @kind: \"storedProc\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "function", "min": 0, "max": 1, "description": "Physical SQL table-function name for source.rdb @kind: \"tableFunction\". Same internal slot as @table." }, { "type": "attr", "subType": "string", "name": "kind", "min": 0, "max": 1, "allowedValues": ["table", "view", "materializedView", "storedProc", "tableFunction"], "description": "The kind of database object this source represents: table (default, writable), view, materializedView, storedProc, or tableFunction. Non-table kinds are read-only." }, - { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica", "index", "cache", "publish", "mirror"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record), replica, index, cache, publish, or mirror." }, + { "type": "attr", "subType": "string", "name": "role", "min": 0, "max": 1, "allowedValues": ["primary", "replica"], "description": "Role this source plays when an object has multiple sources: primary (default, system of record) or replica. The former members index, cache, publish and mirror are reserved-not-registered (ADR-0007 Amendment 2): a role member enters the registry only when a shipping consumer dispatches on it." }, { "type": "attr", "subType": "string", "name": "schema", "min": 0, "max": 1, "description": "Optional database schema name (e.g. 'catalog', 'public'). Postgres defaults to 'public'; SQLite rejects any non-default value." }, { "type": "attr", "subType": "string", "name": "sql", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — a hand-written SQL body the tool REGISTERS + fingerprints + drift-checks but never authors or parses. The body goes INSIDE `CREATE AS …` (never the CREATE wrapper, never the object name). Legal only on a read-only kind (not @kind: table); migrate lowers it on @kind: view (matview/proc/tableFunction: registered but not yet migrate-managed). Mutually exclusive with @unmanaged; forbids origin.* children (two sources of truth)." }, { "type": "attr", "subType": "boolean", "name": "unmanaged", "min": 0, "max": 1, "description": "FR-024/#208 escape valve — this DB object is managed elsewhere (Flyway / a hand-migration owns its DDL). meta migrate does NOT create, drop, or drift-check it; verify --db reports it as external (declared). Legal on any @kind including table (the externally-managed-entity case). Mutually exclusive with @sql." }, diff --git a/spec/roadmap.md b/spec/roadmap.md index 32a0ec522..eb1911a48 100644 --- a/spec/roadmap.md +++ b/spec/roadmap.md @@ -134,7 +134,7 @@ _(FR-032 was developed under the working number "FR-026" — see commit history; - **Group B — 1.0-freeze-window vocab decision (needs an owner ruling before the freeze):** - **#210 — [RULED 2026-08-05: DO IT, batched into the pre-1.0 consolidation, scope narrowed]** Retire the **assembly** origins (`aggregate`/`computed`/`collection`/`first`) on `object.value`, and widen `@payloadRef`/`@responseRef` to accept a **sourceless** `object.projection`. `origin.passthrough` **stays** — FR-015 parameter VOs are out of scope, because the loader already draws that line (FR-024 B5 in `validateOriginPaths`: a value's passthrough is *parameter lineage*, not an assembly path), and retiring it would silently drop the `ERR_PASSTHROUGH_TYPE_MISMATCH` check on proc args. The "total value purity" framing is dropped — ADR-0046's `@enforce: false` reference already made value rules conditional; the durable rule is **"passthrough on a value is lineage; assembly origins live on projections."** No new vocabulary (`origin.collection` is registered; a sourceless projection already loads), but `expected-registry.json` **does** change — `object.value`'s `rules` string says "by assembly" — so all five ports sync in lockstep. Timing: **not** during the Phase-1 window (the churn is in the payload tier, this repo's recent cross-port bug hotspot, and it would churn the very pillar the launch showcases); it rides the single ADR-0035 §3 breaking batch while 1.0 stays unscheduled. Blocked on two non-breaking preludes: **#270 — SHIPPED** (payload typing is declared-type-authoritative: the Kotlin, Python **and Java** payload emitters all derived a field's type, nullability, or nested-payload closure from its `origin.*` child — Java's `SpringPayloadGenerator` carried the identical origin dispatch, surfaced in review after the initial ruling named only Kotlin/Python — and all three now type exactly as if the origin child were absent, converging on the genuinely origin-blind TS/C# reference emitters; the `origin.collection` `@via` closure edge is deleted from the ADR-0044 name map in all three ports (gated positive + negative per port; the declared edge's target-subtype question — no loader constrains a nested `@objectRef` target's subtype today, TS/C#/Python don't filter, Kotlin/Java keep their pre-existing codegen filter as status quo — is adjudicated to #210's loader-validation ruling), and the stale CLAUDE.md open-questions pointer to the codegen-spring `KNOWN_GAPS` entry — an entry that had disappeared because Java *implemented* origin resolution, not because it was moot — is closed) and **#271** (port #248's persistability-from-source contract cross-port; prove every fan-out no-ops on a sourceless projection — four ports have never seen one, since #248 was npm-only). Reversal triggers: real adopters authoring assembly origins on payload VOs; #271 finding a port that needs new vocabulary for sourceless projections (ADR-0023 cost class ⇒ reopen); or 1.0 being scheduled imminently (then ship it and document the rule-of-thumb instead). - **Group C — post-1.0 backend generalization (design-now, build-later):** - - **#212 — [RULED 2026-08-05: RATIFIED with three amendments]** Events emit projections: resolve the **ADR-0007 vs ADR-0028 contradiction** (a stream is a channel, not a source). Drop `source.event`; model emission at the surface as `api.eventing`/`operation.event`/`binding.messaging` with a payload→projection ref; keep the "a stream becomes a source exactly when it is treated as addressable state" escape clause. Lands the CQRS+events symmetry: queries return projections, commands take values, events emit projections. **Amendment 1 — the doctrine's test is single-prong.** The proposed two-prong AND (addressable *and* drift-inspectable) fails in both directions: it would KEEP a schema-registry-backed Kafka topic and KILL `source.memory`, and it wrongly maims `document`/`keyValue`/`objectStore`/`timeSeries`/`graph`. Ratified wording: a `source.*` binds an object to **addressable state at rest** — readable on demand by a declared key or address, not merely observed in flight; **drift-inspectability is a per-paradigm `verify` capability, never an admission criterion**. Only `event`'s flow kinds (topic/stream) fall; `eventStore`/`changelog` are the escape clause; `vector` and `search` survive (both have fetch-by-id). **Amendment 2 — removing `@role: publish` is NOT docs-only**: it is live registered vocabulary (`expected-registry.json`, `spec/metamodel/db.json`, five ports' embedded definitions), so it rides the #210 consolidation batch. **Amendment 3** — FR-024 §7's channel row references a **projection**, not a value. **Sub-decision — [RULED 2026-08-05: option A + doctrine]** Shrink `@role` to `primary | replica`; document `index`/`cache`/`publish`/`mirror` as **reserved-not-registered** (the ADR-0040 treatment), and write the re-entry bar into the ADR-0007 amendment: *a role member enters the registry only when a shipping consumer dispatches on it.* Cross-port verification closed the gate — every read of `@role` in all five ports is an equality test against `primary` (Java OMDB has **zero** role usage; Kotlin `KotlinGenUtil` and Python `_read_source_name` are explicitly documented "role-agnostic", finding the replica by `@kind`), so the consumed information content is one bit and the four unused members are indistinguishable from `replica` to every consumer. A `@primary: boolean` was **rejected**: `@role` defaults to `primary` when omitted and the one-primary invariant rejects two primaries, so a *second* source must carry an explicit non-primary role — `replica` is the registered opt-out that makes multi-source declarable at all, and the enum cannot shrink below two without redesigning the default. `@role` survives ADR-0037 narrowly at step 3 (primacy is a tie-breaking *designation*, not derivable: `table(primary)` + `table(replica)` is legal and becomes inexpressible under any "the writable one is primary" derivation). Decided by the freeze asymmetry — removing a registered member post-1.0 is a **2.0 event**, re-adding a reserved one is **additive**, so pruning now is the reversible move — and by the template argument: `@role` is registered on `source.rdb` specifically, so the shrunk registration is what every future paradigm in this catalog copies. Recorded finding, not acted on: `@role` today is a **designation** mechanism, not a routing one — ADR-0007's "consumers route by `@role`" prose describes a dispatch no port ever built. **Required before the batch lands:** re-run ADR-0036's adopter scan + the public reference app — these are shipped registered members, and zero uses were verified in this repo only. Splits: the ADR-0007/0028/FR-024 doc amendment lands **now** (ADR-0035 §1 excludes reserved-but-unregistered from the compat surface, so no quiet-clock reset); the `@role` registry change rides the #210 batch; the eventing surface rides FR-024 to 1.1 (`api.*`/`operation.*`/`binding.*` are all unregistered today, so it is additive by construction). + - **#212 — SHIPPED (unreleased, closed)** — the actionable content (the `@role` shrink to `primary | replica`) landed in all five ports + `expected-registry.json` + a new `error-source-role-reserved` conformance fixture; the ADR-0007/0028/FR-024 doc amendments landed with the ruling. **[RULED 2026-08-05: RATIFIED with three amendments]** Events emit projections: resolve the **ADR-0007 vs ADR-0028 contradiction** (a stream is a channel, not a source). Drop `source.event`; model emission at the surface as `api.eventing`/`operation.event`/`binding.messaging` with a payload→projection ref; keep the "a stream becomes a source exactly when it is treated as addressable state" escape clause. Lands the CQRS+events symmetry: queries return projections, commands take values, events emit projections. **Amendment 1 — the doctrine's test is single-prong.** The proposed two-prong AND (addressable *and* drift-inspectable) fails in both directions: it would KEEP a schema-registry-backed Kafka topic and KILL `source.memory`, and it wrongly maims `document`/`keyValue`/`objectStore`/`timeSeries`/`graph`. Ratified wording: a `source.*` binds an object to **addressable state at rest** — readable on demand by a declared key or address, not merely observed in flight; **drift-inspectability is a per-paradigm `verify` capability, never an admission criterion**. Only `event`'s flow kinds (topic/stream) fall; `eventStore`/`changelog` are the escape clause; `vector` and `search` survive (both have fetch-by-id). **Amendment 2 — removing `@role: publish` is NOT docs-only**: it is live registered vocabulary (`expected-registry.json`, `spec/metamodel/db.json`, five ports' embedded definitions), so it rides the #210 consolidation batch. **Amendment 3** — FR-024 §7's channel row references a **projection**, not a value. **Sub-decision — [RULED 2026-08-05: option A + doctrine]** Shrink `@role` to `primary | replica`; document `index`/`cache`/`publish`/`mirror` as **reserved-not-registered** (the ADR-0040 treatment), and write the re-entry bar into the ADR-0007 amendment: *a role member enters the registry only when a shipping consumer dispatches on it.* Cross-port verification closed the gate — every read of `@role` in all five ports is an equality test against `primary` (Java OMDB has **zero** role usage; Kotlin `KotlinGenUtil` and Python `_read_source_name` are explicitly documented "role-agnostic", finding the replica by `@kind`), so the consumed information content is one bit and the four unused members are indistinguishable from `replica` to every consumer. A `@primary: boolean` was **rejected**: `@role` defaults to `primary` when omitted and the one-primary invariant rejects two primaries, so a *second* source must carry an explicit non-primary role — `replica` is the registered opt-out that makes multi-source declarable at all, and the enum cannot shrink below two without redesigning the default. `@role` survives ADR-0037 narrowly at step 3 (primacy is a tie-breaking *designation*, not derivable: `table(primary)` + `table(replica)` is legal and becomes inexpressible under any "the writable one is primary" derivation). Decided by the freeze asymmetry — removing a registered member post-1.0 is a **2.0 event**, re-adding a reserved one is **additive**, so pruning now is the reversible move — and by the template argument: `@role` is registered on `source.rdb` specifically, so the shrunk registration is what every future paradigm in this catalog copies. Recorded finding, not acted on: `@role` today is a **designation** mechanism, not a routing one — ADR-0007's "consumers route by `@role`" prose describes a dispatch no port ever built. **Adopter scan re-run before landing (2026-08-06):** ADR-0036's scan + the public reference app + downstream consumer models — zero uses of the four retired members. Splits: the ADR-0007/0028/FR-024 doc amendments **landed** with the ruling (ADR-0035 §1 excludes reserved-but-unregistered from the compat surface, so no quiet-clock reset); the `@role` registry shrink **shipped** with the #210 consolidation batch (loaders reject a retired member via the generic `allowedValues` check → `ERR_BAD_ATTR_VALUE`); the eventing surface rides FR-024 to 1.1 (`api.*`/`operation.*`/`binding.*` are all unregistered today, so it is additive by construction). - **#211** — **Backend-agnostic projection materialization** (one derivation spec, N lowerings): generalize `SelectSpec` from RDB-only to document/search/… with an origin × (paradigm, `@kind`) capability matrix + load-time errors. Large FR; its foundation is Group A landing with the "semantic, not RDB-specific" lens (#207/#208/#209 become the first lowerings). Post-1.0; composes with FR-034 (ecosystem tier). - **FR-019 — Shared + externally-provided enums.** Stop redeclaring a `field.enum` inline in every consuming entity: a package-level abstract `field.enum` materializes ONE standalone enum type per port (the existing D6 `extends` reuse vocabulary), and **`@provided: true`** — a provenance flag on the named-type *declaration* (not the field), shared cross-type with value objects — references an existing hand-written type instead of emitting one (per-port namespace via codegen config, never a metadata FQN — retires the C#-only `@csEnumType`). Decision in [ADR-0026](decisions/ADR-0026-shared-and-provided-named-types.md); implementation spec `docs/superpowers/specs/2026-06-06-fr-019-shared-and-provided-enums-design.md`. (Generators are now subclass-extensible across all 5 ports, so this lands on open seams.) - **FR-020 — `@inheritance` persistence strategy (single-table vs joined).** TPH (one physical table, nullable subtype columns) is the single implicit strategy today (FR-014/FR-017). FR-020 adds an `@inheritance` attr to choose **joined** (a base table + per-subtype tables, joined on read) as an alternative — **additive, default stays TPH** (no breaking change). Status: design (proposed, sequenced after FR-017, which shipped). Design: `docs/superpowers/specs/2026-06-07-fr-020-inheritance-strategy-design.md`. *(Was not previously surfaced in this roadmap.)* From cb991a25be0f9ec78baf220744b1d32d9615b1b1 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 7 Aug 2026 17:42:29 -0400 Subject: [PATCH 03/10] fix(metadata): derive the Java @role diagnostic from VALID_ROLES + amend stale research-doc claim (#212) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-ups on the #212 @role shrink, both stale six-member enumerations the constant-name straggler grep could not see (they were free-text literals, not constant reads): - ValidationPhase.validateSourceNode emitted a self-contradictory error on the exact path the new error-source-role-reserved fixture exercises: "@role 'publish' is not a valid value; allowed: primary, replica, index, cache, publish, mirror" — a hand-written allowed-list listing the rejected value among the allowed ones, and a cross-port diagnostic divergence (TS derives its list from the registry spec). The message now derives from MetaSource.VALID_ROLES via a TreeSet (sorted — Set.of iteration order is unspecified, and a non-deterministic error string would be its own defect), so it cannot drift from the registered set again. The stale block comment above it is corrected too. The sibling hand-written @kind list is pre-existing and deliberately left alone. - docs/superpowers/specs/2026-08-02-multi-persistence-architecture-research.md asserted present-tense that source-constants.ts "already ships" the six-member SOURCE_ROLES — now false, and this file names the very source file the shrink changed. Amended with a dated header note in the same style as the 2026-05-23 source-v2 design spec (specs asserting current behavior get amended; plans stay immutable records), without rewriting the research. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE --- ...2026-08-02-multi-persistence-architecture-research.md | 1 + .../java/com/metaobjects/loader/ValidationPhase.java | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-08-02-multi-persistence-architecture-research.md b/docs/superpowers/specs/2026-08-02-multi-persistence-architecture-research.md index 3f35bce67..d8ced0a8b 100644 --- a/docs/superpowers/specs/2026-08-02-multi-persistence-architecture-research.md +++ b/docs/superpowers/specs/2026-08-02-multi-persistence-architecture-research.md @@ -2,6 +2,7 @@ **Status:** Design research (no code changed) **Date:** 2026-08-02 +**Amended 2026-08-06** ([#212](https://github.com/metaobjectsdev/metaobjects/issues/212), ADR-0007 Amendment 2): `@role`'s registered vocabulary has since shrunk to `primary | replica` — `index`/`cache`/`publish`/`mirror` are now **reserved-not-registered** (`SOURCE_ROLES` in every port carries only the two survivors), so §1's "already ships the full role vocabulary" claim and the worked `@role: index` / `@role: cache` examples describe the pre-shrink state and are not currently loadable. **Question:** #248 made persistability derive from "declares a writable `source.*` child" — but that check, the single `dialect`, and the migrate/codegen rails all assume ONE relational store. How should MetaObjects model an object persisted to multiple, heterogeneous backends at once (two RDBs, RDB + search index, document store, KV cache), and how do rails scope to a specific store? --- diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index 8717da4f4..9232733b8 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -73,6 +73,7 @@ import java.util.Locale; import java.util.Map; import java.util.Set; +import java.util.TreeSet; /** * Post-load validation phase — runs after all sources are parsed and before the @@ -964,7 +965,7 @@ private static void validateDbColumnTypeNode(MetaData node) { // framework — same reason field.enum uses a post-load pass). // // @kind must be one of: table / view / materializedView / storedProc / tableFunction - // @role must be one of: primary / replica / index / cache / publish / mirror + // @role must be one of MetaSource.VALID_ROLES: primary / replica // // Missing attrs are fine (defaults apply); only explicitly-set bad values fail. // ========================================================================= @@ -1026,7 +1027,11 @@ private static void validateSourceNode(MetaData node) { ErrorMessageConstants.ERR_BAD_ATTR_VALUE + ": source '" + node.getName() + "' @role '" + role - + "' is not a valid value; allowed: primary, replica, index, cache, publish, mirror", + // Derived from the registered set (sorted for a deterministic + // message — Set.of iteration order is unspecified) so the + // diagnostic can never drift from MetaSource.VALID_ROLES again. + + "' is not a valid value; allowed: " + + String.join(", ", new TreeSet<>(MetaSource.VALID_ROLES)), ErrorCode.ERR_BAD_ATTR_VALUE, node.getSource()); } } From 6c73e77ad737d7d27c08af7b35fe5c9154d6e1b7 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 7 Aug 2026 18:42:19 -0400 Subject: [PATCH 04/10] feat(metamodel): retire assembly origins on object.value; widen payload refs to sourceless projections (#210) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #210, per the 2026-08-05 ruling. The durable rule: passthrough on a value is lineage; assembly origins live on projections. Half 1 — a field hosted on an object.value may no longer carry origin.aggregate, origin.computed, origin.collection or origin.first: all four loaders (TS / Python / Java — Kotlin inherits it — / C#) now reject the shape with ERR_SUBTYPE_RULE_VIOLATION at the offending origin node, driven by a new ASSEMBLY_ORIGIN_SUBTYPES constant per port. origin.passthrough STAYS legal on a value — FR-015 parameter lineage, the FR-024 B5 exemption — and keeps its ERR_PASSTHROUGH_TYPE_MISMATCH check. The now-unreachable value-host branches inside the aggregate/first validation arms are removed. Half 2 — @payloadRef / @responseRef accept an object.value OR a SOURCELESS object.projection ("sourceless" per the #248 persistability contract: no declared/inherited source.* child; unambiguous for a concrete projection since ERR_PROJECTION_INHERITED_SOURCE). A sourced projection stays ERR_INVALID_TEMPLATE. Message widened identically in all four loaders: "does not resolve to an object.value or sourceless object.projection at root". Codegen resolvers widen in all five ports (TS payload-codegen was already subtype-blind — doc updated; TS prompt-render's batch filter, Python resolve_payload_vo, Java SpringNaming.resolveValueObjectRef, Kotlin KotlinGenUtil.resolveValueObjectRef, C# RenderHelperGenerator. ResolveValueObject). Payload emitters walk projection fields with resolving accessors (ADR-0039) — pinned by the extends-bound field in the new positive fixture across every port's runner. NESTED payload targets stay value-only — a deliberate decision, not an omission: a payload field's `field.object @objectRef` must resolve to an object.value, now enforced FAIL-CLOSED in all four loaders (ERR_SUBTYPE_RULE_VIOLATION on the field node; the #219/ADR-0044-adjudicated gap where TS/C#/Python codegen accepted an entity nested target while Kotlin/Java filtered). The per-port codegen filters stay as belt-and-braces. Registry: object.value's rules + description drop the "by assembly" construction mode and state the new rule; object.projection's rules gain the complement sentence. expected-registry.json, the embedded TS definition, the Python/C# spec copies and the metamodel docs regenerate — diffs confined to those strings. Seven new conformance fixtures gate all four loaders: error-value-origin-{aggregate,computed,collection,first}, template-payload-ref-sourceless-projection (positive, expected + expected-effective), error-template-payload-ref-sourced-projection, error-payload-nested-object-ref-entity. Plus per-port loader unit tests (TS + Python). Re-hosted as sourceless projections: the codegen-kotlin payload-with-origins fixture (its snapshot is byte-identical — payload emitters are declared-type-authoritative per #270), origin-collection-simple, the examples/advanced-modeling payload (generated output regenerated under its drift gate; its config filter now keys on persistability), and every port test that loaded an assembly origin on a value host (TS x5 files, Kotlin x7 models, Java Spring x11 hosts, C# x4 models). Python payload-emitter tests construct trees programmatically (no loader) and stay as #270's defense-in-depth pins. Docs: ADR-0028 amended (assembly origins leave object.value); roadmap #210 marked shipped; new migration guide docs/features/migrations/value-assembly-origins-and-source-role-shrink.md covering this change AND the #212 @role shrink; templates-and-payloads.md target-set statements widened; CONFORMANCE.md fixture count 263 -> 270. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE --- docs/CONFORMANCE.md | 2 +- ...assembly-origins-and-source-role-shrink.md | 162 +++++++++++++++++ docs/features/templates-and-payloads.md | 34 ++-- examples/advanced-modeling/README.md | 7 +- .../advanced-modeling/metaobjects.config.ts | 22 ++- .../metaobjects/meta.prompts.yaml | 21 ++- .../generated/ProgramDescriptionPayload.ts | 2 +- .../src/generated/prompts.ts | 7 - .../expected-errors.json | 15 ++ .../input/meta.demo.json | 40 +++++ .../expected-errors.json | 17 ++ .../input/meta.demo.json | 35 ++++ .../expected-errors.json | 15 ++ .../input/meta.demo.json | 23 +++ .../expected-errors.json | 15 ++ .../input/meta.demo.json | 33 ++++ .../expected-errors.json | 15 ++ .../input/meta.demo.json | 27 +++ .../expected-errors.json | 15 ++ .../input/meta.demo.json | 29 +++ .../origin-collection-simple/expected.json | 2 +- .../input/meta.ai.json | 2 +- .../expected-effective.json | 65 +++++++ .../expected.json | 64 +++++++ .../input/meta.demo.json | 35 ++++ fixtures/metamodel-docs/expected/INDEX.md | 2 +- .../metamodel-docs/expected/types/object.md | 6 +- .../expected-registry.json | 6 +- .../MetaObjects.Codegen.Tests/DemoTests.cs | 2 +- .../PayloadCodegenTests.cs | 4 +- .../PayloadGeneratorTests.cs | 5 +- .../Generators/RenderHelperGenerator.cs | 14 +- .../MetaObjects.Codegen/PayloadCodegen.cs | 3 +- .../MetaObjects/Loader/ValidationPasses.cs | 98 ++++++++-- .../Persistence/Origin/OriginConstants.cs | 14 ++ .../MetaObjects/SpecMetamodel/object.json | 6 +- .../kotlin/KotlinExtractorGenerator.kt | 2 +- .../generator/kotlin/KotlinGenUtil.kt | 34 +++- .../kotlin/KotlinOutputParserGenerator.kt | 6 +- .../kotlin/KotlinOutputPromptGenerator.kt | 6 +- .../generator/kotlin/KotlinGenUtilTest.kt | 4 +- .../kotlin/KotlinPayloadGeneratorTest.kt | 17 +- .../fixtures/payload-with-origins/meta.json | 4 +- .../spring/LlmTraceHelperGenerator.java | 2 +- .../generator/spring/SpringNaming.java | 29 ++- .../spring/SpringPayloadGeneratorTest.java | 22 +-- .../metaobjects/loader/ValidationPhase.java | 103 +++++++++-- .../com/metaobjects/origin/MetaOrigin.java | 15 ++ .../generators/payload_vo_generator.py | 36 +++- .../generators/render_helper_generator.py | 8 +- .../generators/trace_helper_generator.py | 6 +- .../metaobjects/loader/validation_passes.py | 122 +++++++++++-- .../persistence/origin/origin_constants.py | 13 ++ .../metaobjects/spec_metamodel/object.json | 6 +- .../codegen/test_trace_helper_generator.py | 5 +- .../test_template_payload_target_210.py | 158 ++++++++++++++++ .../cli/test/integration/verify.test.ts | 4 +- .../cli/test/unit/payload-field-tree.test.ts | 4 +- .../src/generators/prompt-render-file.ts | 38 +++- .../codegen-ts/src/payload-codegen.ts | 4 +- .../codegen-ts/test/fr004-verify-demo.test.ts | 4 +- .../codegen-ts/test/payload-codegen.test.ts | 11 +- .../core/object/object-definition.embedded.ts | 6 +- .../metadata/src/loader/validation-passes.ts | 112 ++++++++++-- .../persistence/origin/origin-constants.ts | 12 ++ .../metadata/test/origin-collection.test.ts | 6 +- .../test/template-payload-target-210.test.ts | 170 ++++++++++++++++++ ...object-taxonomy-projection-value-purity.md | 35 +++- spec/metamodel/object.json | 6 +- spec/roadmap.md | 2 +- 70 files changed, 1681 insertions(+), 195 deletions(-) create mode 100644 docs/features/migrations/value-assembly-origins-and-source-role-shrink.md create mode 100644 fixtures/conformance/error-payload-nested-object-ref-entity/expected-errors.json create mode 100644 fixtures/conformance/error-payload-nested-object-ref-entity/input/meta.demo.json create mode 100644 fixtures/conformance/error-template-payload-ref-sourced-projection/expected-errors.json create mode 100644 fixtures/conformance/error-template-payload-ref-sourced-projection/input/meta.demo.json create mode 100644 fixtures/conformance/error-value-origin-aggregate/expected-errors.json create mode 100644 fixtures/conformance/error-value-origin-aggregate/input/meta.demo.json create mode 100644 fixtures/conformance/error-value-origin-collection/expected-errors.json create mode 100644 fixtures/conformance/error-value-origin-collection/input/meta.demo.json create mode 100644 fixtures/conformance/error-value-origin-computed/expected-errors.json create mode 100644 fixtures/conformance/error-value-origin-computed/input/meta.demo.json create mode 100644 fixtures/conformance/error-value-origin-first/expected-errors.json create mode 100644 fixtures/conformance/error-value-origin-first/input/meta.demo.json create mode 100644 fixtures/conformance/template-payload-ref-sourceless-projection/expected-effective.json create mode 100644 fixtures/conformance/template-payload-ref-sourceless-projection/expected.json create mode 100644 fixtures/conformance/template-payload-ref-sourceless-projection/input/meta.demo.json create mode 100644 server/python/tests/loader/test_template_payload_target_210.py create mode 100644 server/typescript/packages/metadata/test/template-payload-target-210.test.ts diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 5ee052ab7..98a34ee98 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -25,7 +25,7 @@ regenerate with `ls -d fixtures//*/ | wc -l`. | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 263 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 270 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 15 | 15 / 15 | 14 / 15 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 14 / 15 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 15 / 15 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | diff --git a/docs/features/migrations/value-assembly-origins-and-source-role-shrink.md b/docs/features/migrations/value-assembly-origins-and-source-role-shrink.md new file mode 100644 index 000000000..5d771e3a8 --- /dev/null +++ b/docs/features/migrations/value-assembly-origins-and-source-role-shrink.md @@ -0,0 +1,162 @@ +# Migrating value-hosted assembly origins (#210) and retired `@role` members (#212) + +This release line carries two coordinated breaking metamodel changes. Both fail +at **load time** with a clear error — nothing changes silently — and both have a +mechanical rewrite. + +1. **Assembly origins leave `object.value` (#210).** A field hosted on an + `object.value` may no longer carry `origin.aggregate`, `origin.computed`, + `origin.collection` or `origin.first`. Re-host the payload as a + **sourceless `object.projection`** — `@payloadRef`/`@responseRef` now accept + one. +2. **`source.rdb @role` shrinks to `primary | replica` (#212).** The + `index` / `cache` / `publish` / `mirror` members are retired + (reserved-not-registered, the ADR-0040 treatment). + +--- + +## 1. Assembly origins leave `object.value` (#210) + +### What changed + +The *assembly* origins — `origin.aggregate`, `origin.computed`, +`origin.collection`, `origin.first` — are now illegal on a field hosted by an +`object.value`, in all four loaders. **`origin.passthrough` stays legal on a +value**: there it declares FR-015 *parameter lineage* (e.g. a stored-proc +argument tracing back to an entity column), not an assembly path, and it keeps +its `ERR_PASSTHROUGH_TYPE_MISMATCH` type-preservation check. + +In exchange, the template-level payload references widen: +`template.* @payloadRef` and `@responseRef` now accept a **sourceless +`object.projection`** (no `source.*` child, own or inherited) in addition to an +`object.value`. A *sourced* projection as a payload target remains illegal. +**Nested** payload targets — a payload field's `field.object @objectRef` — stay +value-only. + +### Why + +The durable rule (ADR-0028, amended): **passthrough on a value is lineage; +assembly origins live on projections.** A value is a pure shape — constructed by +a caller or by embedding, never populated from a backing store. Rolling up, +computing, or collecting from related rows is *derivation*, and derivation is +what `object.projection` exists for. A payload that declares its fields' +derivations is a *read model assembled on the wire* — which is exactly a +sourceless projection. + +### The errors you'll see + +``` +ERR_SUBTYPE_RULE_VIOLATION: value object 'acme::ai::AuthorReport' field 'postCount' +hosts origin.aggregate — assembly origins (aggregate, computed, collection, first) +live on object.projection; a value is constructed by a caller or by embedding, +never assembled from a backing store. Re-host this field on a sourceless +object.projection; origin.passthrough (FR-015 parameter lineage) remains legal +on a value (#210, ADR-0028) +``` + +A nested payload `field.object @objectRef` pointing at anything other than an +`object.value` (an entity, or a projection) also fails: + +``` +ERR_SUBTYPE_RULE_VIOLATION: payload 'acme::ai::ReviewRequest' field 'author' +@objectRef 'acme::Author' resolves to object.entity — a nested payload target +must be an object.value (…) +``` + +### Rewrite rule + +Change the payload's host subtype from `object.value` to `object.projection`. +Fields, origins, and the template's `@payloadRef` all stay as they are. One +addition may be needed: an origin **without an explicit `@via`** derives its +base entity from the projection's extends anchors, so `extends`-bind at least +one field (or declare an extended identity) to name the base. + +**Before:** + +```jsonc +{ "object.value": { + "name": "AuthorReport", + "children": [ + { "field.string": { "name": "name", "children": [ + { "origin.passthrough": { "@from": "Author.name" } } ] } }, + { "field.long": { "name": "postCount", "children": [ + { "origin.aggregate": { "@agg": "count", "@of": "Post.id", "@via": "Author.posts" } } ] } } + ] +}}, +{ "template.prompt": { "name": "AuthorBio", "@payloadRef": "AuthorReport", "@textRef": "ai/bio" } } +``` + +**After:** + +```jsonc +{ "object.projection": { + "name": "AuthorReport", + "children": [ + { "field.string": { "name": "name", "extends": "Author.name", "children": [ + { "origin.passthrough": { "@from": "Author.name" } } ] } }, + { "field.long": { "name": "postCount", "children": [ + { "origin.aggregate": { "@agg": "count", "@of": "Post.id", "@via": "Author.posts" } } ] } } + ] +}}, +{ "template.prompt": { "name": "AuthorBio", "@payloadRef": "AuthorReport", "@textRef": "ai/bio" } } +``` + +(The `extends: "Author.name"` anchor is what lets the no-`@via` passthrough +derive `Author` as the base entity; every explicit-`@via` origin needs no +anchor.) + +A value that carries **only** `origin.passthrough` (an FR-015 parameter VO) +needs no change. Generated payload records/interfaces are byte-identical across +the re-host — every port's payload emitter is declared-type-authoritative +(#270), so the host subtype does not affect payload typing. + +### What does NOT change + +- `origin.passthrough` on `object.value` (parameter lineage). +- Assembly origins on `object.projection` and on `object.entity` read-views. +- Nested payload shapes (`field.object @objectRef` → `object.value`). +- Physical schema: a sourceless projection has no DDL, so `meta migrate` emits + nothing for it. + +--- + +## 2. `source.rdb @role` shrinks to `primary | replica` (#212) + +### What changed + +The `@role` enum on `source.rdb` is now exactly `primary | replica`. The four +retired members — `index`, `cache`, `publish`, `mirror` — are **reserved, not +registered** (the ADR-0040 treatment): documented for future re-entry, rejected +by every loader today. A role member re-enters the registry only when a +shipping consumer dispatches on it (ADR-0007 amendment). + +### The error you'll see + +``` +ERR_BAD_ATTR_VALUE: source.rdb attribute '@role' has value 'publish' which is +not one of the allowed values: primary, replica +``` + +(A single-source object with a retired role also reports `ERR_SOURCE_NO_PRIMARY` +— the retired member no longer counts as any role.) + +### Rewrite rule + +Every read of `@role` in every port is an equality test against `primary`, so +the four retired members were always indistinguishable from `replica` to every +consumer. The rewrite is mechanical: replace the retired member with `replica`. + +**Before:** + +```jsonc +{ "source.rdb": { "@kind": "view", "@view": "v_orders_search", "@role": "index" } } +``` + +**After:** + +```jsonc +{ "source.rdb": { "@kind": "view", "@view": "v_orders_search", "@role": "replica" } } +``` + +`@role: primary` (or omitting `@role` — `primary` is the default) is unchanged. +No migration SQL is emitted for either change: `@role` never affected DDL. diff --git a/docs/features/templates-and-payloads.md b/docs/features/templates-and-payloads.md index f956da972..757a0d597 100644 --- a/docs/features/templates-and-payloads.md +++ b/docs/features/templates-and-payloads.md @@ -19,9 +19,10 @@ This buys four guarantees: 4. **Cross-language conformance** — a Python eval renders exactly what the Java production server sends. -The vocabulary is `template.*` (the renderable unit) over a declared -`object.value` payload shape. Mustache is the chosen template engine — it has -the only published cross-language spec + conformance suite. +The vocabulary is `template.*` (the renderable unit) over a declared payload +shape — an `object.value`, or (since #210) a **sourceless** `object.projection`. +Mustache is the chosen template engine — it has the only published +cross-language spec + conformance suite. ## Two template subtypes @@ -34,7 +35,7 @@ Both carry the same generic attributes: | Attr | Required | Purpose | |---|---|---| -| `@payloadRef` | yes | The `object.value` view-object declaring the payload shape | +| `@payloadRef` | yes | The `object.value` — or sourceless `object.projection` (#210) — declaring the payload shape | | `@textRef` | yes | The 2-layer logical reference `group/source` resolved by a provider | | `@format` | no | `text` / `html` / `xml` / `csv` / `json` / `markdown` / `spreadsheet` — drives the escaper. Default: `text`. | | `@maxChars` | no | Build-time size budget | @@ -43,9 +44,10 @@ Both carry the same generic attributes: ## Payload fields are declared -A payload is an `object.value` view-object whose fields DECLARE the payload's -shape — a prompt's payload is a typed projection you author, so payload bloat -shows up as a diff. Every port's payload codegen is **declared-type-authoritative +A payload is an `object.value` — or a **sourceless** `object.projection` +(#210: no `source.*` child, own or inherited) — whose fields DECLARE the +payload's shape; a prompt's payload is a typed projection you author, so +payload bloat shows up as a diff. Every port's payload codegen is **declared-type-authoritative (#270)**: a field's generated type comes only from its declared `field.` + `isArray` + `@objectRef`, and a nested payload is a declared `field.object @objectRef` to another `object.value` (`isArray: true` for a list). An `origin.*` @@ -53,11 +55,17 @@ child on a payload field is **ignored for typing** — it never changes the generated type, nullability, or the nested-payload set. The caller supplies the field values at render time. -Derivation and assembly belong to **projection** read models (`object.projection` -over an entity), which carry the origin vocabulary — `origin.passthrough`, -`origin.aggregate` (incl. the `any` / `all` quantifiers and the `collect` array -rollup), `origin.computed` (a closed `@expr` grammar) and `origin.first` (#195) — -see [source-kinds.md](source-kinds.md). +Derivation and assembly belong to **projections** (`object.projection`), which +carry the origin vocabulary — `origin.passthrough`, `origin.aggregate` (incl. +the `any` / `all` quantifiers and the `collect` array rollup), `origin.computed` +(a closed `@expr` grammar) and `origin.first` (#195) — see +[source-kinds.md](source-kinds.md). #210 draws the host line hard: a +value-hosted field may carry **only `origin.passthrough`** (FR-015 parameter +lineage); the assembly origins (`aggregate` / `computed` / `collection` / +`first`) on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`. An +origin-declared payload lives on a **sourceless projection** instead, and +`@payloadRef` / `@responseRef` accept it at the template level. Nested payload +targets (a payload field's `field.object @objectRef`) stay value-only. ## Authoring @@ -531,7 +539,7 @@ for the per-port pass/skip ledger. ## See also -- [entities.md](entities.md) — `object.value` is the payload's host type +- [entities.md](entities.md) — `object.value` / sourceless `object.projection` are the payload host types (#210) - [field-types.md](field-types.md) — fields in payload VOs - [source-kinds.md](source-kinds.md) — `source.rdb` `@kind: "view"` for materialized payloads (FR-003) - [migrations-and-drift.md](migrations-and-drift.md) — the verify pillar diff --git a/examples/advanced-modeling/README.md b/examples/advanced-modeling/README.md index bcfd8865a..94fb94561 100644 --- a/examples/advanced-modeling/README.md +++ b/examples/advanced-modeling/README.md @@ -34,7 +34,7 @@ through the real CLI path. It does **not** re-prove the patterns' *behavior* | 1 | **Projections** (`object.projection`, `origin.*`) | `metaobjects/meta.catalog.yaml:127-165` — `ProgramSummary`: `origin.passthrough` (a join to `Author`, `:138`), `origin.aggregate @agg:count` (`:143-146`), `origin.aggregate @agg:sum` + `filter` (`:153-157`), `origin.computed @expr` (`:163-164`) | `src/generated/ProgramSummary.ts` (the `pgView` declaration + read schema), `src/generated/ProgramSummary.routes.ts` (read-only routes — no POST/PATCH/DELETE) | | 2 | **Entity views** (`view.*` control family) | `metaobjects/meta.catalog.yaml:25-93` — `Program`: `field.enum` (`:31-33`, → `