feat(metamodel): consolidate projection/payload vocabulary (#210, #212)

[dmealing]

Intent

Units C+D of the projection/payload vocabulary batch — the pre-1.0 BREAKING consolidation, closing #212 and #210, shipping as a coordinated 0.21.0 / Maven 7.21.0 MINOR across all four registries.

THREE breaking changes for metadata authors:
(1) Assembly origins (origin.aggregate/computed/collection/first) are now ILLEGAL on an object.value-hosted field — ERR_SUBTYPE_RULE_VIOLATION. origin.passthrough deliberately STAYS legal on a value: there it is FR-015 parameter lineage, not assembly, and the loaders already drew exactly that line via the FR-024 B5 value-host exemption. Retiring it too would have silently dropped the ERR_PASSTHROUGH_TYPE_MISMATCH check on stored-proc arguments. The durable rule encoded is 'passthrough on a value is lineage; assembly origins live on projections' (ADR-0028).
(2) source.rdb @ROLE shrinks from six registered members to primary|replica; index/cache/publish/mirror become reserved-not-registered. Justification: no port ever built the role-routing dispatch these anticipated — every read of @ROLE in all five ports is an equality test against primary, so the consumed information was one bit. An adopter scan across this repo, the public reference app and downstream consumer models found zero uses. Pruning now is the reversible direction (removing a registered member post-1.0 would be a 2.0 event; re-adding a reserved one is additive).
(3) A payload's nested field.object @objectref must target an object.value. TypeScript, C# and Python previously accepted a non-value target AND emitted code from it, so this is a genuine third break, not a tightening. It is implemented as ONE fail-closed loader rule rather than five per-port codegen filters, deliberately — the ports disagreed, and an independent adjudication ruled that filtering one port's payload closure without its extract closure would emit mappers constructing classes that no longer exist.

The migration path is additive: @payloadRef/@responseRef now accept a sourceless object.projection, so a payload that was assembling values re-hosts as a projection and keeps its origins.

Deliberate design choices a reviewer could not infer: the assembly-origin ban is implemented as a named subtype SET hoisted above the origin dispatch in every loader, so cross-port coverage is a property of the constant rather than of four separate branches. Nested payload targets deliberately stay value-only — the widen covers template-level refs only, and that is recorded in the commit. No new vocabulary, attributes or error codes anywhere (ADR-0023); the registry diff is one shrunk allowedValues plus three object.value/object.projection doctrinal strings.

Also in this branch: a raw NUL byte in constraint-merge.ts became an escape. That is not cosmetic housekeeping — a companion instance in the Java port made that file test as BINARY, so this environment's binary-skipping grep silently returned nothing for it, which mis-scoped an earlier unit onto a false premise. These were the last two NUL-bearing files in the repo.

Cut as MINOR not PATCH because pre-1.0 a caret range ^0.20.x resolves <0.21.0 — a patch would be auto-adopted on a routine npm update and adopters' metadata would stop loading with no deliberate action.

Already cleared by: per-unit task reviews with fix rounds, a release-grade whole-branch review that initially said DON'T SHIP (it found a shipped agent-context skill instructing agents to author now-illegal metadata — that skill ships inside @metaobjectsdev/sdk and installs into consumer repos), a doctrine-surface fix wave, and a scoped re-review that independently re-ran the sweep with a spelling-agnostic pattern and char-checked the migration guide's quoted loader errors against source.

What Changed

• Assembly origins banned on object.value — origin.aggregate/computed/collection/first on a value-hosted field now fail load with ERR_SUBTYPE_RULE_VIOLATION in all five ports, enforced via a shared hoisted subtype-set constant placed above the origin dispatch (cross-port coverage is a property of the constant, not four separate branches); origin.passthrough stays legal (it is FR-015 parameter lineage there). As the additive migration path, @payloadRef/@responseRef now accept a sourceless object.projection, so an assembly-origin payload re-hosts as a projection and keeps its origins.
• source.rdb @role shrinks to primary|replica — index/cache/publish/mirror become reserved-not-registered and the registry allowedValues manifest drops to two members; every port's @role read was already an equality test against primary, so the consumed information was one bit.
• Nested payload @objectRef must target an object.value — enforced by a single fail-closed loader rule rather than five per-port codegen filters; TypeScript, C# and Python previously accepted a non-value target and emitted code from it. Also writes the NUL join delimiter in constraint-merge.ts as an escape, removing the last binary-detected file in the repo.

Risk Assessment

⚠️ Medium: The Round-1 finding (the dbImport/dialect guard keying on object subtype rather than source, which made the new migration-recommended sourceless-projection payload spuriously demand database config) is now fixed by aligning the guard with the #248 source-based predicate the generators already honor, hardened with a positive-case and a new regression test — leaving a coherent, cross-port-consistent, well-tested 5-port BREAKING consolidation with no outstanding source findings, substantial enough that 'low' would understate a coordinated pre-1.0 minor but safe to merge.

Testing

Exercised the change entirely against the TypeScript reference port's source (the only port with a locally-runnable test suite here; the repo runs Java/C#/Python/Kotlin gates on hosted CI only). All targeted tests pass: the new #210 unit test (12 cases), the full TS conformance corpus (532, including all 8 new cross-port error/migration fixtures — each verified a real pass, not a ledger gap), registry-conformance (8, locking the shrunk @ROLE), the codegen payload/vo-only tests (20), the constraint-merge tests after the NUL→escape fix (12), and the modified CLI/codegen/errors tests (15). I additionally drove the loader + payload codegen directly from worktree src to produce reviewer-visible end-to-end evidence: the migration path (a value-hosted assembly origin re-hosted onto a sourceless object.projection) loads with zero errors and emits correct payload TS, while each of the three breaking changes fires its exact error code and each legal counterpart loads clean — including confirmation that origin.passthrough kept its ERR_PASSTHROUGH_TYPE_MISMATCH type-preservation check on values. No failures found. One environmental gotcha resolved mid-testing: scripts run from /tmp resolved @metaobjectsdev/metadata to a stale/non-workspace copy (pre-widen behavior); I re-ran everything against the absolute src paths the tests use. Cross-port loader enforcement is verified via the shared conformance contract here, with each port's own gate owned by remote CI.

Evidence: Migration path end-to-end (load + emitted payload TS)

LOAD errors: (none — migration-path model loads clean) === Emitted payload TS — AuthorReport (sourceless object.projection as @payloadRef) === export interface AuthorReport { name: string; postCount?: number | null; summary?: string | null; } === Emitted payload TS — ReviewRequest (object.value, nested value-only) === export interface ReviewRequest { instructions?: string | null; note?: AuthorNote | null; } export interface AuthorNote { text: string; }

LOAD errors: (none — migration-path model loads clean)

=== Emitted payload TS — AuthorReport (sourceless object.projection as @payloadRef) ===

export interface AuthorReport {
  name: string;
  postCount?: number | null;
  summary?: string | null;
}

=== Emitted payload TS — ReviewRequest (object.value, nested value-only) ===

export interface ReviewRequest {
  instructions?: string | null;
  note?: AuthorNote | null;
}

export interface AuthorNote {
  text: string;
}

Evidence: The three breaking changes + legal carve-outs (against src)

(1) origin.aggregate on value -> [ "ERR_SUBTYPE_RULE_VIOLATION" ] (1) origin.computed on value -> [ "ERR_SUBTYPE_RULE_VIOLATION" ] (1) origin.collection on value -> [ "ERR_SUBTYPE_RULE_VIOLATION" ] (1) origin.first on value -> [ "ERR_SUBTYPE_RULE_VIOLATION" ] ( ) origin.passthrough on value -> [ "(clean — loads)" ] (2) @role: index -> [ "ERR_BAD_ATTR_VALUE" ] ( ) @role: replica (legal) -> [ "(clean — loads)" ] (3) nested @objectRef->entity -> [ "ERR_SUBTYPE_RULE_VIOLATION" ] ( ) nested @objectRef->value -> [ "(clean — loads)" ]

(1) origin.aggregate on value     -> [ "ERR_SUBTYPE_RULE_VIOLATION" ]
(1) origin.computed on value     -> [ "ERR_SUBTYPE_RULE_VIOLATION" ]
(1) origin.collection on value     -> [ "ERR_SUBTYPE_RULE_VIOLATION" ]
(1) origin.first on value     -> [ "ERR_SUBTYPE_RULE_VIOLATION" ]
( ) origin.passthrough on value -> [ "(clean — loads)" ]
(2) @role: index               -> [ "ERR_BAD_ATTR_VALUE" ]
( ) @role: replica (legal)     -> [ "(clean — loads)" ]
(3) nested @objectRef->entity  -> [ "ERR_SUBTYPE_RULE_VIOLATION" ]
( ) nested @objectRef->value    -> [ "(clean — loads)" ]

Evidence: ERR_PASSTHROUGH_TYPE_MISMATCH preserved on values (passthrough carve-out rationale)

passthrough string->int on value (mismatch): [ "ERR_PASSTHROUGH_TYPE_MISMATCH" ] passthrough string->string on value (ok) : [ "(clean)" ]

passthrough string->int on value (mismatch): [ "ERR_PASSTHROUGH_TYPE_MISMATCH" ]
passthrough string->string on value (ok)   : [ "(clean)" ]

Evidence: Migration-path demo source (loads + generates payload codegen from src)

// End-to-end migration-path demo — imports worktree SRC directly (like the tests).
import { MetaDataLoader, InMemoryStringSource } from "/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZF8ZNEVAQQH4VFVHSVSJ20E/server/typescript/packages/metadata/src/index.ts";
import { generatePayloadInterfaces } from "/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZF8ZNEVAQQH4VFVHSVSJ20E/server/typescript/packages/codegen-ts/src/index.ts";

const model = {
  "metadata.root": { package: "demo", children: [
    { "object.entity": { name: "Author", children: [
      { "source.rdb": { "@table": "authors" } },
      { "field.uuid": { name: "id" } },
      { "field.string": { name: "name", "@required": true } },
      { "relationship.aggregation": { name: "posts", "@objectRef": "demo::Post", "@cardinality": "many" } },
      { "identity.primary": { name: "pk", "@fields": ["id"] } },
    ] } },
    { "object.entity": { name: "Post", children: [
      { "source.rdb": { "@table": "posts" } }, { "field.uuid": { name: "id" } },
      { "identity.primary": { name: "pk", "@fields": ["id"] } },
    ] } },
    // MIGRATION: a value that used to host assembly origins -> SOURCELESS object.projection.
    // No source.rdb; the assembly origin (origin.aggregate) re-hosts here legally.
    { "object.projection": { name: "AuthorReport", children: [
      { "field.string": { name: "name", "extends": "demo::Author.name" } },
      { "field.long": { name: "postCount", children: [
        { "origin.aggregate": { "@agg": "count", "@of": "demo::Post.id", "@via": "demo::Author.posts" } } ] } },
      { "field.string": { name: "summary" } },
    ] } },
    // value object that still nests a value (nested targets stay value-only)
    { "object.value": { name: "AuthorNote", children: [
      { "field.string": { name: "text", "@required": true } },
    ] } },
    { "object.value": { name: "ReviewRequest", children: [
      { "field.string": { name: "instructions" } },
      { "field.object": { name: "note", "@objectRef": "demo::AuthorNote" } },
    ] } },
    // @payloadRef -> sourceless projection (the widen) + @responseRef -> value
    { "template.prompt": {
      name: "authorPrompt", "@payloadRef": "AuthorReport",
      "@responseRef": "ReviewRequest", "@textRef": "prompts/author", "@format": "xml",
    } },
  ] },
};
const { root, errors } = await new MetaDataLoader().load([new InMemoryStringSource(JSON.stringify(model))]);
console.log("LOAD errors:", errors.length ? JSON.stringify(errors.map(e=>e.code)) : "(none — migration-path model loads clean)");
if (errors.length) { console.log(JSON.stringify(errors, null, 2)); process.exit(1); }
console.log("\n=== Emitted payload TS — AuthorReport (sourceless object.projection as @payloadRef) ===\n");
console.log(generatePayloadInterfaces(root, "AuthorReport", "demo"));
console.log("=== Emitted payload TS — ReviewRequest (object.value, nested value-only) ===\n");
console.log(generatePayloadInterfaces(root, "ReviewRequest", "demo"));

Evidence: Breaking-changes demo source (three breaks + carve-outs, from src)

// The THREE breaking changes + the passthrough-stays-legal nuance, against worktree src.
import { MetaDataLoader, InMemoryStringSource } from "/home/doug/.no-mistakes/worktrees/4a36a911fd68/01KZF8ZNEVAQQH4VFVHSVSJ20E/server/typescript/packages/metadata/src/index.ts";
async function codes(children) {
  const { errors } = await new MetaDataLoader().load([
    new InMemoryStringSource(JSON.stringify({ "metadata.root": { package: "demo", children } }))]);
  return errors.length ? errors.map(e => e.code) : ["(clean — loads)"];
}
const Ent = (extra=[]) => ({ "object.entity": { name: "Author", children: [
  { "source.rdb": { "@table": "authors" } }, { "field.uuid": { name: "id" } },
  { "field.string": { name: "name" } }, ...extra,
  { "identity.primary": { name: "pk", "@fields": ["id"] } },
] } });
const Note = { "object.value": { name: "Note", children: [{ "field.string": { name: "t" } }] } };

for (const sub of ["aggregate","computed","collection","first"]) {
  const origin = sub==="aggregate" ? { "@agg":"count","@of":"demo::Author.id" }
    : sub==="computed" ? { "@expr": { op:"isNotNull", arg:{field:"name"} } }
    : sub==="first" ? { "@of":"demo::Author.name","@orderBy":["name:desc"] }
    : { "@via":"demo::Author.books" };
  const key = "origin." + sub;
  const fld = sub==="collection"
    ? { "field.object": { name:"x", isArray:true, "@objectRef":"demo::Note", children:[{ [key]: origin }] } }
    : sub==="computed" ? { "field.boolean": { name:"x", children:[{ [key]: origin }] } }
    : sub==="first" ? { "field.string": { name:"x", children:[{ [key]: origin }] } }
    : { "field.int": { name:"x", children:[{ [key]: origin }] } };
  console.log("(1) origin."+sub+" on value     ->", await codes([Ent(), Note, { "object.value": { name:"Bad", children:[fld] } }]));
}
console.log("( ) origin.passthrough on value ->", await codes([Ent(), { "object.value": { name:"Args", children:[
  { "field.string": { name:"n", children:[{ "origin.passthrough": { "@from":"demo::Author.name" } }] } }] } }]));
console.log("(2) @role: index               ->", await codes([Ent([{ "source.rdb": { "@role":"index" } }])]));
console.log("( ) @role: replica (legal)     ->", await codes([{ "object.entity": { name:"E", children:[
  { "source.rdb": { "@table":"e" } }, { "source.rdb": { "@role":"replica", "@kind":"view", "@view":"v" } },
  { "field.uuid": { name:"id" } }, { "identity.primary": { name:"pk","@fields":["id"] } }] } }]));
console.log("(3) nested @objectRef->entity  ->", await codes([Ent(), Note, { "object.value": { name:"Req", children:[
  { "field.object": { name:"a", "@objectRef":"demo::Author" } }] } },
  { "template.prompt": { name:"P","@payloadRef":"Req","@textRef":"x/y","@format":"xml" } }]));
console.log("( ) nested @objectRef->value    ->", await codes([Ent(), Note, { "object.value": { name:"Req", children:[
  { "field.object": { name:"a", "@objectRef":"demo::Note" } }] } },
  { "template.prompt": { name:"P","@payloadRef":"Req","@textRef":"x/y","@format":"xml" } }]));

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅

• ⚠️ server/typescript/packages/codegen-ts/src/runner.ts:160 - The dbImport/dialect guard keys on !e.isAbstract && e.subType !== OBJECT_SUBTYPE_VALUE, but this change makes the sourceless object.projection a first-class, migration-recommended payload shape (the migration guide tells adopters to re-host assembly-origin payloads as sourceless projections, and @payloadRef/@responseRef now accept one). A sourceless projection emits NO database code — queriesFile/routesFile/apiModel all gate emission on hasAnyRdbSource (source-detect.ts, which returns false for a projection with no source.rdb), and the migration guide itself states 'a sourceless projection has no DDL, so meta migrate emits nothing for it.' Yet the guard counts a sourceless projection as 'db-emitting' and hard-throws requiring dialect+dbImport (it runs BEFORE normalizeConfig fills its inert placeholders). Concrete reachable path: a prompt-only model (no sourced entity) whose payload is a sourceless projection — e.g. an origin.computed @expr-only or origin-free self-declared projection that references no entity — fails codegen with 'codegen config is missing dialect and dbImport' for code that is never emitted. The common assembly-origin migration is unaffected (aggregate/collection/first origins reference sourced entities, so the guard trips legitimately anyway); only the computed-only/origin-free-in-a-no-entity-project edge case bites. The principled fix is a one-line alignment with the Persistability is decided by a hardcoded subType === "value" compare — any custom object subtype silently becomes a table #248 R2 invariant the generators already honor — filter on hasAnyRdbSource(e) rather than subType !== OBJECT_SUBTYPE_VALUE — which is exactly the source-based predicate this same branch already applied to the example's own config filter (isPersisted in examples/advanced-modeling/metaobjects.config.ts:37). Flagging ask-user because it governs a user-facing config requirement / when codegen errors; recommend folding the guard alignment into this breaking release since the author already established the correct predicate.

🔧 Fix: fix(runner): gate dbImport/dialect guard on source.rdb, not object subtype
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• cd server/typescript && bun test packages/metadata/test/template-payload-target-210.test.ts (12 pass — all three breaks + migration path + nested-value-only, against src)
• cd server/typescript && bun test packages/metadata/test/conformance.test.ts (532 pass — full TS conformance corpus incl. all 8 new fixtures: error-value-origin-{aggregate,computed,collection,first}, error-source-role-reserved, error-payload-nested-object-ref-entity, error-template-payload-ref-sourced-projection, template-payload-ref-sourceless-projection)
• Per-fixture classification (runFixture + classifyAgainstLedger): all 8 new fixtures classify as real 'pass', none in conformance-expected-failures.json ledger
• cd server/typescript && bun test packages/codegen-ts/test/payload-codegen.test.ts packages/codegen-ts/test/vo-only-config-optional.test.ts packages/metadata/test/origin-collection.test.ts (20 pass — codegen side: sourceless projection needs no DB config + payload emit)
• cd server/typescript && bun test packages/metadata/test/registry-conformance.test.ts (8 pass — shrunk @role allowedValues manifest byte-matches)
• cd server/typescript && bun test packages/metadata/test/constraint-merge.test.ts packages/metadata/test/constraint-validate.test.ts (12 pass — NUL→escape fix exercised)
• cd server/typescript && bun test packages/codegen-ts/test/fr004-verify-demo.test.ts packages/cli/test/integration/verify.test.ts packages/cli/test/unit/payload-field-tree.test.ts packages/metadata/test/errors.test.ts (15 pass — modified test files + error-code ledger)
• Manual end-to-end (worktree src): migration-path model (sourceless projection @payloadRef w/ origin.aggregate + nested-value @responseRef) loads clean and emits payload TS interfaces
• Manual end-to-end (worktree src): each of the three breaks fires its exact code (ERR_SUBTYPE_RULE_VIOLATION / ERR_BAD_ATTR_VALUE) and each legal counterpart loads clean
• Manual check (worktree src): ERR_PASSTHROUGH_TYPE_MISMATCH still fires for a string→int passthrough on a value (carve-out rationale preserved)
• Byte check: tr -cd '\000' on constraint-merge.ts = 0 raw NUL at target (1 at base 726f9debe); delimiter now written as ""

🔧 **Document** - 1 issue found → auto-fixed ✅

• ℹ️ CLAUDE.md:443 - The always-loaded taxonomy summary lists only 'Origin subtypes: passthrough, aggregate', omitting computed/collection/first, which have been registered subtypes since the 0.17.0 line (meta migrate: aggregate/monitoring read-model views can't be modeled as projections — 4 missing origin/field capabilities #195). This is PRE-EXISTING — not made stale by this change, which bans assembly origins on object.value without altering the registered origin-subtype set. Flagging as a follow-up because the line is directly adjacent to this change's subject (which origin subtypes are permitted where); leaving it risks a future session believing only two origin subtypes exist. Out of scope to fix here per scope-discipline (do not opportunistically rewrite unrelated documentation).

🔧 Fix: docs: complete origin subtypes plus #210 host rule
✅ Re-checked - no issues remain.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.