[WIP] sql schema and more

.claude/commands/sync-schema-guides.md

@@ -0,0 +1,86 @@
+---
+description: Evaluate whether a change touches the SQL-schema documentation set (SQLSCHEMA.md + both SQLSCHEMA-GUIDE docs + glossaries) and bring every affected document back in sync. Invoke after ANY change to the SQL schema artifacts, the SQL code generator, the PIM DTOs/serializers, or the Extend derived-property layer — or run it standalone as a consistency audit.
+argument-hint: [optional: short description of what changed, or "audit" for a full consistency check]
+---
+
+# /sync-schema-guides
+
+Keep the SQL-schema documentation set consistent with the code and with itself. The set:
+
+| Document | Role |
+|---|---|
+| `SysML2.NET.CodeGenerator/SQLSCHEMA.md` | Compact reference: artifact table, audit findings, measured numbers, verification recipe, guide section map |
+| `SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.md` | Long-form architectural guide (English) — 19 sections + glossary appendix (§19) |
+| `SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.nl.md` | Dutch translation of the guide — MUST stay structurally identical (same sections, same anchors pattern, same tables) |
+| `SysML2.NET.CodeGenerator/IMPACT-RADIUS.md` | Design sketch for the impact-radius engine (guide obligation §15.1). English-only. Update when propagation kinds, the `derived_dependency` catalog concept, closure indexes, or the §15/§18 obligations it cites change. |
+
+**Language policy for the Dutch guide:** Dutch prose, English conceptual terms (derived
+properties, stored state, fold, checkpoint, overlay, impact radius, census, …). Never
+translate the conceptual vocabulary — it must match the English guide, the schema comments,
+and the code.
+
+## 1. Determine what changed (skip if invoked with an explicit change description)
+
+```
+git status --porcelain
+git diff --stat HEAD
+```
+
+The documentation set is AFFECTED when a change touches any of:
+
+- `SysML2.NET.CodeGenerator/Sql/**` (golden schema, generated schema, smoke test)
+- `SysML2.NET.CodeGenerator/Templates/Uml/core-sql-schema-2.hbs`
+- `SysML2.NET.CodeGenerator/HandleBarHelpers/SqlSchemaHelpers.cs`
+- `SysML2.NET.CodeGenerator/Extensions/SqlSchemaExtensions.cs`
+- `SysML2.NET.CodeGenerator/Generators/UmlHandleBarsGenerators/SQLSchemaGenerator.cs`
+- `SysML2.NET.CodeGenerator/Generators/UmlHandleBarsGenerators/ClassKindRegistry.cs` (and the
+  `ClassKindRegistration` / `ModelVersionRegistration` records) — the append-only registry
+  freezing class_kind ids and model_version ordinals; any append/close here changes the
+  multi-version narrative (guide §6.4/§12.1) and the seed counts
+- `SysML2.NET/PIM/**` and `SysML2.NET.Serializer.Json/PIM/**` (the PIM the schema §3 models)
+- `SysML2.NET/Extend/**` (only when it changes WHICH derived properties exist or their
+  semantics — the guides' census numbers and impact-radius narrative depend on them)
+- `Resources/KerML_only_xmi.uml` / `Resources/SysML_only_xmi.uml` (metamodel change ⇒ census
+  numbers, table counts, catalog counts all suspect)
+
+If nothing relevant changed and the argument is not "audit": report "no sync needed" and stop.
+
+## 2. Sync procedure (English guide is the master)
+
+1. **Update the English guide first**, then mirror into the Dutch guide, then update
+   SQLSCHEMA.md's compact statements and its guide section map. Never let the three diverge.
+2. **Numbers are load-bearing — verify, don't trust.** These recur across all three documents
+   and MUST match reality after any schema/generator change:
+   - smoke-test assertion count (`grep -c "RAISE NOTICE 'PASS" SysML2.NET.CodeGenerator/Sql/schema.smoke.sql`)
+   - subtype-table count (currently 47), link-table count (7), enum count (7), view count (167),
+     class_kind rows (175, ids frozen by ClassKindRegistry), model_version rows (currently 1),
+     reference-validation sources in the two-tier validate functions (currently 42),
+     partitioned-table count × modulus
+   - measured performance numbers: only replace with NEW measurements, never extrapolate
+     silently — label extrapolations as such
+3. **Section integrity:** the guides' section numbers are referenced from SQLSCHEMA.md's
+   section map and from within the guides themselves. When adding a section, prefer appending
+   subsections (x.y) over renumbering; if renumbering is unavoidable, grep all three documents
+   for stale `section N` / `§N` / anchor references and fix the TOCs.
+4. **Glossary appendix (§19 in both guides):** for every new term of art introduced by the
+   change, add a row — shortest clear definition + guide-section reference — to BOTH
+   glossaries, alphabetically placed. When a section is renumbered, re-verify every reference
+   in the *See*/*Zie* columns.
+5. **Schema-file § banners vs guide sections:** the schema files' `§N` banners are a separate
+   numbering space. If a banner is added/renumbered in `schema.golden.sql`, update the
+   template in lockstep (hand-written sections must stay byte-identical) and fix the `(§N)`
+   suffixes in the guides' section headings.
+
+## 3. Verification before reporting done
+
+- The three documents agree on every shared number and section reference.
+- Both guides have identical section structure (`grep -c "^## " both files` — counts match).
+- If the schema itself changed: the generator fixture passes and the smoke test passes
+  against both golden and generated schema (see SQLSCHEMA.md → Verification).
+- Report a short diff summary per document: what was updated and why.
+
+## Standing rule (also enforced via CLAUDE.md)
+
+Any task that edits the affected paths of step 1 is NOT complete until this evaluation has
+run. A change that alters behavior, counts, measured numbers, invariants, or terminology and
+leaves the documentation set untouched is a defect, not a shortcut.

CLAUDE.md

@@ -79,6 +79,32 @@ Generator classes in `SysML2.NET.CodeGenerator/Generators/UmlHandleBarsGenerator
 - Extension methods (Extend) → `SysML2.NET/Extend/`
 - DAL factories → `SysML2.NET.Dal/Core/`
 
+### SQL-schema documentation sync is MANDATORY
+
+The SQL persistence schema carries a documentation set that MUST stay consistent with the
+code and with itself: `SysML2.NET.CodeGenerator/SQLSCHEMA.md` (compact reference),
+`SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.md` (long-form guide, English master),
+`SysML2.NET.CodeGenerator/SQLSCHEMA-GUIDE.nl.md` (Dutch mirror — Dutch prose, English
+conceptual terms, identical section structure) and
+`SysML2.NET.CodeGenerator/IMPACT-RADIUS.md` (design sketch for the impact-radius engine,
+English-only). Both guides end in a glossary appendix (§19) that must list every term of art
+with a short definition and a guide-section reference.
+
+**At the end of EVERY task, evaluate whether the change touched any of:**
+`SysML2.NET.CodeGenerator/Sql/**`, `Templates/Uml/core-sql-schema-2.hbs`,
+`HandleBarHelpers/SqlSchemaHelpers.cs`, `Extensions/SqlSchemaExtensions.cs`,
+`Generators/UmlHandleBarsGenerators/SQLSchemaGenerator.cs`,
+`Generators/UmlHandleBarsGenerators/ClassKindRegistry.cs` (the append-only class-kind /
+model-version registry), `SysML2.NET/PIM/**`,
+`SysML2.NET.Serializer.Json/PIM/**`, `SysML2.NET/Extend/**` (when it changes which derived
+properties exist or their semantics), or the XMI metamodel files. **If yes, run the
+`/sync-schema-guides` skill before reporting the task complete** — it carries the full
+procedure (English-first ordering, load-bearing numbers to re-verify, section/anchor
+integrity, glossary maintenance in BOTH languages). A change that alters behavior, counts,
+measured numbers, invariants, or terminology while leaving the documentation set untouched is
+a defect, not a shortcut. New terms of art introduced anywhere in these documents get a
+glossary row in both guides in the same change.
+
 ### Grounding SysML v2 / KerML work with the Hypha plugin
 
 If the **Hypha** plugin is installed, it is the **preferred grounding source for every SysML v2 / KerML semantic question** — metamodel structure (`hypha:metamodel-lookup`, or the `hypha:metamodel-navigator` agent for cross-cutting fan-out), normative specification intent (`hypha:spec-citation`), and textual-notation validity (`hypha:sysml-validation`). Use it **before** implementing or reviewing anything that depends on the SysML v2 / KerML metamodel — do not rely on a sibling analogue, the doc-comment OCL, or prior knowledge as the source of truth. The metamodel is large and precise; a plausible prior is exactly what produces confident-but-wrong derivations.

SysML2.NET.CodeGenerator.Tests/Generators/UmlHandleBarsGenerators/PostgreSqlSchemaTestHost.cs

@@ -0,0 +1,171 @@
+// -------------------------------------------------------------------------------------------------
+// <copyright file="PostgreSqlSchemaTestHost.cs" company="Starion Group S.A.">
+//
+//   Copyright 2022-2026 Starion Group S.A.
+//
+//   Licensed under the Apache License, Version 2.0 (the "License");
+//   you may not use this file except in compliance with the License.
+//   You may obtain a copy of the License at
+//
+//        http://www.apache.org/licenses/LICENSE-2.0
+//
+//    Unless required by applicable law or agreed to in writing, software
+//    distributed under the License is distributed on an "AS IS" BASIS,
+//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+//    See the License for the specific language governing permissions and
+//    limitations under the License.
+//
+// </copyright>
+// ------------------------------------------------------------------------------------------------
+
+namespace SysML2.NET.CodeGenerator.Tests.Generators.UmlHandleBarsGenerators
+{
+    using System;
+    using System.Collections.Generic;
+    using System.IO;
+    using System.Threading.Tasks;
+
+    using Npgsql;
+
+    using NUnit.Framework;
+
+    using Testcontainers.PostgreSql;
+
+    /// <summary>
+    /// Hosts a disposable PostgreSQL 18 Testcontainer for the SQL-schema integration fixtures,
+    /// configured per the deployment requirement of SysML2.NET.CodeGenerator/SQLSCHEMA.md
+    /// (max_locks_per_transaction=4096 — whole-schema DDL fails on the default of 64). Skips the
+    /// owning fixture with <see cref="Assert.Ignore(string)" /> when Docker is not available.
+    /// </summary>
+    public sealed class PostgreSqlSchemaTestHost : IAsyncDisposable
+    {
+        /// <summary>
+        /// The PostgreSQL Testcontainer, null until <see cref="StartAsync" /> succeeds
+        /// </summary>
+        private PostgreSqlContainer container;
+
+        /// <summary>
+        /// Gets the connection string of the started container
+        /// </summary>
+        public string ConnectionString { get; private set; }
+
+        /// <summary>
+        /// Starts the container, ignoring the owning fixture when Docker is not available
+        /// </summary>
+        /// <returns>
+        /// an awaitable <see cref="Task" />
+        /// </returns>
+        public async Task StartAsync()
+        {
+            try
+            {
+                // the postgres image's entrypoint prepends "postgres" when the first
+                // command argument starts with '-', so this is `docker run postgres:18 -c ...`
+                var builder = new PostgreSqlBuilder("postgres:18")
+                    .WithCommand("-c", "max_locks_per_transaction=4096");
+
+                var dockerEndpoint = ResolveDockerEndpoint();
+
+                if (dockerEndpoint != null)
+                {
+                    builder = builder.WithDockerEndpoint(dockerEndpoint);
+                }
+
+                this.container = builder.Build();
+
+                await this.container.StartAsync();
+            }
+            catch (Exception exception)
+            {
+                Assert.Ignore($"Docker is not available for the SQL-schema integration tests: {exception.Message}");
+            }
+
+            this.ConnectionString = this.container.GetConnectionString();
+        }
+
+        /// <summary>
+        /// Resolves the Docker endpoint when Testcontainers' own probe would miss it: an explicit
+        /// DOCKER_HOST always wins; on Windows, Docker Desktop 4.x exposes the
+        /// dockerDesktopLinuxEngine named pipe instead of the legacy docker_engine one the default
+        /// probe targets. Returns null to let Testcontainers use its own resolution strategies.
+        /// </summary>
+        /// <returns>
+        /// The endpoint to pass to WithDockerEndpoint, or null
+        /// </returns>
+        private static string ResolveDockerEndpoint()
+        {
+            var configuredHost = Environment.GetEnvironmentVariable("DOCKER_HOST");
+
+            if (!string.IsNullOrWhiteSpace(configuredHost))
+            {
+                return configuredHost;
+            }
+
+            if (OperatingSystem.IsWindows() && File.Exists(@"\\.\pipe\dockerDesktopLinuxEngine"))
+            {
+                return "npipe://./pipe/dockerDesktopLinuxEngine";
+            }
+
+            return null;
+        }
+
+        /// <summary>
+        /// Executes a multi-statement SQL script (dollar-quoted DO/CREATE FUNCTION bodies included)
+        /// as a single command
+        /// </summary>
+        /// <param name="sql">
+        /// The script text
+        /// </param>
+        /// <returns>
+        /// an awaitable <see cref="Task" />
+        /// </returns>
+        public async Task ExecuteScriptAsync(string sql)
+        {
+            await using var connection = new NpgsqlConnection(this.ConnectionString);
+            await connection.OpenAsync();
+
+            await using var command = new NpgsqlCommand(sql, connection);
+            command.CommandTimeout = 600;
+            await command.ExecuteNonQueryAsync();
+        }
+
+        /// <summary>
+        /// Executes a multi-statement SQL script and collects the RAISE NOTICE messages it emits —
+        /// the PASS/FAIL channel of schema.smoke.sql and schema.concurrency.verify.sql
+        /// </summary>
+        /// <param name="sql">
+        /// The script text
+        /// </param>
+        /// <returns>
+        /// an awaitable <see cref="Task" /> carrying the notice messages in emission order
+        /// </returns>
+        public async Task<IReadOnlyList<string>> ExecuteScriptCollectingNoticesAsync(string sql)
+        {
+            var notices = new List<string>();
+
+            await using var connection = new NpgsqlConnection(this.ConnectionString);
+            connection.Notice += (_, noticeArguments) => notices.Add(noticeArguments.Notice.MessageText);
+            await connection.OpenAsync();
+
+            await using var command = new NpgsqlCommand(sql, connection);
+            command.CommandTimeout = 600;
+            await command.ExecuteNonQueryAsync();
+
+            return notices;
+        }
+
+        /// <summary>
+        /// Disposes the container
+        /// </summary>
+        /// <returns>
+        /// an awaitable <see cref="ValueTask" />
+        /// </returns>
+        public async ValueTask DisposeAsync()
+        {
+            if (this.container != null)
+            {
+                await this.container.DisposeAsync();
+            }
+        }
+    }
+}

SysML2.NET.CodeGenerator.Tests/Generators/UmlHandleBarsGenerators/SQLSchemaGeneratorTestFixture.cs

@@ -0,0 +1,77 @@
+// -------------------------------------------------------------------------------------------------
+// <copyright file="SQLSchemaGeneratorTestFixture.cs" company="Starion Group S.A.">
+//
+//   Copyright 2022-2026 Starion Group S.A.
+//
+//   Licensed under the Apache License, Version 2.0 (the "License");
+//   you may not use this file except in compliance with the License.
+//   You may obtain a copy of the License at
+//
+//        http://www.apache.org/licenses/LICENSE-2.0
+//
+//    Unless required by applicable law or agreed to in writing, software
+//    distributed under the License is distributed on an "AS IS" BASIS,
+//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+//    See the License for the specific language governing permissions and
+//    limitations under the License.
+//
+// </copyright>
+// ------------------------------------------------------------------------------------------------
+
+namespace SysML2.NET.CodeGenerator.Tests.Generators.UmlHandleBarsGenerators
+{
+    using System.IO;
+    using System.Threading.Tasks;
+
+    using NUnit.Framework;
+
+    using SysML2.NET.CodeGenerator.Generators.UmlHandleBarsGenerators;
+
+    [TestFixture]
+    public class SQLSchemaGeneratorTestFixture
+    {
+        private DirectoryInfo sqlSchemaDirectoryInfo;
+        private SQLSchemaGenerator sqlSchemaGenerator;
+
+        [OneTimeSetUp]
+        public void OneTimeSetUp()
+        {
+            var directoryInfo = new DirectoryInfo(TestContext.CurrentContext.TestDirectory);
+
+            var path = Path.Combine("UML", "_SysML2.NET.Sql");
+
+            this.sqlSchemaDirectoryInfo = directoryInfo.CreateSubdirectory(path);
+
+            this.sqlSchemaGenerator = new SQLSchemaGenerator();
+        }
+
+        [Test]
+        public async Task Verify_that_sql_schema_is_generated()
+        {
+            var generatedSchema = await this.sqlSchemaGenerator.GenerateSqlSchemaAsync(GeneratorSetupFixture.XmiReaderResult, this.sqlSchemaDirectoryInfo);
+
+            Assert.That(generatedSchema, Is.Not.Null.And.Not.Empty);
+
+            Assert.That(generatedSchema, Does.Contain("CREATE TABLE sysml2.element_version"));
+            Assert.That(generatedSchema, Does.Contain("CREATE TABLE sysml2.derived_version"));
+            Assert.That(generatedSchema, Does.Contain("CREATE TYPE sysml2.visibility_kind AS ENUM ('private', 'protected', 'public');"));
+            Assert.That(generatedSchema, Does.Contain("CREATE TABLE sysml2.feature_v"));
+            Assert.That(generatedSchema, Does.Contain("CREATE TABLE sysml2.element_owned_relationship"));
+            Assert.That(generatedSchema, Does.Contain("CREATE VIEW sysml2.vw_part_usage"));
+            Assert.That(generatedSchema, Does.Contain("CREATE TABLE sysml2.model_version"));
+            Assert.That(generatedSchema, Does.Contain("INSERT INTO sysml2.model_version (id, name, source_fingerprint) VALUES"));
+            Assert.That(generatedSchema, Does.Contain("INSERT INTO sysml2.class_kind (id, name, is_abstract, introduced_in, removed_in) VALUES"));
+            Assert.That(generatedSchema, Does.Contain("ON CONFLICT (id) DO NOTHING;"), "the registry seeds must be idempotent");
+            Assert.That(generatedSchema, Does.Contain("(120, 'PartUsage', false, 1, NULL)"), "class_kind ids must come from the frozen registry");
+            Assert.That(generatedSchema, Does.Contain("model_version_id smallint   NOT NULL REFERENCES sysml2.model_version (id)"));
+            Assert.That(generatedSchema, Does.Contain("FOREIGN KEY (identity_id, class_kind) REFERENCES sysml2.data_identity (id, class_kind)"), "the typed-identity composite FK must anchor every version");
+            Assert.That(generatedSchema, Does.Contain("CREATE OR REPLACE FUNCTION sysml2.validate_references_at_commit("));
+            Assert.That(generatedSchema, Does.Contain("CREATE OR REPLACE FUNCTION sysml2.validate_references_in_commit("));
+            Assert.That(generatedSchema, Does.Contain("ANALYZE validation_snapshot;"), "the full pass must feed the planner true snapshot cardinality");
+            Assert.That(generatedSchema, Does.Contain("'wrong-type'"), "the reference validation must type-check via the typed identity");
+            Assert.That(generatedSchema, Does.Not.Contain("CREATE TABLE sysml2.property_catalog"), "the property->storage routing lives in generated C#, not in the database");
+            Assert.That(generatedSchema, Does.Not.Contain("CREATE TABLE sysml2.class_kind_table"), "subtype-table participation lives in generated C#, not in the database");
+            Assert.That(generatedSchema, Does.Not.Contain("{{"), "no unresolved handlebars expressions may survive generation");
+        }
+    }
+}