From d44465f44d246aac12a9da2068a18d7b39c27a1e Mon Sep 17 00:00:00 2001 From: sonika-shah <58761340+sonika-shah@users.noreply.github.com> Date: Fri, 18 Sep 2026 04:52:24 +0000 Subject: [PATCH] fix(glossary): include cardinality on default term relation types (#28376) * fix(glossary): include cardinality on default term relation types The system-seeded glossary term relation types (relatedTo, synonym, broader, narrower, partOf, hasPart, etc.) were created without a cardinality value, so the GET /v1/system/settings/glossaryTermRelationSettings response omitted the field for defaults while returning it for custom relations. The UI fell back to MANY_TO_MANY for every default and, on the next PUT, wrote that fallback back into the system relations - overriding any intended value. Set the expected cardinality on each default in SettingsCache and add a 2.0.2 data migration that backfills the field for installs that were already seeded without it. broader/narrower follow strict SKOS semantics (ONE_TO_MANY / MANY_TO_ONE - a term has at most one broader parent); all other system relations stay MANY_TO_MANY. * chore(migration): move glossary cardinality backfill to 2.0.3 The 2.0.2 / v202 migration slot was taken by search-settings repairs on main. Relocate the glossary term relation cardinality backfill to the next free version (2.0.3 / v203) so it runs after the existing 2.0.2 migrations without colliding with them. * fix(migration): cast settings json to jsonb on postgres in v203 backfill The glossary cardinality backfill bound the settings payload as a string into openmetadata_settings.json, which is jsonb on Postgres - Postgres refuses the implicit varchar->jsonb cast and the whole migration (and therefore TestSuiteBootstrap / every IT in the lane) aborted. Select the DB-specific UPDATE the way PiiRecognizerMigrationUtil / v1126 do: plain :json for MySQL, :json::jsonb for Postgres. * fix(glossary): seed system relation cardinality as unbounded MANY_TO_MANY Resolves the enum/bounds inversion flagged in review by not imposing any per-relation cardinality on the system defaults - the original bug was only that GET returned a null cardinality, not that hierarchy needed enforcing. - SettingsCache: broader/narrower now MANY_TO_MANY with null bounds, matching every other system default. GET returns a non-null cardinality and the UI no longer falls back to (and persists) a guessed value - with zero new enforcement, so existing terms with multiple broader parents keep working. - v203 migration: backfill null cardinality on system-defined relations to MANY_TO_MANY and derive bounds via the canonical normalize(), deleting the hand-written (inverted) bounds switch and per-relation map. - IT: assert exact cardinality (MANY_TO_MANY) and null sourceMax/targetMax for all 10 system relations, so an inverted or bounded value fails the test. (cherry picked from commit 4c33d0a228011ac34c5f07f901cace05a333cb89) --- .../mysql/postDataMigrationSQLScript.sql | 2 + .../native/2.0.3/mysql/schemaChanges.sql | 2 + .../postgres/postDataMigrationSQLScript.sql | 2 + .../native/2.0.3/postgres/schemaChanges.sql | 2 + .../tests/GlossaryTermRelationSettingsIT.java | 58 +++++++++++++ .../migration/mysql/v203/Migration.java | 20 +++++ .../migration/postgres/v203/Migration.java | 20 +++++ .../migration/utils/v203/MigrationUtil.java | 84 +++++++++++++++++++ .../resources/settings/SettingsCache.java | 13 +++ 9 files changed, 203 insertions(+) create mode 100644 bootstrap/sql/migrations/native/2.0.3/mysql/postDataMigrationSQLScript.sql create mode 100644 bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql create mode 100644 bootstrap/sql/migrations/native/2.0.3/postgres/postDataMigrationSQLScript.sql create mode 100644 bootstrap/sql/migrations/native/2.0.3/postgres/schemaChanges.sql create mode 100644 openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v203/Migration.java create mode 100644 openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v203/Migration.java create mode 100644 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v203/MigrationUtil.java diff --git a/bootstrap/sql/migrations/native/2.0.3/mysql/postDataMigrationSQLScript.sql b/bootstrap/sql/migrations/native/2.0.3/mysql/postDataMigrationSQLScript.sql new file mode 100644 index 000000000000..80730edbcbb4 --- /dev/null +++ b/bootstrap/sql/migrations/native/2.0.3/mysql/postDataMigrationSQLScript.sql @@ -0,0 +1,2 @@ +-- Post data migration script for OpenMetadata 2.0.3 +-- Backfill for glossaryTermRelationSettings cardinality runs in the Java migration step. diff --git a/bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql b/bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql new file mode 100644 index 000000000000..f7a1ce546a77 --- /dev/null +++ b/bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql @@ -0,0 +1,2 @@ +-- Schema changes for OpenMetadata 2.0.3 +-- No DDL changes; data backfill runs in the Java migration step. diff --git a/bootstrap/sql/migrations/native/2.0.3/postgres/postDataMigrationSQLScript.sql b/bootstrap/sql/migrations/native/2.0.3/postgres/postDataMigrationSQLScript.sql new file mode 100644 index 000000000000..80730edbcbb4 --- /dev/null +++ b/bootstrap/sql/migrations/native/2.0.3/postgres/postDataMigrationSQLScript.sql @@ -0,0 +1,2 @@ +-- Post data migration script for OpenMetadata 2.0.3 +-- Backfill for glossaryTermRelationSettings cardinality runs in the Java migration step. diff --git a/bootstrap/sql/migrations/native/2.0.3/postgres/schemaChanges.sql b/bootstrap/sql/migrations/native/2.0.3/postgres/schemaChanges.sql new file mode 100644 index 000000000000..f7a1ce546a77 --- /dev/null +++ b/bootstrap/sql/migrations/native/2.0.3/postgres/schemaChanges.sql @@ -0,0 +1,2 @@ +-- Schema changes for OpenMetadata 2.0.3 +-- No DDL changes; data backfill runs in the Java migration step. diff --git a/openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/GlossaryTermRelationSettingsIT.java b/openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/GlossaryTermRelationSettingsIT.java index b6729d702992..5749efddddf2 100644 --- a/openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/GlossaryTermRelationSettingsIT.java +++ b/openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/GlossaryTermRelationSettingsIT.java @@ -107,6 +107,9 @@ void test_glossaryTermRelationSettingsExist() throws Exception { assertNotNull(relationType.get("category"), "category should exist for " + name); assertNotNull(relationType.get("isSymmetric"), "isSymmetric should exist for " + name); assertNotNull(relationType.get("isTransitive"), "isTransitive should exist for " + name); + JsonNode cardinalityNode = relationType.get("cardinality"); + assertNotNull(cardinalityNode, "cardinality should exist for " + name); + assertFalse(cardinalityNode.isNull(), "cardinality should not be null for " + name); } assertTrue(hasRelatedTo, "Should have 'relatedTo' relation type"); @@ -114,6 +117,61 @@ void test_glossaryTermRelationSettingsExist() throws Exception { assertTrue(hasBroader, "Should have 'broader' relation type"); } + @Test + @ResourceLock( + value = SharedResourceLocks.GLOSSARY_TERM_RELATION_SETTINGS, + mode = ResourceAccessMode.READ) + void test_systemRelationTypesAreUnboundedManyToMany() throws Exception { + JsonNode settings = getSettings(); + JsonNode relationTypes = settings.get("config_value").get("relationTypes"); + + Set systemNames = + Set.of( + "relatedTo", + "synonym", + "antonym", + "broader", + "narrower", + "partOf", + "hasPart", + "calculatedFrom", + "usedToCalculate", + "seeAlso"); + + Set verified = new HashSet<>(); + for (JsonNode type : relationTypes) { + String name = type.get("name").asText(); + if (!systemNames.contains(name)) { + continue; + } + verified.add(name); + + JsonNode cardinality = type.get("cardinality"); + assertNotNull(cardinality, "cardinality should exist for " + name); + assertFalse(cardinality.isNull(), "cardinality should not be null for " + name); + assertEquals( + "MANY_TO_MANY", + cardinality.asText(), + "System relation '" + name + "' must be MANY_TO_MANY (unbounded, no enforcement)"); + + // MANY_TO_MANY carries no bounds - sourceMax/targetMax must stay null so no cardinality + // cap is enforced on terms that already have many of these relations. + JsonNode sourceMax = type.get("sourceMax"); + JsonNode targetMax = type.get("targetMax"); + assertTrue( + sourceMax == null || sourceMax.isNull(), + "System relation '" + name + "' must have null sourceMax, got: " + sourceMax); + assertTrue( + targetMax == null || targetMax.isNull(), + "System relation '" + name + "' must have null targetMax, got: " + targetMax); + } + + Set missing = new HashSet<>(systemNames); + missing.removeAll(verified); + assertTrue( + missing.isEmpty(), "All 10 system relation types must be present. Missing: " + missing); + } + @Test @ResourceLock( value = SharedResourceLocks.GLOSSARY_TERM_RELATION_SETTINGS, diff --git a/openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v203/Migration.java b/openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v203/Migration.java new file mode 100644 index 000000000000..c5d5d48a742f --- /dev/null +++ b/openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v203/Migration.java @@ -0,0 +1,20 @@ +package org.openmetadata.service.migration.mysql.v203; + +import lombok.SneakyThrows; +import org.openmetadata.service.migration.api.MigrationProcessImpl; +import org.openmetadata.service.migration.utils.MigrationFile; +import org.openmetadata.service.migration.utils.v203.MigrationUtil; + +public class Migration extends MigrationProcessImpl { + + public Migration(MigrationFile migrationFile) { + super(migrationFile); + } + + @Override + @SneakyThrows + public void runDataMigration() { + MigrationUtil migrationUtil = new MigrationUtil(handle); + migrationUtil.backfillGlossaryTermRelationCardinality(); + } +} diff --git a/openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v203/Migration.java b/openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v203/Migration.java new file mode 100644 index 000000000000..151d15d23ca2 --- /dev/null +++ b/openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v203/Migration.java @@ -0,0 +1,20 @@ +package org.openmetadata.service.migration.postgres.v203; + +import lombok.SneakyThrows; +import org.openmetadata.service.migration.api.MigrationProcessImpl; +import org.openmetadata.service.migration.utils.MigrationFile; +import org.openmetadata.service.migration.utils.v203.MigrationUtil; + +public class Migration extends MigrationProcessImpl { + + public Migration(MigrationFile migrationFile) { + super(migrationFile); + } + + @Override + @SneakyThrows + public void runDataMigration() { + MigrationUtil migrationUtil = new MigrationUtil(handle); + migrationUtil.backfillGlossaryTermRelationCardinality(); + } +} diff --git a/openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v203/MigrationUtil.java b/openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v203/MigrationUtil.java new file mode 100644 index 000000000000..c2be53aa4235 --- /dev/null +++ b/openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v203/MigrationUtil.java @@ -0,0 +1,84 @@ +package org.openmetadata.service.migration.utils.v203; + +import lombok.extern.slf4j.Slf4j; +import org.jdbi.v3.core.Handle; +import org.openmetadata.schema.configuration.GlossaryTermRelationSettings; +import org.openmetadata.schema.configuration.GlossaryTermRelationType; +import org.openmetadata.schema.configuration.RelationCardinality; +import org.openmetadata.schema.utils.JsonUtils; +import org.openmetadata.service.resources.databases.DatasourceConfig; +import org.openmetadata.service.util.GlossaryTermRelationSettingsUtil; + +@Slf4j +public class MigrationUtil { + + private static final String GLOSSARY_TERM_RELATION_SETTINGS = "glossaryTermRelationSettings"; + + // Postgres stores the settings column as jsonb and won't implicitly cast a bound string + // (::jsonb is required); MySQL's JSON column parses the string directly. + private static final String UPDATE_MYSQL = + "UPDATE openmetadata_settings SET json = :json WHERE configType = :configType"; + private static final String UPDATE_POSTGRES = + "UPDATE openmetadata_settings SET json = :json::jsonb WHERE configType = :configType"; + + private final Handle handle; + + public MigrationUtil(Handle handle) { + this.handle = handle; + } + + /** + * Older installs seeded the system glossary relation types before the cardinality field existed, + * so GET returned {@code cardinality: null} and the UI fell back to MANY_TO_MANY. Persist that + * same MANY_TO_MANY on the system defaults whose cardinality is still null. Bounds are derived by + * the canonical normalizer (the settings PUT path), leaving these relations unbounded - metadata + * only, no new enforcement. + */ + public void backfillGlossaryTermRelationCardinality() { + GlossaryTermRelationSettings settings = loadSettings(); + if (settings == null || settings.getRelationTypes() == null) { + return; + } + + boolean changed = false; + for (GlossaryTermRelationType relationType : settings.getRelationTypes()) { + if (relationType != null + && Boolean.TRUE.equals(relationType.getIsSystemDefined()) + && relationType.getCardinality() == null) { + relationType.setCardinality(RelationCardinality.MANY_TO_MANY); + GlossaryTermRelationSettingsUtil.normalize(relationType); + changed = true; + LOG.info( + "Backfilled MANY_TO_MANY cardinality on system relation '{}'", relationType.getName()); + } + } + + if (changed) { + persist(settings); + } + } + + private GlossaryTermRelationSettings loadSettings() { + String json = + handle + .createQuery("SELECT json FROM openmetadata_settings WHERE configType = :configType") + .bind("configType", GLOSSARY_TERM_RELATION_SETTINGS) + .mapTo(String.class) + .findOne() + .orElse(null); + if (json == null) { + LOG.info("No glossaryTermRelationSettings row found; skipping cardinality backfill"); + return null; + } + return JsonUtils.readValue(json, GlossaryTermRelationSettings.class); + } + + private void persist(GlossaryTermRelationSettings settings) { + boolean isMySQL = Boolean.TRUE.equals(DatasourceConfig.getInstance().isMySQL()); + handle + .createUpdate(isMySQL ? UPDATE_MYSQL : UPDATE_POSTGRES) + .bind("configType", GLOSSARY_TERM_RELATION_SETTINGS) + .bind("json", JsonUtils.pojoToJson(settings)) + .execute(); + } +} diff --git a/openmetadata-service/src/main/java/org/openmetadata/service/resources/settings/SettingsCache.java b/openmetadata-service/src/main/java/org/openmetadata/service/resources/settings/SettingsCache.java index 8cd23ada1227..93c93515f408 100644 --- a/openmetadata-service/src/main/java/org/openmetadata/service/resources/settings/SettingsCache.java +++ b/openmetadata-service/src/main/java/org/openmetadata/service/resources/settings/SettingsCache.java @@ -63,6 +63,7 @@ import org.openmetadata.schema.configuration.GlossaryTermRelationType; import org.openmetadata.schema.configuration.HistoryCleanUpConfiguration; import org.openmetadata.schema.configuration.OpenLineageSettings; +import org.openmetadata.schema.configuration.RelationCardinality; import org.openmetadata.schema.configuration.RelationCategory; import org.openmetadata.schema.configuration.WorkflowSettings; import org.openmetadata.schema.email.SmtpSettings; @@ -370,6 +371,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.ASSOCIATIVE, true, "#1570ef", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -383,6 +385,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.EQUIVALENCE, true, "#b42318", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -396,6 +399,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.ASSOCIATIVE, true, "#b54708", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -409,6 +413,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.HIERARCHICAL, true, "#067647", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -422,6 +427,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.HIERARCHICAL, true, "#4e5ba6", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -435,6 +441,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.HIERARCHICAL, true, "#026aa2", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -448,6 +455,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.HIERARCHICAL, true, "#155eef", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -461,6 +469,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.ASSOCIATIVE, true, "#6938ef", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -474,6 +483,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.ASSOCIATIVE, true, "#ba24d5", + RelationCardinality.MANY_TO_MANY, null, null), createRelationType( @@ -487,6 +497,7 @@ private static void createDefaultConfiguration(OpenMetadataApplicationConfig app RelationCategory.ASSOCIATIVE, true, "#c11574", + RelationCardinality.MANY_TO_MANY, null, null)); @@ -532,6 +543,7 @@ private static GlossaryTermRelationType createRelationType( RelationCategory category, boolean isSystemDefined, String color, + RelationCardinality cardinality, Integer sourceMax, Integer targetMax) { return new GlossaryTermRelationType() @@ -545,6 +557,7 @@ private static GlossaryTermRelationType createRelationType( .withCategory(category) .withIsSystemDefined(isSystemDefined) .withColor(color) + .withCardinality(cardinality) .withSourceMax(sourceMax) .withTargetMax(targetMax); }