Skip to content

fix(gen-shacl): correct rule-converter path parity, combined bounds, and SPARQL escaping - #21

Closed
jdsika wants to merge 4 commits into
feat/shaclgen-compositional-rule-fallbackfrom
fix/shaclgen-rule-converter-hardening
Closed

fix(gen-shacl): correct rule-converter path parity, combined bounds, and SPARQL escaping#21
jdsika wants to merge 4 commits into
feat/shaclgen-compositional-rule-fallbackfrom
fix/shaclgen-rule-converter-hardening

Conversation

@jdsika

@jdsika jdsika commented Jul 10, 2026

Copy link
Copy Markdown

Summary

Review-hardening pass over the rules → SHACL-SPARQL converters. Three latent
defects, each with regression tests that fail before the fix (re-verified
on this stack: running the new tests against the pre-fix generator source
yields 5 failures):

  1. Induced-slot parity (silent false negative). _slot_uri and
    _resolve_enum_value_ref resolved the base slot, so a slot_usage
    override of slot_uri (or a narrowed enum range) made the SPARQL body
    query a property / enum IRI the data never uses, while sh:path used the
    induced IRI. The constraint then silently never fired. Both now resolve
    the induced slot for the class, matching the sh:path logic in the main
    slot loop. Affected code originates in feat(gen-shacl): generate sh:sparql constraints from LinkML rules #11 (framework) — this fix must be
    folded into the framework commit when submitting upstream.

  2. Combined operators (silent under-constraint / false positives). A
    single precondition or member condition dispatched on the first matching
    operator, so a bounded range {minimum_value: X, maximum_value: Y} dropped
    the lower bound. A shared _scalar_filters helper now emits every
    recognised operator (and still returns None — skip, never mis-translate —
    when none is recognised). Affected code originates in the compositional
    fallback (base PR of this one).

  3. SPARQL string escaping (invalid / injectable output). An
    equals_string or permissible-value name containing a ", \, or newline
    produced unparseable SPARQL. New _sparql_string_literal escapes per
    SPARQL 1.1 §19.7.
    Affects every converter that renders string terms (framework,
    presence-implies-value, fallback).

Stack position

Base branch: feat/shaclgen-compositional-rule-fallback — only the hardening
delta shows in the diff.

main
└─ feat/shaclgen-rules-sparql                      #11 (green)
   └─ feat/shaclgen-presence-implies-value-stacked   (PR: presence-implies-value)
      └─ feat/shaclgen-compositional-rule-fallback   (PR: fallback M1–M5)
         └─ fix/shaclgen-rule-converter-hardening   ← this PR

Kept as a separate commit (rather than folded into the feature branches) so
the defects, their failure modes, and the regression tests remain visible to
review. When upstreaming to linkml/linkml, fold each fix into the submission
that introduces the affected code, as noted per defect above.

Provenance

Cherry-pick of e263def7 from feat/shacl-rule-converters (#18), where the
full CI matrix is green.

How was this tested?

  • tests/linkml/test_generators/test_shaclgen.py on this branch:
    106 passed (base suites + 6 regression tests: structural, prepareQuery
    syntax, and pyshacl end-to-end — including the below-threshold case proving
    the lower bound is enforced, and an end-to-end proof that a
    slot_usage-overridden constraint actually fires).
  • Negative control: with this branch's tests but the pre-fix generator
    source, the regression tests fail (5 failures) — the tests bite.
  • ruff check and ruff format --check clean on both changed files.

Areas of uncertainty

Checklist

  • My code follows the contributor guidelines
  • I have added tests that prove my fix/feature works
  • Existing tests pass locally with my changes

AI Assistance

If you used AI tools while preparing this PR, you are still the author and responsible for understanding, verifying, and defending your submission. Please engage with reviewers personally rather than through your agent during feedback and revisions. See our AI Covenant for details.

Implement SHACL-SPARQL constraint generation for the boolean-guard
pattern commonly used in conditional validation rules. When a LinkML
class has rules: blocks with preconditions (value_presence: PRESENT)
and postconditions (equals_string: true), the generator now emits
sh:SPARQLConstraint nodes on the corresponding sh:NodeShape.

Features:
- New _add_rules() method translates recognised rule patterns to SPARQL
- Boolean-guard pattern: if value present then flag must be true
- Rule description mapped to sh:message on the constraint
- Deactivated rules are skipped
- Warnings emitted for bidirectional/open_world rule flags
- New --emit-rules/--no-emit-rules CLI flag (default: enabled)
- Full URI references in SPARQL (no PREFIX declarations needed)

The generated SPARQL follows W3C SHACL Section 5 and uses the pre-bound
\ variable per Section 5.3.1. Constraints are validated by pyshacl
with advanced=True.

Refs: linkml#2464
Signed-off-by: Carlo van Driesten <carlo.van-driesten@bmw.de>
@jdsika

jdsika commented Jul 10, 2026

Copy link
Copy Markdown
Author

Adversarial audit of the hardening fixes (probed at this tip)

Fix claims that held under attack: top-level induced-slot parity (sh:path and SPARQL body byte-agree for slot_usage-overridden slot_uri, incl. attributes — class_induced_slots never mangles names, so the element_by_schema_map membership test matches the main slot loop); combined {minimum_value, maximum_value} emits both bounds; quote/backslash/newline escaping produces parseable queries (\b/\f are legal raw in SPARQL STRING_LITERAL2, so leaving them unescaped is fine).

C1 — real bug: the parity fix stops one hop short, and the new class_slots check regresses the name-collision case.
Nested conditions (range_expression, has_member members) still resolve the inner slot via _slot_uri(sv, inner_name, cls) with the outer class:

  • Without collision: a child class narrowing an inner slot's slot_uri via its own slot_usage gets sh:path ≠ SPARQL body (the claimed defect class, one hop deeper).
  • With collision (regression): when the inner slot name also exists on the outer class with a different slot_usage URI, the new if slot_name in sv.class_slots(cls.name): induced_slot(...) makes the outer induced slot win. The pre-fix code (plain get_slot) emitted the correct child URI in this case; the post-fix code emits the parent's. For has_member the wrong predicate makes FILTER NOT EXISTS vacuously true → false positives on conforming data (demonstrated with pyshacl).
    Same family: _resolve_member_enum_ref uses non-induced sv.get_slot(container_slot_name), so container-slot range narrowing resolves the wrong enum. See also the matching finding on the base PR (fallback), where the fix belongs.

C2 — edge case: _scalar_filters drops value_presence: ABSENT when combined with another operator.
Pure ABSENT preconditions are correctly rejected (verified — no inverted-semantics constraint is emitted). But {value_presence: ABSENT, minimum_value: 5} translates as if the slot must be present with the bound — the opposite of the declared trigger. Only reachable via a self-contradictory condition, but it breaches this PR's own "skip, never mis-translate" contract. Fix: return None when value_presence is set to anything other than PRESENT (same hole in _member_conditions inner conditions).

C3 — real bug (pre-existing; this PR's new docstring mis-states it): _sparql_number passes non-numeric values raw.
The docstring added here claims LinkML parses minimum_value/maximum_value as int/float — the metamodel range is Anything. A YAML date bound 2020-01-01 emits FILTER ( ?pre0 >= 2020-01-01 ), which parses as the arithmetic expression 2020−01−01 = 2018 (silent false negative); "abc"/.nan produce unparseable sh:select that makes pyshacl raise on the entire shapes graph. Gate on real numerics and skip otherwise.

C4 — cosmetic: alias-form rule keys silently diverge. A rule key written my_slot for a slot named my slot misses class_slots/get_slot and falls back to default_prefix:my_slot while sh:path uses the overridden URI — a vacuous constraint with no warning. linkml-runtime's induced_slot normalizes aliases via slot_name_mappings(); here it diverges silently instead of skipping/warning.

Also verified: malformed meaning CURIEs fail loudly at generation (no injection path); remaining interpolation sites (rule description via rdflib.Literal, int-cast cardinalities, get_uri/expand_curie IRIs) are safe.

Suggest a follow-up commit on this stack for C1–C3 before upstreaming (C1's fix best lands where the nested emitters live, i.e. the fallback commit, when folding for upstream submission).

@jdsika

jdsika commented Jul 11, 2026

Copy link
Copy Markdown
Author

The substantive audit findings above are resolved in #22 (fix/shaclgen-rule-converter-audit-findings, stacked on this series) — one commit, 19 regression tests, 18 of which fail on the pre-fix source. See the finding→fix table in the #22 description.

rmessaou and others added 3 commits July 11, 2026 12:35
Generalise the boolean-guard SHACL-SPARQL pattern to enum-valued targets.
A rule whose precondition is `value_presence: PRESENT` on a value slot and
whose postcondition is `equals_string` / `equals_string_in` on a target
slot now emits an `sh:sparql` constraint requiring the target slot to be
present and hold one of the allowed values. Each allowed value resolves to
its enum `meaning` IRI, with a string-literal fallback.

Motivating case (aiSim environment):
- "if texture_sky_color is set, sky_model must be TextureSky"
- "if overcast_sky_illuminance is set, sky_model must be OvercastSky or
  MeasuredOvercastSky"

The existing boolean-guard (`equals_string: "true"`) and exclusive-value
patterns are unchanged; boolean guard keeps priority over the new branch.

Adds focused unit tests (enum IRI vs. literal fallback, single value vs.
set membership, message emission, SPARQL syntax) plus pyshacl end-to-end
validation for the new pattern.

(cherry picked from commit ea5cf57)
Add a compositional fallback in _rule_to_sparql for rule-operator
combinations outside the three named patterns (boolean guard,
presence-implies-value, exclusive value). Tried only after the named
patterns, so their output is unchanged. The fallback translates a
conjunction of precondition slot conditions plus a single postcondition
into one SELECT $this violation query; any unsupported operator makes
the converter return None -- skip, never mis-translate.

Supported combinations:

- M1 conditional-required: equals_string / value_presence: PRESENT
  precondition + required: true postcondition (violation = FILTER NOT
  EXISTS on the target slot).
- M2 conditional-absent: value_presence: ABSENT postcondition
  (violation = the forbidden slot is present).
- M3 numeric threshold preconditions: minimum_value / maximum_value
  inclusive bounds on the trigger slot, rendered via _sparql_number.
- M4 nested precondition: one hop into an inlined child object via
  range_expression.slot_conditions (e.g. sun_position.elevation <= 0);
  adds _member_conditions (shared inner-condition emitter) and
  _resolve_member_enum_ref, which resolves inner enum values against
  the container slot's range class (handles slot_usage-specialised
  enums).
- M5 has_member list-membership: a multivalued slot must contain a
  member matching a nested range_expression (violation = FILTER NOT
  EXISTS over the members); reuses _member_conditions.

Tests per converter: structural triple assertions, prepareQuery syntax
validation, and pyshacl end-to-end (conforming instances pass, crafted
violations fail) with advanced=True.

Squashed from the five M1-M5 commits on feat/shacl-rule-converters
(7566e30, fac681e, f8ea709, 25f8cd2, 4776977).

Signed-off-by: Carlo van Driesten <carlo.van-driesten@vdl.digital>
…and SPARQL escaping

Review hardening for the SHACL-SPARQL rule converters. Three defects, each
with a regression test that fails before this change:

- Induced-slot parity: _slot_uri and _resolve_enum_value_ref resolved the
  *base* slot, so a slot_usage override of slot_uri (or a narrowed enum range)
  made the generated SPARQL query a property/enum the data never uses while
  sh:path used the induced IRI. The constraint then silently never fired
  (false negative). Both now resolve the induced slot for the class, matching
  the sh:path logic in the main slot loop.

- Combined operators: a single precondition / member condition dispatched on
  the first matching operator, so {minimum_value: X, maximum_value: Y} dropped
  the lower bound and under-constrained the trigger (false positives). A shared
  _scalar_filters helper now emits every recognised operator, and still returns
  None -- skip, never mis-translate -- when none is recognised.

- SPARQL string escaping: an equals_string / permissible-value name containing
  a double quote, backslash or newline produced invalid, unparseable SPARQL.
  New _sparql_string_literal escapes per SPARQL 1.1 section 19.7.

Tests: 6 new regression tests (structural, prepareQuery syntax, and pyshacl
end-to-end) covering all three defects; full shaclgen suite green (111).

Signed-off-by: Carlo van Driesten <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
(cherry picked from commit e263def)
@jdsika
jdsika force-pushed the fix/shaclgen-rule-converter-hardening branch from 15c4d0d to 4b2ef26 Compare July 11, 2026 10:38
@jdsika
jdsika force-pushed the feat/shaclgen-compositional-rule-fallback branch from 8442eb8 to 1ea1ea2 Compare July 11, 2026 10:38
@jdsika
jdsika force-pushed the feat/shaclgen-compositional-rule-fallback branch from 1ea1ea2 to 59544b4 Compare September 11, 2026 12:53
@jdsika

jdsika commented Sep 11, 2026

Copy link
Copy Markdown
Author

Closing: the content of this PR is not dropped, it has been folded into the
feature it corrects.

The original five-PR stack introduced two features and then corrected them in
two follow-up PRs. That meant shipping a PR that introduces a defect plus
another that fixes it — reviewer noise, and a stack whose intermediate commits
were never in a state anyone should merge.

The stack has been re-cut into one clean commit per feature:

main
└─ #19  presence-implies-value            (this PR's fixes to that pattern folded in)
   └─ #20  compositional fallback (M1–M5)  (this PR's fixes to the fallback folded in)
      └─ #23  documentation

Every fix from this PR is present in the re-cut branches; the tip of #23 is
byte-identical to the tip of the old stack apart from one removal, described
below. The re-cut also rebased onto current main — the old stack was 221
commits behind
— and dropped its base commit, whose content is already in
main via linkml#3451.

Two defects were found while re-cutting and are fixed in #19:

  • the boolean guard was not gated on the target slot's range, so a string
    slot with equals_string: "true" was compared as a boolean and flagged
    conforming data as violating;
  • test_rule_with_elseconditions_warns was defined twice, so the stack's copy
    silently shadowed the existing test of the same name. The redundant copy is
    the one removal referred to above.

Please review #19, #20 and #23 instead.

@jdsika jdsika closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants