Skip to content

Commit 87d775d

Browse files
committed
feat(loader): warn on enum vocabularies ambiguous under @normalize: strip (cross-port)
@normalize: strip -- the DEFAULT -- upper-cases and keeps only [A-Z0-9]. That erasure is what lets "SOCIAL-ATTACK" match the member SOCIAL_ATTACK, which is desired. It also means a DELIMITED value collapses into one token, so where a vocabulary contains a member equal to the concatenation of others, a stray delimited value coerces SUCCESSFULLY to the wrong member: values = {READ, WRITE, READWRITE}; input "read|write" -> READWRITE The field is reported EXTRACTED, not MALFORMED -- a plausible wrong value that anything branching on field state will trust. Proven against the real coercion path before writing the guard; the control case "friendly|hostile" against {FRIENDLY, HOSTILE} correctly returns MALFORMED, which isolates the hazard to concatenation-collision vocabularies. It cannot be fixed at coercion time: "read-write" legitimately means READWRITE, so the two readings are indistinguishable from the value alone. But the collision IS detectable from metadata, so all four loaders now warn the author at declaration time -- WARN_ENUM_NORMALIZE_AMBIGUOUS -- when a field.enum's own @values contains a member that word-breaks into two or more other members and the effective mode is strip. `collapse` is immune (folds only [\s_-]+, so a "|" survives and the value fails cleanly) and is the documented fix. Design notes: - WARNING, never an error: such a vocabulary is legal and completely unambiguous for exact matching, and the author may have no delimited input at all. - Word-break (DP), not a pairwise scan, so A + B + C == ABC is caught too. Deterministic, since every port must produce the identical warning. - Own-@values only: warns once at the declaring node, not on every field that extends it. Self excluded BY INDEX, not by value -- two distinct members can strip to the same string, which is a separate (duplicate) concern. - Mode-gated: collapse/none are structurally immune and are skipped. Cross-port: TS reference + Java + Python + C# (Kotlin inherits the JVM loader), gated by the shared warning-enum-normalize-ambiguous conformance fixture. A scan of all 54 @values sets in the corpus found zero pre-existing collisions, so no existing fixture changes and no generated output changes. Also records in the extract KNOWN_GAPS that splitting a delimited scalar into array elements (a @delimiter attr) is intentionally NOT offered, with the prior art and the ADR-0037 step-0 reasoning, so it does not get re-litigated. The supported shape is repeated elements / a JSON array + field.enum isArray: true. Suites: TS metadata 2278, Python 1637, Java metadata 1279, C# conformance 836 + render 290 + codegen 339 + cli 46 -- all green.
1 parent 831472e commit 87d775d

16 files changed

Lines changed: 868 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,35 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
1010
**npm**`metadata` + `codegen-ts`; **Maven Central**`maven-plugin`; PyPI / NuGet
1111
unchanged.
1212

13+
### Added — `WARN_ENUM_NORMALIZE_AMBIGUOUS`, an authoring guard for a silent enum mis-extraction
14+
15+
`@normalize: strip` — the **default** — upper-cases and keeps only `[A-Z0-9]`, which is what lets
16+
`"SOCIAL-ATTACK"` match the member `SOCIAL_ATTACK`. The same erasure means a *delimited* value
17+
collapses into a single token, so where a vocabulary contains a member equal to the concatenation of
18+
others, a stray delimited value coerces **successfully** to the wrong member:
19+
20+
```
21+
values = {READ, WRITE, READWRITE}; input "read|write" -> READWRITE
22+
```
23+
24+
The field is reported `EXTRACTED`, not `MALFORMED` — a plausible, wrong value that anything
25+
branching on field state will trust. It cannot be fixed at coercion time (`"read-write"`
26+
legitimately means `READWRITE`, so the two readings are indistinguishable from the value alone), but
27+
the collision **is** detectable from metadata. All four loaders now warn at declaration time when a
28+
`field.enum`'s own `@values` contains a member that word-breaks into two or more other members and
29+
the effective mode is `strip`. `collapse` is immune — it folds only `[\s_-]+`, so a `|` survives and
30+
the value fails cleanly — and is the documented fix for a field that can receive delimited input.
31+
32+
Advisory, never an error: such a vocabulary is legal and completely unambiguous for exact matching.
33+
Detection is word-break (not pairwise), so three-way collisions are caught; the warning fires once at
34+
the declaring node rather than on every field that `extends` it. Cross-port, gated by the shared
35+
`warning-enum-normalize-ambiguous` conformance fixture (TS / Java / Python / C#; Kotlin inherits the
36+
JVM loader). No existing fixture in the corpus collides, and no generated output changes.
37+
38+
Also recorded in the extract engine's `KNOWN_GAPS.md`: splitting a delimited scalar into array
39+
elements (a `@delimiter` attribute) is **intentionally not offered** — the supported way to express a
40+
multi-valued response field is repeated elements / a JSON array plus `field.enum` + `isArray: true`.
41+
1342
### Fixed — a shared `enums.ts` no longer collides across `entityFile()` instances (#266)
1443

1544
Declaring a root-level abstract `field.enum` made **every** `entityFile()` instance emit
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"errors": [],
3+
"warnings": [
4+
{
5+
"code": "WARN_ENUM_NORMALIZE_AMBIGUOUS",
6+
"source": {
7+
"format": "json",
8+
"files": [
9+
"meta.access.json"
10+
],
11+
"jsonPath": "$['metadata.root'].children[0]['field.enum']"
12+
}
13+
}
14+
]
15+
}
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"field.enum": {
7+
"name": "Access",
8+
"package": "acme",
9+
"abstract": true,
10+
"@values": [
11+
"READ",
12+
"WRITE",
13+
"READWRITE"
14+
]
15+
}
16+
}
17+
]
18+
}
19+
}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"metadata.root": {
3+
"package": "acme",
4+
"children": [
5+
{
6+
"field.enum": {
7+
"name": "Access",
8+
"abstract": true,
9+
"@values": [
10+
"READ",
11+
"WRITE",
12+
"READWRITE"
13+
]
14+
}
15+
}
16+
]
17+
}
18+
}

server/csharp/MetaObjects/Errors.cs

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,16 @@ public static class WarningCodes
224224
/// across different sources.
225225
/// </summary>
226226
public const string WARN_ORIGIN_UNDER_UNMANAGED = "WARN_ORIGIN_UNDER_UNMANAGED";
227+
228+
/// <summary>
229+
/// A field.enum whose @values contains a member equal to the concatenation of two or
230+
/// more OTHER members once <c>strip</c>-normalized (the default mode erases separators).
231+
/// A delimited value then collapses into that member and coerces SUCCESSFULLY — reported
232+
/// EXTRACTED with a wrong-but-valid value rather than MALFORMED. Advisory: such a
233+
/// vocabulary is legal and unambiguous for exact matching; <c>@normalize: collapse</c>
234+
/// is the fix when delimited input is possible.
235+
/// </summary>
236+
public const string WARN_ENUM_NORMALIZE_AMBIGUOUS = "WARN_ENUM_NORMALIZE_AMBIGUOUS";
227237
}
228238

229239
/// <summary>

server/csharp/MetaObjects/Loader/MetaDataLoader.cs

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -566,6 +566,18 @@ public LoadResult Load(IReadOnlyList<IMetaDataSource> sources)
566566
}
567567
}
568568

569+
// Authoring guard: a field.enum vocabulary ambiguous under the default
570+
// @normalize: strip. WARN_ENUM_NORMALIZE_AMBIGUOUS.
571+
var enumAmbiguity = ValidationPasses.ValidateEnumNormalizeAmbiguity(root);
572+
if (enumAmbiguity.Count > 0)
573+
{
574+
envelopeWarnings.AddRange(enumAmbiguity);
575+
foreach (var w in enumAmbiguity)
576+
{
577+
warnings.Add(w.Message);
578+
}
579+
}
580+
569581
// FR-014: TPH discriminator cross-attribute rules.
570582
errors.AddRange(ValidationPasses.ValidateDiscriminator(root));
571583

server/csharp/MetaObjects/Loader/ValidationPasses.cs

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2960,6 +2960,135 @@ public static IReadOnlyList<MetaError> ValidateTemplatePayloadRefs(MetaData root
29602960
return errors.AsReadOnly();
29612961
}
29622962

2963+
// =========================================================================
2964+
// Authoring guard — enum vocabularies ambiguous under @normalize: strip.
2965+
// WARN_ENUM_NORMALIZE_AMBIGUOUS
2966+
// Mirrors TS core/field/validate-enum-normalize-ambiguity.ts.
2967+
//
2968+
// `strip` (the DEFAULT) upper-cases and keeps only [A-Z0-9], erasing every
2969+
// separator. That is what makes "SOCIAL-ATTACK" match SOCIAL_ATTACK — desired.
2970+
// But it also collapses a DELIMITED value into one token, and if that token
2971+
// equals another member the extract engine coerces it SUCCESSFULLY:
2972+
// values = {READ, WRITE, READWRITE}; input "read|write" -> READWRITE
2973+
// reported EXTRACTED, not MALFORMED — a plausible wrong value.
2974+
//
2975+
// WARNING, not error: such a vocabulary is legal and unambiguous for exact
2976+
// matching. `collapse` folds only [\s_-]+ and `none` folds nothing, so neither
2977+
// can merge tokens across a delimiter like "|" — both are skipped.
2978+
// =========================================================================
2979+
2980+
/// <summary>`strip` normalization: ASCII upper-case, then keep only [A-Z0-9]. Mirrors Normalize.STRIP.</summary>
2981+
private static string StripNormalize(string s)
2982+
{
2983+
var sb = new System.Text.StringBuilder(s.Length);
2984+
foreach (var ch in s.ToUpperInvariant())
2985+
{
2986+
if ((ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9')) sb.Append(ch);
2987+
}
2988+
return sb.ToString();
2989+
}
2990+
2991+
/// <summary>
2992+
/// Word-break: can <paramref name="target"/> be segmented into two or more dictionary
2993+
/// entries? Returns the member names in order, or null. Word-break rather than a pairwise
2994+
/// scan so a three-way collision (A + B + C == ABC) is caught too.
2995+
/// </summary>
2996+
private static List<string>? SegmentInto(string target, List<(string Member, string Stripped)> dict)
2997+
{
2998+
int n = target.Length;
2999+
var best = new List<string>?[n + 1];
3000+
best[0] = new List<string>();
3001+
for (int i = 0; i < n; i++)
3002+
{
3003+
var prefix = best[i];
3004+
if (prefix is null) continue;
3005+
foreach (var (member, stripped) in dict)
3006+
{
3007+
int end = i + stripped.Length;
3008+
if (end > n || string.CompareOrdinal(target, i, stripped, 0, stripped.Length) != 0) continue;
3009+
var cand = new List<string>(prefix) { member };
3010+
var cur = best[end];
3011+
if (cur is null || cand.Count < cur.Count) best[end] = cand;
3012+
}
3013+
}
3014+
var full = best[n];
3015+
// Two or more segments: a single-segment match is just another member that strips
3016+
// to the same string — a different (duplicate) concern.
3017+
return (full is not null && full.Count >= 2) ? full : null;
3018+
}
3019+
3020+
/// <summary>Effective @normalize for an enum field: own/inherited → owning object → default.</summary>
3021+
private static string EffectiveNormalizeMode(MetaData field)
3022+
{
3023+
// ADR-0039: resolving accessor — an enum extending an abstract enum must see the
3024+
// super's @normalize.
3025+
if (field.Attr(FIELD_ATTR_NORMALIZE) is string own) return own;
3026+
var parent = field.Parent;
3027+
if (parent is not null && parent.Type == TYPE_OBJECT
3028+
&& parent.Attr(FIELD_ATTR_NORMALIZE) is string objMode)
3029+
{
3030+
return objMode;
3031+
}
3032+
return NORMALIZE_DEFAULT;
3033+
}
3034+
3035+
public static IReadOnlyList<LoaderWarning> ValidateEnumNormalizeAmbiguity(MetaData root)
3036+
{
3037+
var warnings = new List<LoaderWarning>();
3038+
VisitEnumNormalizeAmbiguity(root, warnings);
3039+
return warnings;
3040+
}
3041+
3042+
private static void VisitEnumNormalizeAmbiguity(MetaData node, List<LoaderWarning> warnings)
3043+
{
3044+
if (node.Type == TYPE_FIELD && node.SubType == FIELD_SUBTYPE_ENUM)
3045+
{
3046+
// ADR-0039 sanctioned own: check the vocabulary DECLARED here. A concrete enum
3047+
// inheriting @values shares the super's member set, already checked at the super —
3048+
// one hazard yields one warning, not one per referring field.
3049+
if (node.OwnAttr(FIELD_ATTR_VALUES) is System.Collections.IEnumerable rawEnum
3050+
&& node.OwnAttr(FIELD_ATTR_VALUES) is not string)
3051+
{
3052+
var members = new List<string>();
3053+
foreach (var o in rawEnum) members.Add(o?.ToString() ?? string.Empty);
3054+
if (members.Count > 1
3055+
&& EffectiveNormalizeMode(node) == NORMALIZE_DEFAULT)
3056+
{
3057+
var entries = members.Select(m => (Member: m, Stripped: StripNormalize(m))).ToList();
3058+
for (int i = 0; i < entries.Count; i++)
3059+
{
3060+
var self = entries[i];
3061+
if (self.Stripped.Length == 0) continue; // e.g. "_" — nothing to collide with
3062+
// Exclude self BY INDEX, not by value: two distinct members can strip
3063+
// to the same string, which is a separate (duplicate) concern.
3064+
var others = entries.Where((_, j) => j != i)
3065+
.Where(e => e.Stripped.Length > 0).ToList();
3066+
var seg = SegmentInto(self.Stripped, others);
3067+
if (seg is not null)
3068+
{
3069+
var plus = string.Join(" + ", seg.Select(s => $"'{s}'"));
3070+
var delimited = string.Join("|", seg.Select(s => s.ToLowerInvariant()));
3071+
warnings.Add(new LoaderWarning(
3072+
Code: WarningCodes.WARN_ENUM_NORMALIZE_AMBIGUOUS,
3073+
Message:
3074+
$"field.enum \"{node.Name}\" member '{self.Member}' is the " +
3075+
$"concatenation of {plus} under @{FIELD_ATTR_NORMALIZE}: " +
3076+
$"'{NORMALIZE_DEFAULT}' (the default), which erases " +
3077+
$"separators. A delimited value such as \"{delimited}\" would coerce " +
3078+
$"silently to '{self.Member}' and be reported as extracted rather " +
3079+
$"than malformed. Set @{FIELD_ATTR_NORMALIZE}: " +
3080+
"'collapse' on this field if it can receive delimited input.",
3081+
Source: node.Source));
3082+
break; // one warning per declaring node
3083+
}
3084+
}
3085+
}
3086+
}
3087+
}
3088+
// ADR-0039 sanctioned own: structural walk of what each node declares.
3089+
foreach (var child in node.OwnChildren()) VisitEnumNormalizeAmbiguity(child, warnings);
3090+
}
3091+
29633092
// =========================================================================
29643093
// FR-013 — field-level @readOnly cross-attribute rules.
29653094
// ERR_READONLY_ASSIGNED_PRIMARY / ERR_READONLY_DOWNGRADE / WARN_READONLY_VALUE_OBJECT

0 commit comments

Comments
 (0)