From f7c18a4e7866b7c073baf652817042cd6376a17e Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 15:58:15 -0700 Subject: [PATCH 1/9] docs(design): record the vocabulary-collision criterion The ambiguous-subset mechanism shipped twice -- particles (39/67, PARTICLE_OR_GIVEN) and suffix_acronyms (4/613, SUFFIX_OR_NAME) -- without its criterion written down anywhere, and titles (711) has no ambiguous subset at all. Record it. C-i: a word earns its set's ambiguous marking iff it is also borne as an ordinary name somewhere, defaulting to ambiguous under uncertainty -- the evidence standard already in NON_GIVEN_NAME_PARTICLES' docstring, generalized from "which set" to "which subset". C-ii: where two sets claim a word and neither reading is a name (vd: never-given particle vs. the British Volunteer Decoration), precedence is a per-word frequency judgement; vd reads as the Dutch van der. Also record that the 58%-vs-0.65% gap between the two shipped subsets is base rate, not disagreement -- most particles double as names, most credential acronyms do not. It reads as an inconsistency, and a reviewer who harmonizes the two shares breaks one of them. Applies to #348 (the 711-entry census, blocked on a given-name corpus this repo lacks), #360, #342, and #385 -- which the criterion resolves outright: "Do" is a borne surname, so it anchors family_base, while "van der" is never anyone's name and genuinely has no base. R2 gains the deviates: marker. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 70 +++++++++++++++++++++++++++++++++++++--- docs/design/rules.md | 11 ++++--- 2 files changed, 72 insertions(+), 9 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 9e27ac5f..2b441e24 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -195,6 +195,57 @@ Declined: unambiguously). Documented side effect: parenthesized bare "(MA)"/"(DO)" no longer escape to suffix as in 1.x. +### vocabulary-collisions — when a word earns the ambiguous marking + +The mechanism shipped twice before anyone wrote down its criterion. +Sizes as of 2.2.0dev: particles 67 with 39 ambiguous (58%, +PARTICLE_OR_GIVEN); suffix_acronyms 613 with 4 ambiguous (0.65%, +SUFFIX_OR_NAME); titles 711 with no ambiguous subset and no +AmbiguityKind at all. + +- 2026-08-16 (collision keystone; #348, #360, #342, #385) — the + 58%-vs-0.65% gap is BASE RATE, not disagreement. Both sets apply + the same test; most particles are short words that double as + names (van, bin, le, do, bar, mac) while most credential acronyms + are not (abpp, acp). Recorded because the gap reads as an + inconsistency and is not one — a reviewer who "harmonizes" the + two shares will break one of them. +- **C-i, vocabulary vs. name.** A word belongs in its set's + ambiguous subset iff it is also borne as an ordinary name (given + or family) in some tradition. Under uncertainty, default to + AMBIGUOUS. This generalizes the evidence standard already stated + in NON_GIVEN_NAME_PARTICLES' docstring — a wrong unambiguous + claim misparses a real person, a wrong ambiguous marking only + adds a flag — from "which set" to "which subset". Applies + uniformly to particles, suffix_acronyms and titles. +- **C-ii, vocabulary vs. vocabulary.** Where two sets claim a word + and NEITHER reading is a name, precedence is a frequency judgment + recorded per word. vd is the live case: never-given particle AND + credential acronym (the British Volunteer Decoration), neither of + them a name. Decision: the Dutch van der reading, as the more + common. That is what unblocks #380, whose trailing-orphan half is + a separate decision recorded under its own rule. +- The concrete demonstration is "do", which three vocabularies + claim — titles, particles/ambiguous, suffix_acronyms/ambiguous. + Two mark it ambiguous; the third, TITLES, is the one that + actually decides "Do Quang Minh" (title="Do", given="Quang") and + reports nothing. Same word behind #385's "Anh Do". +- Applications, each still its own work: #360 (mc, ste — neither is + a borne given name, so both leave the ambiguous half); #342 (rai + — Rai IS a borne surname, so it earns the marking rather than + moving); #385 (do — resolved at decisions.md#R2). +- Caution when applying C-i to the particle set: TITLES ∩ ambiguous + == {do, freiherr, st} is load-bearing, per the Excluded note in + the W2 section. Emptying it makes the particle-or-given emitter + dead code. + +Open: [#348](https://github.com/derek73/python-nameparser/issues/348) +applying C-i to the 711 title entries, then titles_ambiguous plus a +TITLE_OR_GIVEN kind. Blocked on data, not on judgement — the census +needs a given-name frequency corpus this repo does not have, which +is why the criterion is recorded here and the census is not +attempted. + ### deviates-registry — packs stay pure data (option C) - 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` @@ -1041,11 +1092,20 @@ Declined: v2 core does not (parse("Anh Do").family_base == "", family_particles == "Do"), and the surname vanishes from initials (parse("Anh Do").initials() == "A."). - -Open: -[#385](https://github.com/derek73/python-nameparser/issues/385) -whether an all-particle family should have an empty base (three -options weighed in the issue). +- 2026-08-16 #385 RESOLVED by the collision criterion, not on its + own terms: the issue's option 3 ("guard the view only when a word + is vocabulary-ambiguous") is what + decisions.md#vocabulary-collisions produces when applied here. + "Do" is borne as an ordinary surname, so it is ambiguous + vocabulary and anchors the base; "van der" is never anyone's + name, so an all-particle family there genuinely has no base. The + two rows of the issue's table were never one case. + This is the keystone's clearest payoff: #385 was filed as a leaf + with three options and no way to choose between them, and the + criterion picks one without arguing about family_base at all. +- Still open inside the resolution: whether "Do" remains in + family_particles once it is also the base. Recorded here rather + than left to the implementing PR to decide by accident. ### removed-v1-surface diff --git a/docs/design/rules.md b/docs/design/rules.md index f5f4a36a..10edfff6 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -709,10 +709,13 @@ R2. Rationale: callers need the surname with and without its "Dr. Juan Q. Xavier de la Vega III" → family_base="Vega" "Dr. Juan Q. Xavier de la Vega III" → family_particles="de la" "Sean O'Connor" → family_base="O'Connor" · boundary - Accepted: an all-particle family reads an empty base today; - whether it should is #385. - "Anh Do" → family_base="" - history: decisions.md#R2 · implemented: nameparser/_types.py + A family name written wholly out of particle vocabulary still + has a base where one of those words is itself borne as an + ordinary surname: that word anchors the base, and only the words + that are never anyone's name stay particles. + "Anh Do" → family_base="Do" deviates: #385 (today: family_base="") + "Juan van der" → family_base="" + history: decisions.md#R2 · interacts: R3 · implemented: nameparser/_types.py R3. Rationale: initials abbreviate the person's name words; titles, suffixes, particles and nicknames are not name words. From 9000cc2fb2ff90dc72edf7f771c4379c0b670142 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 15:59:53 -0700 Subject: [PATCH 2/9] docs(design): the order-precedence keystone, and reverse #368 Grouping is vocabulary's job and is order-independent; assignment is name_order's job. #364, #365 and #368 are one question seen from three angles: P1's fold sites and P2's chain are expressed over ROLES, and roles come from _effective_order, so both rules' coverage varied with the declared order by accident rather than by design. P1 now states that the fold takes the particle and the ONE name word it attaches to. "Takes everything" was never argued for -- it was the shape of v1's handle_non_first_name_prefix. Measured against the three differential corpora (782 names), filtered to the shape that actually changes, this moves exactly one name, "de Mesnil Garcia"; #364's warning that "each ledger would need re-examining" was written without the filter. #368 REVERSES: shipped behavior is correct. A mid-name chain has a head word and is positioned like any other group, so "Juan de la Vega" under FAMILY_FIRST is family="Juan". What NON_GIVEN_NAME_PARTICLES guarantees is that the bare word never reads as a given name, not that no name part may begin with one. The asymmetry with the leading case is P4's: a leading particle chains nothing, so without the fold pure position makes the bare particle the given name -- measurable today on the ambiguous half, where "van Mesnil Juan" gives given="van". The deviates: marker comes out and the issue closes won't-fix. New rule P6 records the trailing orphan: a particle ending the name attaches to the family beside it and renders before it, scoped to the comma form. The words-to-spare guard is load-bearing -- #379's own subject "van" is in the ambiguous half, so a never-given-only rule would not fix the issue it was filed for, while an unguarded rule breaks Vietnamese "Nguyen, Van". Rules P1 and P6 carry deviates: markers for #364, #379 and #380; P6 has no implemented: pointer because nothing implements it yet. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 125 ++++++++++++++++++++++++---- docs/design/rules.md | 47 +++++++++-- nameparser/_pipeline/_post_rules.py | 11 ++- 3 files changed, 158 insertions(+), 25 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 2b441e24..1c1fb1e6 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -77,15 +77,9 @@ The counter-example set has since shrunk, and its shrinkage is the section's history: "Sir de Mesnil" fell to #367 (titles became transparent); "Juan de la Vega" under family-first — the whole chain in the given position — was called working-as-intended by -#359, but #368 SUPERSEDES that sentence: the recorded decision is -that the particle wins and a chain becomes the family name -whatever order was declared, so that case is now P1's tracked -deviation, not its boundary. The survivor is the degenerate bare -"de". The MIDDLE position is deliberately not a fold site — and -not merely unimplemented: the two family-first orders disagree -there ("Mesnil Garcia de" strands middle="de" under FAMILY_FIRST -and folds under FAMILY_FIRST_GIVEN_LAST, 464 measured inputs), -which is what makes #365 a decision rather than a gap. +#359, contested by #368, and is working-as-intended again as of +the 2026-08-16 entries below. The survivor is the degenerate bare +"de". - 2026-08 #359 — the opening site is read from joining structure (pieces), not from assigned roles, so the fold holds under every @@ -94,6 +88,59 @@ which is what makes #365 a decision rather than a gap. - 2026-08 #367 — titles are transparent to the fold: "Sir de Mesnil" now reads like "de Mesnil". Fixed by removing the title→particle chain in grouping, not by touching this rule. +- 2026-08-16 (order-precedence keystone; #364, #365, #368) — the + stage split is the decision, and the three issues are one + question seen from three angles. GROUPING is vocabulary's job and + is order-independent: a particle joins forward through + consecutive particles and stops at the first non-particle, and no + name_order moves that stopping point. ASSIGNMENT is name_order's + job: groups take roles by the declared order. The bugs existed + because the implementation runs the two in the opposite + dependency — `assign` hands out positions from `_effective_order` + and `post_rules` then inspects a fixed list of ROLES, so P1's + fold sites and P2's chain had coverage that varied with the + declared order by accident. + Consequences, each recorded in its own right below: the fold + takes only the particle's own group (#364); the middle position + needs no third site once grouping is order-independent (#365); + and #368 reverses. +- 2026-08-16 #364 — the fold takes the particle and the ONE name + word it attaches to, not every remaining word. "de Mesnil Juan" + is family="de Mesnil" plus given="Juan". Nothing ever argued for + "takes everything"; it was the shape of v1's + handle_non_first_name_prefix, not a decision. + Measured before deciding: filtering the three differential + corpora (782 names) to the shape that actually changes — no + comma, so C1 does not take it first; leading never-given + particle; three or more words — gives exactly ONE name, + "de Mesnil Garcia". #364's own body warns the change "breaks the + v1 parity tools/differential protects" and that "each ledger + would need re-examining"; measured, it is one name and one ledger + entry. The warning was written without the filter and is + corrected on the issue. +- 2026-08-16 #368 REVERSED — shipped behavior is correct. + "Juan de la Vega" under FAMILY_FIRST groups [Juan][de la Vega] + and assigns family="Juan", given="de la Vega". The earlier + decision ("the particle wins ... whatever order was declared") + was made before the grouping/assignment split was stated and + cannot survive it: a mid-name chain HAS a head word and is + positioned like any other group. What NON_GIVEN_NAME_PARTICLES + guarantees is that the bare word never reads as a given name, not + that no name part may begin with one. + The asymmetry with the leading case is P4's, not an exception + invented here: a leading particle chains nothing, so without the + fold pure position makes the BARE particle the given name. That + is measurable today on the ambiguous half, where no fold fires — + "van Mesnil Juan" gives given="van", middle="Mesnil", + family="Juan". given="de" is the reading the vocabulary exists to + forbid, and the fold is what prevents it. + The all-orders agreement in P1 is deliberate and is W4's shape: a + wholly-hangul name reads family="김" under every declared order + because the script carries a signal the order does not override, + and a leading never-given particle is the Latin-script analogue. + decisions.md#O4 already draws the line — "Words no vocabulary has + claimed read by position" — so name_order governs the unclaimed + remainder, which is most inputs. Declined: @@ -106,13 +153,20 @@ Declined: "St John Smith" into one given name and broke test_add_title (which adds "te", also a particle). The shipped predicate is "not a title or a prefix". - -Open: [#364](https://github.com/derek73/python-nameparser/issues/364) -how much the fold takes · -[#365](https://github.com/derek73/python-nameparser/issues/365) -should the middle position be a third site · -[#360](https://github.com/derek73/python-nameparser/issues/360) -which particles count as never-given. +- 2026-08-16 — deleting P4 so a leading particle chains and is then + positioned, which is the only way to make "de Mesnil Juan" vary + by declared order. It avoids given="de" (the group would be + [de Mesnil]) but breaks "de la Vega": measured, a single group is + positioned by the declared order — "Cher" reads given under + GIVEN_FIRST — so "de la Vega" would read given="de la Vega" + unless a further rule forced a particle-headed group into the + family. Add that rule and [de Mesnil][Juan] yields the #364 + reading anyway, so the deletion buys nothing and costs P4. + +Open: [#360](https://github.com/derek73/python-nameparser/issues/360) +which particles count as never-given (the criterion is settled at +decisions.md#vocabulary-collisions; the 39-member application is +not). ### P2 — particles join forward @@ -132,6 +186,45 @@ which particles count as never-given. the same shape with opposing wants, which is why the combined reading lives in the surnames VIEW and the split in the fields. +### P6 — the trailing orphan particle + +- 2026-08-16 (order-precedence keystone; #379, #380, #365) — a + particle ending the name has nothing to link forward to, and no + particle is a name by itself, so it attaches to the family name + standing beside it and renders BEFORE it. The distinction from a + chain is what makes this a rule rather than an exception: a + chained group has a head word and can be positioned; an orphan + has no head, so position has nothing to work with. +- Scope: the COMMA form only, deliberately. "Jong, Anke de" is + unambiguous — the comma has already named the family. Without the + comma the written shape is not settled: "Jong Anke de" may be a + misformatted listing (arguably a missing comma under a declared + family-first order) and "Jong de" may be a given name beside a + particle. Those keep their positional reading and are not tracked + as deviations. +- The words-to-spare guard is load-bearing, not incidental. #379's + own subject is "van", which is in the AMBIGUOUS half — so a rule + keyed to never-given particles alone would not fix the issue it + was filed for, while a rule with no guard breaks Vietnamese + "Nguyen, Van" (given="Van") by eating the only given word. The + guard is S2's shape, reused: consume only when the name has words + to spare. +- Rendering before the family has precedent in rules.md#R1 — folded + family words under O3 already "render before the rest of the + family wherever they stood in the string". The surnames view + renders backwards today ("Vega de la", "Jong de") and flips with + this rule. +- Measured, default-order shapes that move: "Vega, Juan de la", + "Smith van der", "Sander van". Unlike #364 this is NOT a one-name + change; it wants a tools/differential run at all three baselines + with ledger entries, which belongs to the implementing PR rather + than to this record. + +Open: [#380](https://github.com/derek73/python-nameparser/issues/380) +covers "Berg, Jan vd" under this rule, but the vd reading itself is +decisions.md#vocabulary-collisions (C-ii); and the no-given-word +case "Jong, vd" is deliberately unresolved — see the scope note. + ### M2 — the maiden-marker rule - 2026-07-03 (maiden-bucket design; #274 filed 2026-07-07, landed diff --git a/docs/design/rules.md b/docs/design/rules.md index 10edfff6..244bc3b2 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -163,19 +163,31 @@ P1. Rationale: a never-given particle standing alone cannot be one as the given name, is a surname written out in full. A never-given particle standing alone where the given name would go — or opening the name — marks the name as surname-only: the - given and middle words fold into the family. It needs another - name word to fold into. An ambiguous particle keeps whatever - reading its position gives it. Whether the fold should stop at - the particle group instead of taking everything is open (#364). + particle and the one name word it attaches to are the family, + and any name words beyond that read by position. It needs + another name word to attach to. An ambiguous particle keeps + whatever reading its position gives it. The reading holds under + every declared name order: a never-given particle is evidence + about how the name is written, and a declared order governs only + what no vocabulary has claimed (O4) — the same precedence the + script license takes in W4. "de la Vega" → family="de la Vega" "Sir de Mesnil" → family="de Mesnil" "Mesnil de" family-first → family="Mesnil de" - "Juan de la Vega" family-first → family="de la Vega" deviates: #368 (today: family="Juan") + "de Mesnil Juan" → family="de Mesnil" deviates: #364 (today: family="de Mesnil Juan") + "de Mesnil Juan" → given="Juan" deviates: #364 (today: given="") "van Gogh" → given="van" · boundary Accepted: a bare "de" stays the given name — there is nothing to fold into, and inventing a surname would be worse. "de" → given="de" - history: decisions.md#P1 · interacts: P2 · implemented: nameparser/_pipeline/_post_rules.py + Accepted: only the OPENING position is this rule's subject. A + particle chain standing inside the name is grouped normally (P2) + and positioned by the declared order, so a family-first reading + may report it as the given name; what the vocabulary forbids is + the bare particle reading as a given name, not any name part + that begins with one. + "Juan de la Vega" family-first → family="Juan" + history: decisions.md#P1 · interacts: P2, P4, P6 · implemented: nameparser/_pipeline/_post_rules.py P2. Rationale: a particle is written as part of the surname it precedes, and a title stands outside the name entirely. @@ -234,6 +246,29 @@ P5. Rationale: some given-name words are incomplete alone — "abdul" "mohamad ali smith" → given="mohamad" · boundary history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py +P6. Rationale: a particle ending the name has nothing to link + forward to, and no particle is a name by itself — so it belongs + to the family name written beside it. Dutch and Flemish names + are listed exactly this way ("Beethoven, Ludwig van"), the + tussenvoegsel trailing the given name but belonging to the + surname. + Where a family comma has already named the family, a particle + ending the name attaches to that family name and is written + before it — provided at least one given word remains, so that a + name whose only given word is the particle keeps it (the + words-to-spare test S2 applies to ambiguous suffixes). + "Jong, Anke de" → family="de Jong" deviates: #379 (today: family="Jong") + "Beethoven, Ludwig van" → family="van Beethoven" deviates: #379 (today: family="Beethoven") + "Berg, Jan vd" → family="vd Berg" deviates: #380 (today: family="Berg") + "Nguyen, Van" → given="Van" · boundary + Accepted: without a family comma the name's written shape is not + settled — "Jong Anke de" may be a misformatted listing, and a + bare "Jong de" may be a given name beside a particle — so the + attachment is scoped to the comma form, and the comma-less + shapes keep their positional reading. + "Jong Anke de" → family="de" + history: decisions.md#P6 · interacts: C1, P1, S2 + ## Suffixes: generational & credentials (S) Background: what follows a name is one of two different things — diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index cca18904..5e470768 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -95,9 +95,14 @@ def post_rules(state: ParseState) -> ParseState: # rules.md#P1: "a never-given particle standing alone where the # given name would go — or opening the name — marks the name as - # surname-only: the given and middle words fold into the family. - # It needs another name word to fold into." (v1 - # handle_non_first_name_prefix; history: decisions.md#P1) + # surname-only: the particle and the one name word it attaches + # to are the family, and any name words beyond that read by + # position." (v1 handle_non_first_name_prefix; history: + # decisions.md#P1) + # DEVIATION #364: the fold below still takes every remaining + # name word, not just the particle's own; "de Mesnil Juan" gives + # family="de Mesnil Juan" where the rule says family="de Mesnil" + # plus given="Juan". Pinned by the deviates: markers on P1. # Code-local: a lone PIECE is the test at both sites, so a # particle group already chained forward is not a lone particle, # and rule H1 above cannot be what produces the fold's family From 1024578c23713cc7944ebfef9d4c7fdfb56dcab7 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 16:00:31 -0700 Subject: [PATCH 3/9] docs(design): record what the suffix field actually holds #326 asks whether suffix should split into generation and credentials, arguing the split is tractable because "the vocabulary is already split". That was true when the issue was written (2026-08-02). The 2.1.0 East Asian work landed five days later and put 24 postnominal honorifics -- 20 CJK, 4 Hebrew -- in suffix_words, which now reads 11 generational, 5 neither, 24 honorific. The largest group is neither of CLDR's two buckets, so "adopt CLDR's model" is not available as the cheap answer. Decision: do not split. Record the composition so #296, #291, #325 and #289 stop waiting on a distinction the model cannot express, and note the question this leaves unexamined -- whether the honorifics belong in `suffix` at all, given rules.md#W3 already calls an honorific "no part of the name on either side". Recorded as a failure mode rather than a fact: the field widened three times without any step being recorded as a widening of its MEANING. rules.md#S2's Background still calls it "two different things", accurately as written. The rules doc pinned the behavior; what slipped is the field's definition, which no rule owns. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 1c1fb1e6..2da74d75 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -339,6 +339,43 @@ needs a given-name frequency corpus this repo does not have, which is why the criterion is recorded here and the census is not attempted. +### suffix-field-composition — three kinds of thing in one field + +- 2026-08-16 (suffix keystone; #326) — measured composition of + suffix_words (40 entries): 11 generational (i, ii, iii, iv, v, + jr, jnr, sr, snr, junior, 2), 5 neither (dr, esq, esquire, ret, + vet), and 24 POSTNOMINAL HONORIFICS — 20 CJK (さん, さま, くん, + ちゃん, 様, 殿, 氏, 先生, 博士, 教授, 女士, 小姐, 씨, 양, 군, 님, + 박사, 박사님, 교수님, 선생님) and 4 Hebrew (ז"ל, ז״ל, שליט"א, + שליט״א). The honorifics are the LARGEST group. +- Decision: do NOT split the field. Record the composition; #296, + #291, #325 and #289 proceed on their own terms rather than + waiting on it. +- Why #326 cannot be taken at face value: it argues the split is + tractable because "the vocabulary is already split — + suffix_acronyms is credentials, entirely; suffix_words is + generational plus a handful". That was true when written + (2026-08-02); the 2.1.0 East Asian work landed 2026-08-07 and put + the 24 honorifics in the same set. CLDR's two buckets do not + cover what the set now holds, so "adopt CLDR's model" is not + available as the cheap answer. The issue's table is corrected on + the issue. +- The unexamined question this leaves: whether the honorific bucket + belongs in `suffix` at all. rules.md#W3 already argues an + honorific is "no part of the name on either side" — the same + language the H section uses for prenominal titles — so the + conflation with PhD may be worse than the generation/credentials + one #326 was filed about. Deliberately not decided here. +- Recorded as a documentation failure mode, not just a fact: + `suffix` started generational, absorbed credentials, then + absorbed postnominal honorifics, each step locally reasonable and + none recorded as a widening of the field's MEANING. rules.md#S2's + Background still calls it "two different things", which was + accurate when written. The rules doc pinned the behavior + faithfully; what slipped is the field's definition, which no rule + states because no rule owns it. Field definitions need the same + discipline rule statements get. + ### deviates-registry — packs stay pure data (option C) - 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` From bd80f6f6434ae3003930e3860748ade07f68413a Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 18:15:26 -0700 Subject: [PATCH 4/9] docs(design): pre-merge fact-check delta, and the spec-residue sweep Two measurement claims in this branch were wrong, both found by re-running rather than re-reading. decisions.md#P1 said the #364 shape filter "gives exactly ONE name". It gives three; one moves. Restated from a prefilter-free scan of all 782 corpus names -- exactly one family holds words beyond its particle's group -- with the two non-movers named and the reason each stays put. The substantive claim (one name, one ledger entry) survives; the stated method did not support it. decisions.md#P6 listed three default-order movers, two of which its own comma scope excludes: "Smith van der" and "Sander van" have no comma. They were measured before the comma scoping was chosen and carried forward unfiltered. The real count under the committed scope is one. The entry now also says why that number is not reassuring -- 245 comma names in the corpus and exactly one with a trailing particle, so the Dutch listing this rule exists for is essentially unsampled, and the implementing PR should run the differential regardless. Two field notes to mechanisms.md, both traps hit during the fact-check itself: a corpus can be near-blind to a writing CONVENTION even where the vocabulary is well covered (report the population that could move, not just how many did); and a detector that re-implements a rule's grouping gets it wrong unless it reads the same vocabulary the rule does -- walking the never-given run alone split "de la Vega" after "de la" and reported 50 false movers. AGENTS.md gains the shape-before-value triage note: the spec residue with no committed home, per the Landing-a-design checklist. The remaining homeless spec sections -- the per-issue "what this buys" table and the suggested sequencing -- are navigational and a plan respectively, and die with the branch as intended. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 15 +++++++++++++ docs/design/decisions.md | 46 +++++++++++++++++++++++++++------------ docs/design/mechanisms.md | 15 +++++++++++++ 3 files changed, 62 insertions(+), 14 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8880729d..7f267a24 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -78,6 +78,21 @@ not deferred (a 2026-08-16 sweep of eight weeks of specs recovered nine such items). The same-PR amendment rule above covers code-driven changes; this covers the design-driven ones. +**Triaging a design backlog: shape before value.** When a pile of +open questions has to be ordered, relatedness will not partition it — +in a parser nearly everything touches particles or suffixes. The line +that does is whether a decision changes WHAT THE MODEL CAN EXPRESS or +fills in a value in a shape already fixed. Settle the shape questions +first, as a batch: they are few, they are usually independent of each +other (so their own ordering does not matter), and each one collapses +or re-frames a run of the value questions below it. The rest are +leaves needing one measurement and one answer apiece, in any order. +Worked example, 2026-08-16 (#386): 30 open issues and ~13 open design +questions reduced to three shape decisions, which decided or +re-framed fourteen of them — and two of the three turned out smaller +than their issues claimed, because the issues had gone stale. Check +the age of an issue's premises before believing its cost estimate. + **Primary-source review.** When doc content is distilled from a session's work, have that session (or its transcript) review its own sections before or soon after landing — attribution flattening diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 2da74d75..24dda85c 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -109,15 +109,24 @@ the 2026-08-16 entries below. The survivor is the degenerate bare is family="de Mesnil" plus given="Juan". Nothing ever argued for "takes everything"; it was the shape of v1's handle_non_first_name_prefix, not a decision. - Measured before deciding: filtering the three differential - corpora (782 names) to the shape that actually changes — no - comma, so C1 does not take it first; leading never-given - particle; three or more words — gives exactly ONE name, - "de Mesnil Garcia". #364's own body warns the change "breaks the - v1 parity tools/differential protects" and that "each ledger - would need re-examining"; measured, it is one name and one ledger - entry. The warning was written without the filter and is - corrected on the issue. + Measured before deciding, over all 782 names of the three + differential corpora with NO string prefilter: exactly ONE family + holds words beyond its particle's group — "de Mesnil Garcia". + #364's own body warns the change "breaks the v1 parity + tools/differential protects" and that "each ledger would need + re-examining"; measured, it is one name and one ledger entry. + A shape filter (no comma, leading never-given particle, three or + more words) returns THREE candidates, of which two do not move: + "de la Vega" is one group start to finish, and "de Mesnil Jr." + has only two name words because Jr. is a suffix. The pre-merge + fact-check caught this stated as "the filter gives one name", + which it does not. + Measurement trap, recorded because the fact-check fell into it + twice: the particle group runs through ANY particle, not only the + never-given ones — "de la Vega" chains never-given "de" through + AMBIGUOUS "la" onto "Vega". A detector that walks only the + never-given run splits the group after "de la" and reports 50 + false movers. - 2026-08-16 #368 REVERSED — shipped behavior is correct. "Juan de la Vega" under FAMILY_FIRST groups [Juan][de la Vega] and assigns family="Juan", given="de la Vega". The earlier @@ -214,11 +223,20 @@ not). family wherever they stood in the string". The surnames view renders backwards today ("Vega de la", "Jong de") and flips with this rule. -- Measured, default-order shapes that move: "Vega, Juan de la", - "Smith van der", "Sander van". Unlike #364 this is NOT a one-name - change; it wants a tools/differential run at all three baselines - with ledger entries, which belongs to the implementing PR rather - than to this record. +- Measured under this rule's OWN comma scope, over 782 corpus names + (245 of them comma-bearing): exactly one moves, + "Vega, Juan de la" → family="de la Vega". A pre-merge fact-check + corrected an earlier count of three here — "Smith van der" and + "Sander van" have no comma, so the rule as scoped does not reach + them; they were measured before the comma scoping was chosen and + carried forward unfiltered. +- What that number means is the opposite of reassuring. The corpus + holds 245 comma names and exactly ONE with a trailing particle, + so it is very nearly blind to the shape this rule governs — + Dutch and Flemish listings are barely represented. Treat the one + as evidence about the corpus, not about the blast radius, and + run tools/differential at all three baselines in the implementing + PR regardless of how small this looks. Open: [#380](https://github.com/derek73/python-nameparser/issues/380) covers "Berg, Jan vd" under this rule, but the vd reading itself is diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index 495c9037..3f86b207 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -536,6 +536,21 @@ decline, not delete the test. down, and an unrecognized word is by definition outside the vocabulary — a green run over the corpus proves nothing about such a rule. +- The corpus can be near-blind to a WRITING CONVENTION even where + the vocabulary is well covered, and a small honest count then + reads as a small blast radius. Measured for rules.md#P6: 245 of + 782 corpus names carry a comma and exactly ONE ends in a + particle, so the Dutch trailing-tussenvoegsel listing the rule + exists for is essentially unsampled. Before reporting "N names + move", report the size of the population that COULD move; when + that is ~1, the number is evidence about the corpus. +- A detector that re-implements a rule's grouping will get the + grouping wrong. Derive the boundary from the same vocabulary the + rule reads, not from the half you happen to be thinking about: + walking a particle run over the NEVER-GIVEN set alone (the rule + chains through ANY particle) split "de la Vega" after "de la" + and reported 50 false movers for #364, where the true count is + one. Both wrong answers were plausible and printed cleanly. - Guard the whole family, parametrize over it: a defect on one of N parallel entry points hides behind a per-example test — three times in one session (a guard on one class of two, a decode hint From b1d38335b5b1824f05fdb3b75c082fdcb7f182c3 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 18:35:18 -0700 Subject: [PATCH 5/9] docs(design): make the perishable counts recomputable Review question: are the numbers in these entries useful later? Audited what this branch introduced. rules.md is clean -- no bare numbers, only rule IDs, which is right for a normative doc. The perishable counts are all in decisions.md, and they split three ways. Counts that carry an argument stay: "245 of 782 comma names, exactly one with a trailing particle" IS the corpus-blindness finding, and the two one-name movers are the findings themselves. One count carried no argument and is deleted rather than dated: "over all 782 names of the three differential corpora" becomes "over every name in the three differential corpora". The phrase already said whole-corpus-no-prefilter; the integer only offered a way to go stale. The two vocabulary compositions get a recompute one-liner. These are the #326 hazard reproduced one level up -- the suffix entry's own thesis is that #326 quoted a composition and rotted in five days, and the mitigation I gave it was a date, which is the same mitigation #326 had. decisions.md's convention already makes dated entries frozen evidence; what was missing is a way for a reader who wants today's answer to get it without trusting the snapshot. A test asserting the counts is explicitly declined in the entry: that is the constant-content pattern and would fail on every legitimate vocabulary addition. AGENTS.md gains the general rule, since this will recur wherever an entry quotes something that drifts. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 15 +++++++++++++++ docs/design/decisions.md | 23 +++++++++++++++++++++-- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7f267a24..9a5d1065 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -93,6 +93,21 @@ re-framed fourteen of them — and two of the three turned out smaller than their issues claimed, because the issues had gone stale. Check the age of an issue's premises before believing its cost estimate. +**A count in a dated entry is evidence, not a live fact.** decisions.md +entries are snapshots by convention, so measurements belong in them — +but a reader wanting TODAY's number must not have to trust the +snapshot's date. Where an entry quotes something that drifts +(vocabulary sizes, corpus counts, set compositions), give the +one-liner that recomputes it, and phrase the argument so it survives +the digits moving — "the two shares differ by orders of magnitude" +outlives "58% vs 0.65%". A count that carries no argument is better +deleted than dated: "over every name in the corpus, no prefilter" +says what "over all 782 names" says, and cannot go stale. Do NOT +reach for a test asserting the count — that is the constant-content +pattern, and it fails on every legitimate vocabulary addition. #326 +is the cautionary case: it quoted a vocabulary composition, carried a +date, and was stale in five days. + **Primary-source review.** When doc content is distilled from a session's work, have that session (or its transcript) review its own sections before or soon after landing — attribution flattening diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 24dda85c..c06b7510 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -109,7 +109,7 @@ the 2026-08-16 entries below. The survivor is the degenerate bare is family="de Mesnil" plus given="Juan". Nothing ever argued for "takes everything"; it was the shape of v1's handle_non_first_name_prefix, not a decision. - Measured before deciding, over all 782 names of the three + Measured before deciding, over every name in the three differential corpora with NO string prefilter: exactly ONE family holds words beyond its particle's group — "de Mesnil Garcia". #364's own body warns the change "breaks the v1 parity @@ -312,7 +312,13 @@ The mechanism shipped twice before anyone wrote down its criterion. Sizes as of 2.2.0dev: particles 67 with 39 ambiguous (58%, PARTICLE_OR_GIVEN); suffix_acronyms 613 with 4 ambiguous (0.65%, SUFFIX_OR_NAME); titles 711 with no ambiguous subset and no -AmbiguityKind at all. +AmbiguityKind at all. Those counts are evidence from the decision +date, not live facts — they drift with every vocabulary addition. +The argument does not depend on the digits (it depends on the two +shares differing by orders of magnitude), but recompute before +quoting them: + + uv run python -c "from nameparser import Parser; L=Parser().lexicon; print({s: len(getattr(L,s)) for s in ('particles','particles_ambiguous','suffix_words','suffix_acronyms','suffix_acronyms_ambiguous','titles')})" - 2026-08-16 (collision keystone; #348, #360, #342, #385) — the 58%-vs-0.65% gap is BASE RATE, not disagreement. Both sets apply @@ -393,6 +399,19 @@ attempted. faithfully; what slipped is the field's definition, which no rule states because no rule owns it. Field definitions need the same discipline rule statements get. +- The 11/5/24 split above is this entry's own version of the hazard + it describes: a quoted vocabulary composition, dated, exactly as + #326 quoted one. A date does not stop rot — #326 carried one too + — so before relying on the split, check the set still looks like + it: + + uv run python -c "from nameparser import Parser; print(sorted(Parser().lexicon.suffix_words))" + + What is durable here is the SHAPE of the finding — three kinds of + thing, the honorifics the largest — not the three integers. A + test asserting the counts is deliberately not the answer: that is + the constant-content pattern, and it would fail on every + legitimate vocabulary addition. ### deviates-registry — packs stay pure data (option C) From a1243d6d7bd2664339e88469a9fa41699bd1ef4e Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 18:52:31 -0700 Subject: [PATCH 6/9] docs(design): coherence-pass fixes, and scope the primary-source rule Two findings from a targeted pass over the contested pairs this branch creates. Prose reading found neither; naming the pairs and interrogating them one at a time found both. P6 and S2 contest "Berg, Jan vd" -- today S2 wins and reports suffix="vd", while P6's marker asserts family="vd Berg". Nothing said who wins. P6 does, and its statement now says so, which is how rules.md pins a contest (H2, M1 and W3 all carry precedence clauses in their statements; _CROSS_RULE_WINNERS is the ledger's instrument, not this document's). The precedence is scoped so it cannot creep: both particle and suffix vocabulary, trailing-orphan position, family comma, given word to spare. "John Smith, PhD" and "Smith, Jr." are not particles; "Jong, vd" has no given word left. R2's #385 resolution moves R3's output and R3 was silent about it. Initials read the BASE family word, so anchoring "Do" takes parse("Anh Do").initials() from "A." to "A. D."; R3 now carries its own deviates: marker, plus the "Juan van der" row where no borne name means no base and initials stay "J.". The general lesson is recorded with it: a marker lands on the rule whose STATEMENT changed, but a rule can move another rule's OUTPUT without touching its statement, and the runner cannot see it -- an unmarked downstream rule stays green because its own examples avoid the input. Walk `interacts:` when adding a marker. AGENTS.md's primary-source rule gains its missing scope. It was written for CROSS-SESSION distillation, where the source transcript is an artifact independent of the reviewing session -- #381 harvested nine such sessions. Same-session design work has no such artifact, and the instruction degenerates into re-reading the memory that produced the error. The amendment names both shapes and records the measured yield ordering from this branch: re-derived measurements 2, spec-vs-doc fidelity 1, named contested pairs 2, unaided prose reading 0. Not decided here: whether a rule with no implemented: pointer (P6 is the first) is a shape rules.md wants. Co-Authored-By: Claude Opus 5 --- AGENTS.md | 18 ++++++++++++++++++ docs/design/decisions.md | 33 +++++++++++++++++++++++++++++++++ docs/design/rules.md | 14 ++++++++++++-- 3 files changed, 63 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9a5d1065..9b127180 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -116,6 +116,24 @@ state which tree each measurement ran on (stale fetches produced three rounds of already-fixed findings), and landed corrections are re-verified here before committing. +Two shapes, needing different instruments. CROSS-SESSION distillation +is what the rule above is written for — the #381 arc harvested nine +named sessions, and for each the source transcript is an artifact +independent of the reviewing session, so "review your own sections" +is a two-artifact comparison. SAME-SESSION design work (#386) has no +such artifact: spec and docs were written in one context, so the same +instruction degenerates into re-reading the working memory that +produced any error, and the rationale inverts — when the source is +NOW, the source is what got it wrong. There, compare against +artifacts instead, in this measured order of yield: re-derive every +number with a freshly written script (2 wrong claims on #386); +re-read the written spec against a checklist for FIDELITY, not just +presence (1); interrogate NAMED contested rule pairs one at a time +(2). Unaided prose reading found none — so a general "coherence read" +by the author is the one form to distrust, because a green report +from the weakest instrument manufactures confidence rather than +supplying it. + **Guard tests** SHOULD carry a recorded negative control — the answer with the guard off, stored as data (the _EXCLUSION_EFFECT shape; see mechanisms.md's Verification shapes). diff --git a/docs/design/decisions.md b/docs/design/decisions.md index c06b7510..f4f445a9 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -237,6 +237,25 @@ not). as evidence about the corpus, not about the blast radius, and run tools/differential at all three baselines in the implementing PR regardless of how small this looks. +- 2026-08-16 (pre-merge coherence pass) — P6 and S2 CONTEST + "Berg, Jan vd": today S2 wins and reports suffix="vd", while + P6's marker asserts family="vd Berg". P6 wins, and the rule says + so in its statement rather than leaving the pair to file order. + The reason is C-ii's, not a new judgement: vd as the British + Volunteer Decoration is rarer than vd as van der, and a trailing + abbreviation AFTER A FAMILY COMMA is the tussenvoegsel position + specifically. + Scope check on the precedence, so it cannot creep: it reaches + only words that are both particle and suffix vocabulary, in the + trailing-orphan position, under a family comma, with a given word + to spare. "John Smith, PhD" and "Smith, Jr." are untouched (not + particles), and "Jong, vd" is untouched (no given word remains), + which is why that row stays out of scope rather than becoming a + counter-example. + Recorded because the pair was declared in `interacts:` and left + unresolved — `interacts:` is advisory by design and pins nothing, + so a declared interaction is a prompt to state the outcome, not a + substitute for stating it. Open: [#380](https://github.com/derek73/python-nameparser/issues/380) covers "Berg, Jan vd" under this rule, but the vd reading itself is @@ -1273,6 +1292,20 @@ Declined: - Still open inside the resolution: whether "Do" remains in family_particles once it is also the base. Recorded here rather than left to the implementing PR to decide by accident. +- 2026-08-16 (pre-merge coherence pass) — the resolution moves R3 + too, and R3 now carries its own marker. Initials read the BASE + family word, so anchoring "Do" changes + parse("Anh Do").initials() from "A." to "A. D." while + "Juan van der" stays "J." (no borne name, no base, and initials + of a bare particle run would be nonsense). + The general lesson, worth more than this instance: a deviates: + marker gets written on the rule whose STATEMENT changed, but a + rule can change another rule's OUTPUT without touching its + statement, and nothing looks for that — the runner asserts per + example line, so an unmarked downstream rule stays green + precisely because its own examples avoid the affected input. + When adding a marker, walk the changed rule's `interacts:` + targets and ask whether any of THEIR examples move. ### removed-v1-surface diff --git a/docs/design/rules.md b/docs/design/rules.md index 244bc3b2..e1e7229d 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -256,7 +256,11 @@ P6. Rationale: a particle ending the name has nothing to link ending the name attaches to that family name and is written before it — provided at least one given word remains, so that a name whose only given word is the particle keeps it (the - words-to-spare test S2 applies to ambiguous suffixes). + words-to-spare test S2 applies to ambiguous suffixes). Where + the word is BOTH a particle and suffix vocabulary, this + attachment outranks the suffix reading (S2): a trailing + abbreviation after a family comma is the tussenvoegsel far more + often than the decoration it collides with. "Jong, Anke de" → family="de Jong" deviates: #379 (today: family="Jong") "Beethoven, Ludwig van" → family="van Beethoven" deviates: #379 (today: family="Beethoven") "Berg, Jan vd" → family="vd Berg" deviates: #380 (today: family="Berg") @@ -758,8 +762,14 @@ R3. Rationale: initials abbreviate the person's name words; titles, family word; titles, suffixes, particles and nicknames contribute nothing. "Dr. Juan Q. Xavier de la Vega III" → initials="J. Q. X. V." + "Anh Do" → initials="A. D." deviates: #385 (today: initials="A.") "Sean O'Connor" → initials="S. O." · boundary - implemented: nameparser/_render.py + Accepted: a family that is ALL particles contributes nothing, + so the initials are the given words alone — "van der" has no + borne name to anchor a base (R2), and initials of a bare + particle run would be nonsense. + "Juan van der" → initials="J." + history: decisions.md#R2 · interacts: R2 · implemented: nameparser/_render.py R4. Rationale: case repair is a display concern, applied only on request and never destructively. From 1bc40399f0c76e2123ba608f91d7e6aaa502a442 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 19:09:56 -0700 Subject: [PATCH 7/9] docs(rules): correct P6's rationale, and give an unshipped rule a pointer Two review findings. P6's rationale claimed "no particle is a name by itself". That is true only of the never-given half, and decisions.md#vocabulary-collisions -- committed three commits earlier on this branch -- says the opposite in as many words: "most particles are short words that double as names (van, bin, le, do, bar, mac)". The error was substantive, not cosmetic: the words-to-spare guard exists BECAUSE the rule reaches ambiguous particles, #379's own subject being "van", so the rationale undercut its own guard. Rewritten so the guard follows from the reasoning: a never-given particle in trailing position cannot be a name and must join the family; an ambiguous one could be the name, which is what the guard is for. Noted with it: the coherence pass that ran two commits ago interrogated rule-vs-rule pairs and never checked rule-vs-decision-record, which is how this survived. P6 was also the first rule pointing at nothing -- no implemented:, because nothing implements it. Rather than allow that as a silent exception, the grammar grows `tracked: #N` as its counterpart, and every rule must carry exactly one of the two. An unimplemented rule can no longer sit in the doc untracked, and a shipped rule cannot keep a stale tracking pointer once its issues close. Documented in rules.md's preamble, where rules_doc.py says the grammar is described for humans. The new guard was mutation-tested before being believed, per the mechanisms.md field note: dropping the pointer, carrying both pointers, and malformed issue refs each fail it, and the restore was verified by diff rather than trusted. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 21 +++++++++++++++++++++ docs/design/rules.md | 22 +++++++++++++++------- tests/v2/rules_doc.py | 15 +++++++++++++-- tests/v2/test_rules_doc.py | 17 +++++++++++++++++ 4 files changed, 66 insertions(+), 9 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index f4f445a9..a144a955 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -256,6 +256,27 @@ not). unresolved — `interacts:` is advisory by design and pins nothing, so a declared interaction is a prompt to state the outcome, not a substitute for stating it. +- 2026-08-16 — P6 is the first rule in rules.md that nothing + implements. Legitimate per the preamble (the document is + normative, and a gap is a tracked deviation), but it left the + rule pointing at nothing, so the shape got a pointer rather than + an exception: `tracked: #379, #380` in place of `implemented:`, + with exactly one of the two required of every rule. An + unimplemented rule can no longer sit here untracked, and a + shipped rule cannot keep a stale tracking pointer after its + issues close. Mutation-tested three ways before being believed + (drop the pointer, carry both, malformed refs); each fails. +- The RATIONALE first shipped here was wrong and is corrected: it + read "no particle is a name by itself", which is true only of the + never-given half — decisions.md#vocabulary-collisions says the + opposite in as many words ("most particles are short words that + double as names"). The error mattered rather than merely reading + badly: the words-to-spare guard exists BECAUSE the rule reaches + ambiguous particles (#379's own subject is "van"), so the + rationale undercut its own guard. Caught in review, after a + coherence pass that interrogated rule-vs-rule pairs and never + checked rule-vs-decision-record — which is the gap to close next + time, not a one-off. Open: [#380](https://github.com/derek73/python-nameparser/issues/380) covers "Berg, Jan vd" under this rule, but the vd reading itself is diff --git a/docs/design/rules.md b/docs/design/rules.md index e1e7229d..e0f5eea9 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -22,7 +22,12 @@ decisions.md. Cross-references use the anchor form `decisions.md#P2` / `mechanisms.md#SPANS`; a bare ID is never a citation. The `interacts:` field on a pointer line is advisory — the citation-integrity test checks the ID exists, not that the -interaction is real. +interaction is real. `implemented:` and `tracked:` are not +advisory: every rule carries exactly one. `implemented:` names the +modules honoring the rule, checked against the modules that cite +it; `tracked:` names the issues that would ship a rule nothing +implements yet, so a wholly-aspirational rule cannot sit here +untracked, and a shipped rule cannot keep a stale tracking pointer. Every example line is EXECUTABLE. The grammar (its executable definition is `tests/v2/rules_doc.py`; `tests/v2/test_rules_doc.py` @@ -247,11 +252,14 @@ P5. Rationale: some given-name words are incomplete alone — "abdul" history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py P6. Rationale: a particle ending the name has nothing to link - forward to, and no particle is a name by itself — so it belongs - to the family name written beside it. Dutch and Flemish names - are listed exactly this way ("Beethoven, Ludwig van"), the - tussenvoegsel trailing the given name but belonging to the - surname. + forward to, so it is not doing a particle's work there. A + never-given particle in that position cannot be a name at all + and must belong to the family written beside it; an ambiguous + particle could genuinely be the name (Vietnamese "Van"), which + is what the words-to-spare test below is for, not an + afterthought to it. Dutch and Flemish names are listed exactly + this way ("Beethoven, Ludwig van"), the tussenvoegsel trailing + the given name but belonging to the surname. Where a family comma has already named the family, a particle ending the name attaches to that family name and is written before it — provided at least one given word remains, so that a @@ -271,7 +279,7 @@ P6. Rationale: a particle ending the name has nothing to link attachment is scoped to the comma form, and the comma-less shapes keep their positional reading. "Jong Anke de" → family="de" - history: decisions.md#P6 · interacts: C1, P1, S2 + history: decisions.md#P6 · interacts: C1, P1, S2 · tracked: #379, #380 ## Suffixes: generational & credentials (S) diff --git a/tests/v2/rules_doc.py b/tests/v2/rules_doc.py index 76c9e1d1..cf2781e0 100644 --- a/tests/v2/rules_doc.py +++ b/tests/v2/rules_doc.py @@ -8,6 +8,10 @@ plus per-rule ``no-boundary: reason`` lines and the trailing pointer line ``history: ... · interacts: A1, B2 · implemented: path, path``. +A rule nothing implements yet carries ``tracked: #N, #M`` in place of +``implemented:`` — the issues that would ship it. Exactly one of the +two is required, so a normative rule always points either at the code +that honors it or at the work that will. Inside a rule block, any line whose first non-space character is a double quote (or an opening bracket, the D-section subject form) is an @@ -35,8 +39,9 @@ rf"(?P[a-z_]+)=(?P{_VALUE})\))?" r"\s*$") _NO_BOUNDARY_RE = re.compile(r"^\s*no-boundary:\s+(?P\S.*)$") -_POINTER_RE = re.compile(r"^\s*(history|interacts|implemented):") -_POINTER_PART_RE = re.compile(r"(history|interacts|implemented):\s*([^·]+)") +_POINTER_RE = re.compile(r"^\s*(history|interacts|implemented|tracked):") +_POINTER_PART_RE = re.compile( + r"(history|interacts|implemented|tracked):\s*([^·]+)") ASSERTABLE_FIELDS = frozenset({ "title", "given", "middle", "family", "suffix", "nickname", "maiden", @@ -64,6 +69,10 @@ class Rule: no_boundary: str | None = None interacts: tuple[str, ...] = () implemented: tuple[str, ...] = () + #: Issues that would ship a rule nothing implements yet. Mutually + #: exclusive with ``implemented:`` -- a rule points at code or at + #: the issues that will produce it, never at neither. + tracked: tuple[str, ...] = () def has_boundary_or_waiver(self) -> bool: return self.no_boundary is not None or any( @@ -184,4 +193,6 @@ def parse_rules_doc(text: str) -> list[Rule]: current.interacts = items elif key == "implemented": current.implemented = items + elif key == "tracked": + current.tracked = items return rules diff --git a/tests/v2/test_rules_doc.py b/tests/v2/test_rules_doc.py index 48776256..579a6704 100644 --- a/tests/v2/test_rules_doc.py +++ b/tests/v2/test_rules_doc.py @@ -8,6 +8,7 @@ from __future__ import annotations import importlib.util +import re import pytest @@ -32,6 +33,22 @@ def test_every_rule_has_examples_and_boundary(rule: Rule) -> None: f"'no-boundary: '") +@pytest.mark.parametrize("rule", RULES, ids=lambda r: r.rule_id) +def test_every_rule_points_at_code_or_at_the_work(rule: Rule) -> None: + """A normative rule names the code honoring it, or the issues that + would ship it -- never neither, so an unimplemented rule cannot sit + in the doc untracked, and never both, so a shipped rule cannot keep + a stale tracking pointer once its issues close.""" + assert rule.implemented or rule.tracked, ( + f"{rule.rule_id}: add 'implemented: ' or, if nothing " + f"implements it yet, 'tracked: #N' naming the issues that would") + assert not (rule.implemented and rule.tracked), ( + f"{rule.rule_id}: has both implemented: and tracked:; drop " + f"tracked: once the rule ships") + bad = [t for t in rule.tracked if not re.fullmatch(r"#\d+", t)] + assert not bad, f"{rule.rule_id}: tracked: wants #N issue refs, got {bad}" + + def _check_diagnostic(example: Example) -> None: import re as _re from collections.abc import Callable From 1f5e0b8843073cb051de6345c81142dd2b91480f Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 19:14:49 -0700 Subject: [PATCH 8/9] docs(design): rule-vs-decision-record pass -- two more corrections Running the axis named in the previous commit, over the rules this branch touched and the decision entries they rest on. P1's statement said the fold takes "the particle and the ONE name word it attaches to" -- contradicted by its own example, since "de la Vega" is two particles onto one word. Now "the particle run", with a clause saying the run is every particle in sequence, never-given and ambiguous alike. decisions.md#P1's #364 entry carried the same wording and is corrected with it, as is the P1 excerpt quoted in _post_rules.py. P6's precedence over S2 is stated for a SHAPE ("where the word is both a particle and suffix vocabulary"), but decisions.md#vocabulary-collisions C-ii says precedence between two non-name readings is a per-word frequency judgement. Measured, three words are both: vd, do and mc. Only vd was weighed, so do and mc inherit its answer unexamined. Recorded on both sides rather than papered over -- stating a per-word judgement as a general clause is how an unexamined word acquires a decision, and naming the two lets the next reader see which was actually argued. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 18 +++++++++++++++--- docs/design/rules.md | 13 ++++++++++--- nameparser/_pipeline/_post_rules.py | 6 +++--- 3 files changed, 28 insertions(+), 9 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index a144a955..c7f532a1 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -104,9 +104,12 @@ the 2026-08-16 entries below. The survivor is the degenerate bare takes only the particle's own group (#364); the middle position needs no third site once grouping is order-independent (#365); and #368 reverses. -- 2026-08-16 #364 — the fold takes the particle and the ONE name - word it attaches to, not every remaining word. "de Mesnil Juan" - is family="de Mesnil" plus given="Juan". Nothing ever argued for +- 2026-08-16 #364 — the fold takes the particle RUN and the ONE + name word it attaches to, not every remaining word. + "de Mesnil Juan" is family="de Mesnil" plus given="Juan". Run, + not particle: "de la Vega" is two particles onto one word, and an + earlier wording here said "the particle", which its own example + contradicted (rule-vs-decision-record review). Nothing ever argued for "takes everything"; it was the shape of v1's handle_non_first_name_prefix, not a decision. Measured before deciding, over every name in the three @@ -391,6 +394,15 @@ quoting them: a borne given name, so both leave the ambiguous half); #342 (rai — Rai IS a borne surname, so it earns the marking rather than moving); #385 (do — resolved at decisions.md#R2). +- C-ii's per-word framing versus a rule stated for a SHAPE: measured + 2026-08-16, the words that are both particle and suffix + vocabulary are vd, do and mc — three, not the one this criterion + adjudicated. rules.md#P6 states its precedence for the shape, so + do and mc inherit vd's answer without being weighed. Recorded + rather than papered over: stating a per-word judgement as a + general clause is how an unexamined word acquires a decision, and + the two are named here so the next reader knows which one was + actually argued. - Caution when applying C-i to the particle set: TITLES ∩ ambiguous == {do, freiherr, st} is load-bearing, per the Excluded note in the W2 section. Emptying it makes the particle-or-given emitter diff --git a/docs/design/rules.md b/docs/design/rules.md index e0f5eea9..210d353d 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -168,9 +168,11 @@ P1. Rationale: a never-given particle standing alone cannot be one as the given name, is a surname written out in full. A never-given particle standing alone where the given name would go — or opening the name — marks the name as surname-only: the - particle and the one name word it attaches to are the family, - and any name words beyond that read by position. It needs - another name word to attach to. An ambiguous particle keeps + particle run and the one name word it attaches to are the + family, and any name words beyond that read by position. It + needs another name word to attach to. The run is every particle + in sequence, never-given and ambiguous alike ("de la Vega" is + one group, not "de" plus a separate "la Vega"). An ambiguous particle keeps whatever reading its position gives it. The reading holds under every declared name order: a never-given particle is evidence about how the name is written, and a declared order governs only @@ -279,6 +281,11 @@ P6. Rationale: a particle ending the name has nothing to link attachment is scoped to the comma form, and the comma-less shapes keep their positional reading. "Jong Anke de" → family="de" + Accepted: the precedence over S2 is stated for the shape, so it + sweeps in every word that is both particle and suffix + vocabulary — today vd, do and mc. Only vd's reading was + weighed; the other two inherit it, which is the shape's cost + and is tracked with the other contested memberships. history: decisions.md#P6 · interacts: C1, P1, S2 · tracked: #379, #380 ## Suffixes: generational & credentials (S) diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 5e470768..fdd98c78 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -95,9 +95,9 @@ def post_rules(state: ParseState) -> ParseState: # rules.md#P1: "a never-given particle standing alone where the # given name would go — or opening the name — marks the name as - # surname-only: the particle and the one name word it attaches - # to are the family, and any name words beyond that read by - # position." (v1 handle_non_first_name_prefix; history: + # surname-only: the particle run and the one name word it + # attaches to are the family, and any name words beyond that + # read by position." (v1 handle_non_first_name_prefix; history: # decisions.md#P1) # DEVIATION #364: the fold below still takes every remaining # name word, not just the particle's own; "de Mesnil Juan" gives From 6787631e24f36a9a69f6ea0d69ff00a5b5609b22 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Sun, 16 Aug 2026 19:19:48 -0700 Subject: [PATCH 9/9] test(rules-doc): close both /code-review findings The exactly-one-pointer guard added two commits ago claimed more than it enforced. Its docstring said a shipped rule "cannot keep a stale tracking pointer once its issues close", but the test reads the doc only and cannot see code. Meanwhile test_implemented_matches_citing_modules skipped any rule without implemented:, so a rule that gained code and a citation while keeping tracked: passed both. That is exactly the state the docstring called unrepresentable. The citations test now handles the tracked: branch, and the docstring states its own scope instead of the pair's: neither test alone makes a stale pointer unrepresentable, the pair does. Mutation-tested on the finding's own scenario -- adding a P6 citation to _post_rules.py while leaving tracked: in place now fails with the swap instruction, and the restore was verified by diff. The DEVIATION note on P1 wrote its example values in double quotes, which put four extra quoted spans inside the citation block that test_citations_are_verbatim_excerpts parses -- it passed only because the check takes the FIRST span. Moving the note above the citation line, a natural edit, would have made the excerpt resolve to "de Mesnil Juan" and failed against prose that is entirely correct. Values are unquoted now, and the note says why so the next editor does not helpfully re-add them. Considered and rejected: asserting exactly one quoted span per citation block, which would be the unrepresentable-states fix. Measured first -- 38 blocks carry one, but _group.py's M1 block legitimately carries six, so the invariant does not hold today and imposing it would mean rewriting an unrelated comment. Co-Authored-By: Claude Opus 5 --- nameparser/_pipeline/_post_rules.py | 12 ++++++++---- tests/v2/test_doc_citations.py | 12 +++++++++++- tests/v2/test_rules_doc.py | 11 +++++++++-- 3 files changed, 28 insertions(+), 7 deletions(-) diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index fdd98c78..906db723 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -99,10 +99,14 @@ def post_rules(state: ParseState) -> ParseState: # attaches to are the family, and any name words beyond that # read by position." (v1 handle_non_first_name_prefix; history: # decisions.md#P1) - # DEVIATION #364: the fold below still takes every remaining - # name word, not just the particle's own; "de Mesnil Juan" gives - # family="de Mesnil Juan" where the rule says family="de Mesnil" - # plus given="Juan". Pinned by the deviates: markers on P1. + # DEVIATION #364: the fold below still takes every remaining name + # word, not just the particle run's own -- de Mesnil Juan gives + # family=de Mesnil Juan where the rule says family=de Mesnil plus + # given=Juan. Pinned by the deviates: markers on P1. + # Values written unquoted deliberately: this note sits INSIDE the + # citation block above (# decisions.md#P1) does not close it -- + # _CITE_RE wants a colon after the ID), and the excerpt check + # takes the first quoted span in the block. # Code-local: a lone PIECE is the test at both sites, so a # particle group already chained forward is not a lone particle, # and rule H1 above cannot be what produces the fold's family diff --git a/tests/v2/test_doc_citations.py b/tests/v2/test_doc_citations.py index 227b14e7..7153c45d 100644 --- a/tests/v2/test_doc_citations.py +++ b/tests/v2/test_doc_citations.py @@ -102,13 +102,23 @@ def test_implemented_matches_citing_modules() -> None: citing.setdefault(cid, set()).add(str(path.relative_to(REPO))) problems = [] for rule in parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")): + actual = citing.get(rule.rule_id, set()) if rule.implemented: - actual = citing.get(rule.rule_id, set()) declared = set(rule.implemented) if actual != declared: problems.append( f"{rule.rule_id}: implemented: says {sorted(declared)} " f"but citations found in {sorted(actual)}") + elif rule.tracked and actual: + # The other half of test_rules_doc.py's exactly-one-pointer + # rule: that test cannot see code, so a rule that SHIPPED + # while keeping tracked: would pass it. Without this branch + # the stale pointer is invisible -- the loop above skips + # any rule with no implemented: at all. + problems.append( + f"{rule.rule_id}: declares tracked: {sorted(rule.tracked)} " + f"but code cites it in {sorted(actual)}; swap tracked: for " + f"implemented: now that something implements it") assert not problems, "\n".join(problems) diff --git a/tests/v2/test_rules_doc.py b/tests/v2/test_rules_doc.py index 579a6704..83bc151e 100644 --- a/tests/v2/test_rules_doc.py +++ b/tests/v2/test_rules_doc.py @@ -37,8 +37,15 @@ def test_every_rule_has_examples_and_boundary(rule: Rule) -> None: def test_every_rule_points_at_code_or_at_the_work(rule: Rule) -> None: """A normative rule names the code honoring it, or the issues that would ship it -- never neither, so an unimplemented rule cannot sit - in the doc untracked, and never both, so a shipped rule cannot keep - a stale tracking pointer once its issues close.""" + in the doc untracked, and never both. + + Scope, precisely: this test reads the DOC only, so it cannot tell + that a tracked: rule has since been implemented -- adding the code + and its citation while leaving tracked: in place passes here. That + half is test_doc_citations.py::test_implemented_matches_citing_ + modules, which sees the citing modules. Neither test alone makes a + stale pointer unrepresentable; the pair does. + """ assert rule.implemented or rule.tracked, ( f"{rule.rule_id}: add 'implemented: ' or, if nothing " f"implements it yet, 'tracked: #N' naming the issues that would")