From d3259811c62d88a409fa5f8ba79d768029ce9e0d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Mon, 17 Aug 2026 23:41:35 -0700 Subject: [PATCH 1/5] docs(decisions): a family-first order may move the particle stop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 2026-08-16 order-precedence keystone said no name_order moves a leading particle run's stopping point. Two implementation attempts (#390, closed unmerged) showed why that is wrong: declaring FAMILY_FIRST is precisely an assertion that what follows the family is not more surname, which is the same question the stopping point asks. The grouping/assignment split survives; only that one sentence is narrowed. Under the default order the run stays greedy, which makes "de Mesnil Juan" read like "pennie von bergen wessels" — the same shape, one real name. Also records the 3.0 reevaluation the reversal surfaced: every argument in the thread is a language judgement, and locale packs are where a caller supplies language instead of the parser guessing. Refs #395 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 0c3cee30..26ba14f4 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -46,6 +46,13 @@ the 2026-08-16 entries below. The survivor is the degenerate bare "van Mesnil Juan" gives given="van", middle="Mesnil", family="Juan". given="de" is the reading the vocabulary exists to forbid, and the fold is what prevents it. The all-orders agreement in P1 is deliberate and is W4's shape: a wholly-hangul name reads family="김" under every declared order because the script carries a signal the order does not override, and a leading never-given particle is the Latin-script analogue. decisions.md#O4 already draws the line — "Words no vocabulary has claimed read by position" — so name_order governs the unclaimed remainder, which is most inputs. +- 2026-08-17 — SUPERSEDES the order-independence half of the 2026-08-16 order-precedence keystone entry above (#364, #365, #368). That entry says "no name_order moves that stopping point"; it may. Declaring FAMILY_FIRST or FAMILY_FIRST_GIVEN_LAST is precisely an assertion that what follows the family is NOT more surname, and a particle run's stopping point is exactly the question of where surname material ends — so the declaration is evidence about it, not merely about which slot the result lands in. The grouping/assignment split survives everywhere else; what is narrowed is the claim that the stopping point is one of the things order cannot touch. + Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name, which makes "de Mesnil Juan" read exactly like "pennie von bergen wessels" — the same shape, and this session decided the greedy reading is right for that one. Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle. + Accepted, and it is the cost that decided #364 three separate ways before this: "de la Vega Juan" under the DEFAULT order stays wholly a surname. A caller who means family "de la Vega" plus given "Juan" writes the comma. The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language. + Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions.md#O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried. + Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing. + Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand. + Declined: - A strict xfail asserting "de Mesnil" → family under FAMILY_FIRST (#359 review) — #359 deliberately left those semantics open, and a strict xfail decides the question by the back door. @@ -536,3 +543,4 @@ Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion appro - (A) Parking lot from the 2026-07-06 config-model discussion, 3.0-shaped by design: scoped config override via contextvars (`with nameparser.config.use(c):`, the decimal.localcontext pattern — fixes test pollution and config wars while keeping the shared default), and a requests-Session-style Parser(constants) API (rejected then as a full rewrite; the shared CONSTANTS survives 2.x because it is the only config channel that reaches parses inside code users don't own — #262's contract framing). - (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it. - (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script. +- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. From a5ccd13d291b96dc0ae249fd9d970402531ee817 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Mon, 17 Aug 2026 23:41:44 -0700 Subject: [PATCH 2/5] docs(rules): a conjunction-joined run is one name word MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P3 already says a connective binds name words into one part; what it did not say is how OTHER rules count that part. They count it as one. The case that forced the question is #395, where P1's run must stop "after the first name word" and would otherwise be free to stop INSIDE a part P3 built, splitting "de la Vega y Santos Juan" between "Vega" and "Santos". The clause describes shipped behavior, not an aspiration: H1 counts this way today, so the examples are the H1 pair — "Mr. Jack and Jill" -> family="Jack and Jill" "Mr. Jack Jill" -> given="Jack" (boundary) decisions.md#P3 records the clause and how it was nearly shipped as an unexaminable one: the first draft asserted no rule could show it and carried an Accepted note saying so, having reasoned about which rules count rather than parsing a name. Refs #395 Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 1 + docs/design/rules.md | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 26ba14f4..a7cbbd31 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -197,6 +197,7 @@ Open: [#348](https://github.com/derek73/python-nameparser/issues/348) applying C ### P3 — connectives - 2026-07-30 #267 — the four-word single-letter asymmetry: a bare Latin capital connective is vetoed as an initial while a Cyrillic capital joins. #267's closure ("v2.0 behaves this way by default … verify it was the right call") showed only the Cyrillic half; the Latin-capital veto was never separately adjudicated, which rules.md#P3 records as an Accepted consequence. +- 2026-08-17 — a conjunction-joined run counts as ONE name word in every rule that counts them. Raised while bounding the leading particle run (#395): a run that stops "after the first name word" would otherwise be able to stop INSIDE a part P3 built, splitting "de la Vega y Santos Juan" between "Vega" and "Santos". This is not new behavior — H1 already counts this way, measured: "Mr. Jack and Jill" gives family="Jack and Jill" while "Mr. Jack Jill" gives given="Jack", family="Jill". What is new is that rules.md says so, which is what binds #395's implementation. Recorded because the first draft of the clause asserted that no rule could show it today and shipped an Accepted note to that effect; the shipped H1 pair was simply never run. Detector failure of the usual shape — reasoning about which rules count instead of parsing a name and looking. Open: [#383](https://github.com/derek73/python-nameparser/issues/383) diff --git a/docs/design/rules.md b/docs/design/rules.md index fdd8e31d..0a749418 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -150,16 +150,25 @@ P3. Rationale: connective words ("y", "of the") bind name words into connective runs included — except a single-letter connective in a three-word name, which stays a name word, and a single-letter connective written as a bare Latin capital, which reads as an - initial and never joins. + initial and never joins. The joined part is ONE name word + wherever another rule counts them, so a rule taking "one name + word" takes the whole join and never half of it. "Juan y Eva Garcia" → given="Juan y Eva" "Jose E Maria Santos" → middle="E Maria" "Juan y Garcia" → middle="y" · boundary + "Mr. Jack and Jill" → family="Jack and Jill" + "Mr. Jack Jill" → given="Jack" · boundary Accepted: the initial veto is a LATIN shape — a Cyrillic capital joins ("И".isupper() is true, so this is not a Unicode-uppercase rule); #267's closure blessed the Cyrillic side, and whether the Latin-capital half should stand is #383. "Хосе И Мария Сантос" → given="Хосе И Мария" - history: decisions.md#P3 · implemented: nameparser/_pipeline/_group.py + H1 is the counting rule that shows the one-word clause today: a + title plus the join reads the whole join as the family, where the + same two words unjoined are two name words and H1 does not fire. + P1's leading run becomes the second once #395 lands — its run + must take the "Vega y Santos" join whole or stop before it. + history: decisions.md#P3 · interacts: H1, P1 · implemented: nameparser/_pipeline/_group.py P4. Rationale: a particle links forward from inside a name; at the very front there is no name yet to be inside. From 499c4026d984ee457e8fca20b802fbefd5dbb9be Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Tue, 18 Aug 2026 00:16:24 -0700 Subject: [PATCH 3/5] docs(rules): P3's examples were all single-letter conjunctions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every P3 example used y, E or И — the carve-out cases — so the ordinary multi-letter join was never shown, and the boundary line ("Juan y Garcia" keeps y a name word) read as if three-word names never join at all. Two contrast examples fix it: the same shape with "and" joins, and so does "&", which is single-character but not a letter. Measured, the predicate is len(text) == 1 and total < 4 and text.isalpha(). The prose now says the carve-out reaches every single-letter connective the vocabulary holds — Cyrillic и/і/й and Arabic و, not just the two Latin ones — while the capital veto reaches only the Latin ones, which is the asymmetry #383 is open about. decisions.md#3-0-reevaluations gets the locale consequence: Spanish, Portuguese and Dutch each want a different subset of single letters joined, so a locale pack would have to carry the join threshold and the initial veto as per-word policy. That is a stronger requirement than yesterday's particle entry, which only asked packs to override vocabulary membership. Co-Authored-By: Claude Opus 5 --- docs/design/decisions.md | 1 + docs/design/rules.md | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index a7cbbd31..f4aaeb50 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -545,3 +545,4 @@ Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion appro - (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it. - (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script. - (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one. +- (A) 2026-08-18 — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT, and the sharper requirement it puts on the entry above: a pack would have to carry per-word POLICY, not just membership. Which single letters a tradition wants joined differs by language — Spanish writes "y" and would want "e" read as an initial, Portuguese wants the reverse, Dutch would want every single letter read as an initial. One global rule serves all three today: a single ALPHABETIC character in a name of fewer than four pieces stays a name word, and a bare Latin capital never joins (#383). It is v1's Google Code issue 11 fix ("john e smith") generalized to a vocabulary v1 did not have — the set now holds six single-letter conjunctions (y, e, и, і, й, و), and the carve-out reaches all of them because it counts characters, while the capital veto reaches only the Latin ones because it tests a Latin shape. That asymmetry is the visible seam and it is why #383 is open. A pack that could only ADD or REPLACE vocabulary could not express any of it; the join threshold and the initial veto would have to be per-word, per-pack values. Measured while documenting P3: "&" is single-character but not a letter, so it joins at any length ("Juan & Garcia" is one part while "Juan y Garcia" is three). diff --git a/docs/design/rules.md b/docs/design/rules.md index 0a749418..ebdf1ea1 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -156,8 +156,16 @@ P3. Rationale: connective words ("y", "of the") bind name words into "Juan y Eva Garcia" → given="Juan y Eva" "Jose E Maria Santos" → middle="E Maria" "Juan y Garcia" → middle="y" · boundary + "Juan and Garcia" → given="Juan and Garcia" + "Juan & Garcia" → given="Juan & Garcia" "Mr. Jack and Jill" → family="Jack and Jill" "Mr. Jack Jill" → given="Jack" · boundary + Both exceptions are about the written FORM, not the word: the + three-word carve-out counts letters, so a symbol connective joins + at any length, and it reaches every single-letter connective the + vocabulary holds — Cyrillic и/і/й and Arabic و as well as y and + e. Which single letters a tradition actually wants joined differs + by language, and no locale gets its own answer today. Accepted: the initial veto is a LATIN shape — a Cyrillic capital joins ("И".isupper() is true, so this is not a Unicode-uppercase rule); #267's closure blessed the Cyrillic From 491396d892cb271affb7449f561efabf3edce834 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Tue, 18 Aug 2026 00:22:18 -0700 Subject: [PATCH 4/5] docs(rules): say what a boundary example is, and unmark a contrast MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Juan y Garcia" reads oddly as a boundary — y is inside P3's carve-out, not outside the rule. Surveying all 39 markers shows that is the convention rather than a slip: H1's boundary is "Sir John" (its given-name-title exception), H2's is "J. Smith" (bare initials, which its statement excludes), H3's is a title word mid-name. The marker means the rule's effect does not occur, and the informative case is almost always the rule's own exception. The preamble now says so. What was genuinely wrong is that P3 carried two markers meaning different things. "Mr. Jack Jill" is the contrast half of the H1 one-word pair — no connective is present at all — so it is not a limit of P3 and no longer claims to be. Co-Authored-By: Claude Opus 5 --- docs/design/rules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/rules.md b/docs/design/rules.md index ebdf1ea1..9d964e38 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -11,7 +11,7 @@ Every example line is EXECUTABLE. The grammar (its executable definition is `tes "INPUT" [annotation] → field="value" [· boundary] [deviates: #N (today: field="value")] -An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry — or the rule declares `no-boundary: ` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep deviates:` on this file is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions.md). +An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry: an input shaped like the rule's subject where its effect does NOT occur. That is usually the rule's OWN stated exception — H1's given-name title, P3's single-letter carve-out — not an input the rule never reaches, so the exception is executable rather than merely asserted. Or the rule declares `no-boundary: ` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep deviates:` on this file is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions.md). ## Not in scope @@ -159,7 +159,7 @@ P3. Rationale: connective words ("y", "of the") bind name words into "Juan and Garcia" → given="Juan and Garcia" "Juan & Garcia" → given="Juan & Garcia" "Mr. Jack and Jill" → family="Jack and Jill" - "Mr. Jack Jill" → given="Jack" · boundary + "Mr. Jack Jill" → given="Jack" Both exceptions are about the written FORM, not the word: the three-word carve-out counts letters, so a symbol connective joins at any length, and it reaches every single-letter connective the From 89f31f811ff52be817013532864136a104db9b3c Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Tue, 18 Aug 2026 00:35:07 -0700 Subject: [PATCH 5/5] docs(group): cite P3 where the carve-out actually is _group.py cites rules.md#P3 on _is_conj_piece, which is the joining predicate; the single-letter carve-out lives in the merge loop of a different function and carried only a v1 pointer ("Google Code issue 11"), so a reader there had no route to the rule. Records what the code says and the rule does not: the threshold counts ROOTNAMES, so a conjunction that is also suffix vocabulary is not counted and raises its own bar by one name word. #397 measures that on "i" (the Roman numeral), where it is the difference between fixing Catalan's common shape and not. Refs #397 Co-Authored-By: Claude Opus 5 --- nameparser/_pipeline/_group.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/nameparser/_pipeline/_group.py b/nameparser/_pipeline/_group.py index eda3b39d..399c87e5 100644 --- a/nameparser/_pipeline/_group.py +++ b/nameparser/_pipeline/_group.py @@ -187,10 +187,12 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(), merge(k, k + 2, add={"conjunction"}) else: k += 1 - # each conjunction joins its neighbors (v1's Google Code issue 11 - # carve-out, the "john e smith" bug: - # a single-letter alphabetic conjunction in a short name is more - # likely an initial) + # each conjunction joins its neighbors, rules.md#P3: "except a + # single-letter connective in a three-word name, which stays a + # name word" (v1's Google Code issue 11 carve-out, the + # "john e smith" bug). The threshold reads the ROOTNAME count, + # so a conjunction that is also suffix vocabulary raises the + # bar for itself -- #397 measures that on "i". k = 0 while k < len(pieces): if not conj(k):