Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,15 @@ the 2026-08-16 entries below. The survivor is the degenerate bare
family="Juan". given="de" is the reading the vocabulary exists to forbid, and the fold is what prevents it. The all-orders agreement in P1 is deliberate and is W4's shape: a wholly-hangul name reads family="김" under every declared order because the script carries a signal the order does not override, and a leading never-given particle is the Latin-script analogue. decisions.md#O4 already draws the line — "Words no vocabulary has claimed read by position" — so name_order governs the unclaimed remainder, which is most inputs.

- 2026-08-17 — SUPERSEDES the order-independence half of the 2026-08-16 order-precedence keystone entry above (#364, #365, #368). That entry says "no name_order moves that stopping point"; it may. Declaring FAMILY_FIRST or FAMILY_FIRST_GIVEN_LAST is precisely an assertion that what follows the family is NOT more surname, and a particle run's stopping point is exactly the question of where surname material ends — so the declaration is evidence about it, not merely about which slot the result lands in. The grouping/assignment split survives everywhere else; what is narrowed is the claim that the stopping point is one of the things order cannot touch.
Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name, which makes "de Mesnil Juan" read exactly like "pennie von bergen wessels" — the same shape, and this session decided the greedy reading is right for that one. Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle.
Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name. (This entry originally said that makes "de Mesnil Juan" read "exactly like pennie von bergen wessels — the same shape". Measured while implementing #395, that is false and is corrected here rather than left standing: pennie is the GIVEN name there, von is ambiguous so this fold can never fire on it, and the surname reading comes from P2's mid-name chain. What the real name does support is the weaker claim the argument needs — that a particle followed by several words can be all surname, "von Bergen Wessels" being one.) Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle.
Accepted, and it is the cost that decided #364 three separate ways before this: "de la Vega Juan" under the DEFAULT order stays wholly a surname. A caller who means family "de la Vega" plus given "Juan" writes the comma. The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language.
Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions.md#O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried.
Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing.
Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand.
- 2026-08-18 #395 — WHERE THE STOP LIVES, settling what the entry above left open. Not grouping and not assignment: the fold in post_rules, which already retags tokens and is the only site that can express the stop without moving a piece boundary. Grouping was PR #394 (four regressions) and assignment was PR #391, where the finding was that assign cannot split an existing piece — and the piece is the problem: "de la Cruz Juan Carlos" groups as [de][la Cruz Juan Carlos], the ambiguous particle having chained forward, so any rule that stops "after the first name word" must cut INSIDE a piece. post_rules can, because roles are per token and nothing downstream reads pieces (measured: only _assign, which runs before it, and post_rules itself). Grouping therefore stays order-independent, which keeps the 2026-08-16 keystone intact everywhere except the reach of this one fold.
The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given.
A unit, not a token: a particle chain (P2), a conjunction join (P3) and a bound given-name pair (P5) each count once. All three are read off the TAGS rather than the pieces, but for two different reasons, and the difference matters to anyone who tries to simplify this. The conjunction join grouping DID build and the prefix chain then swallowed: the name reaches the fold as [de][la Vega y Santos Juan], the ambiguous particle having chained forward over the join, so the JOIN's boundary is gone. The leading particle keeps its own piece, and must — the fold's site test wants a lone piece, so a one-piece name would not fold at all. The bound-given join grouping never built at all: P5 joins only where the bound word is the first non-title piece, and at a fold site the first piece is the particle, so "ibn Awf abdul Rahman" arrives as four separate pieces. Restoring piece boundaries in group would fix the first and silently split the second. It also makes P5's new implemented: pointer stronger than a citation — this is the only place that join happens in this shape. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules.md#P3 was amended for on 2026-08-17, now executable.
Measured, with the denominator this log requires: of 751 corpus names, SEVEN reach P1's leading fold (DE MESNIL, De Groot, de Mesnil, de Mesnil Garcia, de Mesnil Jr., de la Vega, dos Santos) and exactly ONE of those has anything past the run to lose, so one moves (de Mesnil Garcia) under each family-first order and none under the default. The single mover is a fact about the corpus holding no family-first listing with leftovers, not about the blast radius. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is the two-leftover shape: mutation-checked, discarding name_order from the leftover placement fails three tests across three layers — the FAMILY_FIRST_GIVEN_LAST case row (one of the two, the FAMILY_FIRST row being blind to it by construction), test_post_rules.py's test_the_two_family_first_orders_differ_at_two_leftovers, and rules.md's P1 example for that order. An earlier draft of this sentence said "exactly one, and no other test in the suite": true when measured, false one commit later, because two of the three guards were added after the measurement and it was not re-run. Recording the failure mode as well as the number, since it is this log's recurring one.

Declined:

Expand Down
58 changes: 45 additions & 13 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,35 +95,63 @@ P1. Rationale: a never-given particle standing alone cannot be
one as the given name, is a surname written out in full.
A never-given particle standing alone where the given name would
go — or opening the name — marks the name as surname-only: the
particle run and the one name word it attaches to are the
family, and any name words beyond that read by position. It
particle run and the name words it attaches to are the family. It
needs another name word to attach to. The run is every particle
in sequence, never-given and ambiguous alike ("de la Vega" is
one group, not "de" plus a separate "la Vega"). An ambiguous particle keeps
whatever reading its position gives it. The reading holds under
every declared name order: a never-given particle is evidence
about how the name is written, and a declared order governs only
what no vocabulary has claimed (O4) — the same precedence the
script license takes in W4.
whatever reading its position gives it. That the particle claims
the FAMILY rather than a given name holds under every order: a
never-given particle is evidence about how the name is written,
and a declared order governs only what no vocabulary has claimed
(O4) — the same precedence the script license takes in W4.

How MANY name words it attaches to depends on the order, and on
which of the two positions above the particle stands in. Opening
the name, under a family-first order, it takes exactly ONE —
declaring that order asserts that what follows the family is not
more surname. Opening the name under the default order, or
standing in the given position under any order, it takes the rest
of the name: nothing there marks where the surname ends. One name
word means one UNIT — a particle chain (P2), a conjunction join
(P3) or a bound given-name pair (P5) is taken whole or not at
all. A title does not move the opening position (P4), but a
family comma does end the question: the comma has already fixed
the surname, so there is no positional read left for an order to
narrow, and a particle opening the part AFTER it takes the rest
of that part whatever order is declared. What is left over is not read by O4's rule for a whole name,
which would make the first leftover a second family name; it is
laid out as the positions AFTER the family in the declared order,
the family slot being already filled.
"de la Vega" → family="de la Vega"
"Sir de Mesnil" → family="de Mesnil"
"Mesnil de" family-first → family="Mesnil de"
"de Mesnil Juan" → family="de Mesnil" deviates: #364 (today: family="de Mesnil Juan")
"de Mesnil Juan" → given="Juan" deviates: #364 (today: given="")
"de Mesnil Juan" → family="de Mesnil Juan"
"de Mesnil Juan" family-first → family="de Mesnil"
"de Mesnil Juan" family-first → given="Juan"
"Smith, de Mesnil Juan" family-first → family="Smith de Mesnil Juan"
"de la Vega y Santos Juan" family-first → family="de la Vega y Santos"
"ibn Awf abdul Rahman" family-first → given="abdul Rahman"
"de la Cruz Juan Carlos" family-first-given-last → given="Carlos"
"Mc Donald" → family="Mc Donald"
"de los Santos" → family="de los Santos"
"van Gogh" → given="van" · boundary
Accepted: a bare "de" stays the given name — there is nothing to
fold into, and inventing a surname would be worse.
"de" → given="de"
Accepted: stopping the run leaves a MIDDLE where the fold never
left one before, so O3 has something to claim that it could not
reach until now. The family it then reports is discontiguous in
the input — words 1-3 plus word 5 — and renders the folded word
first, which is R1's order, not this rule's doing.
"de la Cruz Juan Carlos" family-first+middle_as_family → family="Carlos de la Cruz"
Accepted: only the OPENING position is this rule's subject. A
particle chain standing inside the name is grouped normally (P2)
and positioned by the declared order, so a family-first reading
may report it as the given name; what the vocabulary forbids is
the bare particle reading as a given name, not any name part
that begins with one.
"Juan de la Vega" family-first → family="Juan"
history: decisions.md#P1 · interacts: P2, P4, P6 · implemented: nameparser/_pipeline/_post_rules.py
history: decisions.md#P1 · interacts: O3, O4, P2, P3, P4, P5, P6 · implemented: nameparser/_pipeline/_post_rules.py

P2. Rationale: a particle is written as part of the surname it
precedes, and a title stands outside the name entirely.
Expand All @@ -132,16 +160,20 @@ P2. Rationale: a particle is written as part of the surname it
or the name ends. The final group reads as the family name;
earlier groups read by position. The chain begins wherever the
name begins, and a preceding title does not move that point.
Where P1's fold has claimed the opening, the fold decides the
family instead — and may take only PART of the final group,
since it counts name words and the group is one part.
"John van der Berg" → family="van der Berg"
"John van der Berg Smith" → family="van der Berg Smith"
"Vincent van Gogh van Beethoven" → middle="van Gogh"
"Dr. John van der Berg" → family="van der Berg"
"Juan de" → family="de" · boundary
"de la Cruz Juan Carlos" family-first → family="de la Cruz"
Accepted: a caller wanting the combined double-surname reading
(#132's ask) has it as the surnames view rather than the
family field.
"Vincent van Gogh van Beethoven" → surnames="van Gogh van Beethoven"
history: decisions.md#P2 · implemented: nameparser/_pipeline/_group.py
history: decisions.md#P2 · interacts: P1, P4 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py

P3. Rationale: connective words ("y", "of the") bind name words into
one name part; but a single letter in a short name is more
Expand Down Expand Up @@ -176,7 +208,7 @@ P3. Rationale: connective words ("y", "of the") bind name words into
same two words unjoined are two name words and H1 does not fire.
P1's leading run becomes the second once #395 lands — its run
must take the "Vega y Santos" join whole or stop before it.
history: decisions.md#P3 · interacts: H1, P1 · implemented: nameparser/_pipeline/_group.py
history: decisions.md#P3 · interacts: H1, P1 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py

P4. Rationale: a particle links forward from inside a name; at the
very front there is no name yet to be inside.
Expand All @@ -197,7 +229,7 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
one given name.
"abdul salam ahmed salem" → given="abdul salam"
"mohamad ali smith" → given="mohamad" · boundary
history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py
history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py, nameparser/_pipeline/_post_rules.py

P6. Rationale: a particle ending the name has nothing to link
forward to, so it is not doing a particle's work there. A
Expand Down
Loading