You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(particles): a family-first order stops the leading run
A never-given particle opening the name took every remaining word into
the family in all three orders. Under a declared family-first order it
now takes one name word and leaves the rest to the order:
FAMILY_FIRST "de Mesnil Juan" -> family 'de Mesnil',
given 'Juan'
FAMILY_FIRST_GIVEN_LAST "de la Cruz Juan Carlos" -> middle 'Juan',
given 'Carlos'
Declaring a family-first order asserts that what follows the family is
not more surname, which is the question the stopping point asks. The
DEFAULT order is unchanged and that is the accepted cost: with nothing
declared, "de Mesnil Juan" has the shape of "pennie von bergen
wessels", whose whole text is the surname. Callers who mean otherwise
write the comma, which already parses that way.
Where it lives, and why not the two sites that failed before: the fold
in post_rules. Grouping was PR #394 and assignment PR #391 -- and the
piece is the obstacle, since "de la Cruz Juan Carlos" groups as [de]
[la Cruz Juan Carlos] once the ambiguous particle chains, so the stop
must cut INSIDE a piece. post_rules can: roles are per token, and
nothing downstream reads pieces (measured -- only _assign, which runs
before it). Grouping stays order-independent.
The order is read, not re-derived. assign records the order it used on
ParseState.order and the fold keys on that; policy.name_order would
disagree with the roles assign already wrote whenever a script_orders
entry overrides it.
The run counts UNITS: a conjunction join (P3) and a bound given-name
pair (P5) each count once, so "de la Vega y Santos Juan" cannot stop
between Vega and Santos, and "abdul Rahman" cannot be halved. Both are
read off the tags -- the prefix chain has already merged the joined
piece away by then.
Measured: one differential corpus name moves, "de Mesnil Garcia",
under each family-first order; all 751 are byte-identical in the
default order. The corpus cannot see more than that -- it runs under
the default policy against 1.4.0, which has no name_order -- so the
verification that counts is the two-leftover case rows, which
mutation-checking confirms are the only thing in the suite that fails
when name_order is discarded from the leftover placement.
Closes#395
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/design/decisions.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,6 +52,10 @@ the 2026-08-16 entries below. The survivor is the degenerate bare
52
52
Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions.md#O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried.
53
53
Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing.
54
54
Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand.
55
+
- 2026-08-18 #395 — WHERE THE STOP LIVES, settling what the entry above left open. Not grouping and not assignment: the fold in post_rules, which already retags tokens and is the only site that can express the stop without moving a piece boundary. Grouping was PR #394 (four regressions) and assignment was PR #391, where the finding was that assign cannot split an existing piece — and the piece is the problem: "de la Cruz Juan Carlos" groups as [de][la Cruz Juan Carlos], the ambiguous particle having chained forward, so any rule that stops "after the first name word" must cut INSIDE a piece. post_rules can, because roles are per token and nothing downstream reads pieces (measured: only _assign, which runs before it, and post_rules itself). Grouping therefore stays order-independent, which keeps the 2026-08-16 keystone intact everywhere except the reach of this one fold.
56
+
The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given.
57
+
A unit, not a token: the fold counts what other rules built, so a conjunction join (P3) and a bound given-name pair (P5) each count once. Both are read back off the TAGS rather than the pieces, because the prefix chain has already merged the joined piece into a longer one and the boundary is gone by then. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules.md#P3 was amended for on 2026-08-17, now executable.
58
+
Measured: ONE corpus name moves (de Mesnil Garcia), under each family-first order, none under the default. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is tests/v2/cases.py's two-leftover rows: mutation-checked, discarding name_order from the leftover placement fails exactly one of them and no other test in the suite.
Copy file name to clipboardExpand all lines: docs/release_log.rst
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,6 +32,8 @@ Release Log
32
32
33
33
- Fix a name opening with a particle that is *never* a given name being split at the particle under a family-first name order -- ``Policy(name_order=FAMILY_FIRST)`` and ``Policy(name_order=FAMILY_FIRST_GIVEN_LAST)`` alike, and identically: ``"de Mesnil"`` read as family ``de``, given ``Mesnil``, and ``"de la Vega"`` as family ``de``, given ``la Vega``. Each is now the whole surname, as it has always been in the default order. The rule enforcing it asked for the particle by the ``GIVEN`` role, which under a family-first order belongs to the token *after* the particle, so the test read the wrong word and declined. It now also asks by position -- the piece that opens the name -- so both shapes of the same rule are caught: where such a particle stands alone as a piece, either opening the name or in the given position, the name is left with no given name at all, the given and the middles folding into the family. Standing *alone* is the whole of it, and the rule claims nothing wider: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``, because there the particle chained onto the words after it rather than standing alone, and a bare ``"de"`` with nothing to fold into is still reported as the given name. The decision behind the fix: a word that can never be a given name leaves ``name_order`` nothing to decide, so declaring family-first is not a reason to make ``de`` a surname on its own. A leading particle that *may* be a given name is genuinely order-dependent and is untouched -- ``"van Gogh"`` still reads as family ``van``, given ``Gogh`` under both family-first orders. This is also what gives ``Lexicon.particles_ambiguous`` an effect outside the default order: taking a word out of it now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Seven of the 751 differential corpus names move, the same seven under each family-first order; default-order output is byte-identical over all 751, at the 1.4.0, 2.0.0 and 2.1.0 differential baselines alike (closes #359)
34
34
35
+
- Change how far a leading never-given particle takes the surname when a family-first ``name_order`` is declared. ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Juan"`` as family ``de Mesnil Juan`` -- the whole name -- and now reads family ``de Mesnil``, given ``Juan``. Declaring a family-first order asserts that what follows the family is not more surname, and where the surname run ends is exactly that question, so the declaration settles it. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and ``"de Mesnil Juan"`` has the same shape as ``"pennie von bergen wessels"``, a real name whose whole text is the surname. A caller who means family ``de la Vega`` plus given ``Juan`` in the default order writes the comma, which already parses that way. The run takes one name WORD rather than one token: a conjunction-joined run and a bound given-name pair each count once, so the stop cannot land inside one -- ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, and ``"ibn Awf abdul Rahman"`` reads given ``abdul Rahman``. Where two or more words are left over the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and middle ``Juan``, given ``Carlos`` under ``FAMILY_FIRST_GIVEN_LAST``. An ambiguous leading particle is untouched in every order -- ``"van Gogh Jan Pieter"`` still reads family ``van`` under both family-first orders -- and so is a family comma, where the comma has already fixed the surname (``"Smith, de Mesnil"`` keeps family ``Smith de Mesnil``). One of the 751 differential corpus names moves, ``"de Mesnil Garcia"`` to family ``de Mesnil``, given ``Garcia``, under each family-first order; default-order output is byte-identical over all 751. This reverses the answer #364 was closed on, and the reasoning is recorded at ``docs/design/decisions.md#P1`` (closes #395)
36
+
35
37
- Change the ``detail`` text of a ``PARTICLE_OR_GIVEN`` ambiguity to name the role the leading particle was actually given. It said "read as a given name" under every ``name_order``, which is false under ``Policy(name_order=FAMILY_FIRST)`` -- there ``"Van Johnson"`` reads as family ``Van``, given ``Johnson``, and the report described the reading not taken. It now ends "read as a family name" in that case, reading the role off the assigned token the way ``SUFFIX_OR_NAME`` already did -- that kind names both parts (``read as a family name rather than a post-nominal``), while this one names only the part it took. The ``kind`` is unchanged and stays ``PARTICLE_OR_GIVEN``: the fork really is particle-or-given, and only the human-readable text moved. Default-order output is identical (#355)
0 commit comments