Skip to content
Merged
48 changes: 48 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,36 @@ not deferred (a 2026-08-16 sweep of eight weeks of specs recovered
nine such items). The same-PR amendment rule above covers code-driven
changes; this covers the design-driven ones.

**Triaging a design backlog: shape before value.** When a pile of
open questions has to be ordered, relatedness will not partition it —
in a parser nearly everything touches particles or suffixes. The line
that does is whether a decision changes WHAT THE MODEL CAN EXPRESS or
fills in a value in a shape already fixed. Settle the shape questions
first, as a batch: they are few, they are usually independent of each
other (so their own ordering does not matter), and each one collapses
or re-frames a run of the value questions below it. The rest are
leaves needing one measurement and one answer apiece, in any order.
Worked example, 2026-08-16 (#386): 30 open issues and ~13 open design
questions reduced to three shape decisions, which decided or
re-framed fourteen of them — and two of the three turned out smaller
than their issues claimed, because the issues had gone stale. Check
the age of an issue's premises before believing its cost estimate.

**A count in a dated entry is evidence, not a live fact.** decisions.md
entries are snapshots by convention, so measurements belong in them —
but a reader wanting TODAY's number must not have to trust the
snapshot's date. Where an entry quotes something that drifts
(vocabulary sizes, corpus counts, set compositions), give the
one-liner that recomputes it, and phrase the argument so it survives
the digits moving — "the two shares differ by orders of magnitude"
outlives "58% vs 0.65%". A count that carries no argument is better
deleted than dated: "over every name in the corpus, no prefilter"
says what "over all 782 names" says, and cannot go stale. Do NOT
reach for a test asserting the count — that is the constant-content
pattern, and it fails on every legitimate vocabulary addition. #326
is the cautionary case: it quoted a vocabulary composition, carried a
date, and was stale in five days.

**Primary-source review.** When doc content is distilled from a
session's work, have that session (or its transcript) review its
own sections before or soon after landing — attribution flattening
Expand All @@ -86,6 +116,24 @@ state which tree each measurement ran on (stale fetches produced
three rounds of already-fixed findings), and landed corrections are
re-verified here before committing.

Two shapes, needing different instruments. CROSS-SESSION distillation
is what the rule above is written for — the #381 arc harvested nine
named sessions, and for each the source transcript is an artifact
independent of the reviewing session, so "review your own sections"
is a two-artifact comparison. SAME-SESSION design work (#386) has no
such artifact: spec and docs were written in one context, so the same
instruction degenerates into re-reading the working memory that
produced any error, and the rationale inverts — when the source is
NOW, the source is what got it wrong. There, compare against
artifacts instead, in this measured order of yield: re-derive every
number with a freshly written script (2 wrong claims on #386);
re-read the written spec against a checklist for FIDELITY, not just
presence (1); interrogate NAMED contested rule pairs one at a time
(2). Unaided prose reading found none — so a general "coherence read"
by the author is the one form to distrust, because a green report
from the weakest instrument manufactures confidence rather than
supplying it.

**Guard tests** SHOULD carry a recorded negative control — the
answer with the guard off, stored as data (the _EXCLUSION_EFFECT
shape; see mechanisms.md's Verification shapes).
Expand Down
335 changes: 314 additions & 21 deletions docs/design/decisions.md

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,21 @@ decline, not delete the test.
down, and an unrecognized word is by definition outside the
vocabulary — a green run over the corpus proves nothing about
such a rule.
- The corpus can be near-blind to a WRITING CONVENTION even where
the vocabulary is well covered, and a small honest count then
reads as a small blast radius. Measured for rules.md#P6: 245 of
782 corpus names carry a comma and exactly ONE ends in a
particle, so the Dutch trailing-tussenvoegsel listing the rule
exists for is essentially unsampled. Before reporting "N names
move", report the size of the population that COULD move; when
that is ~1, the number is evidence about the corpus.
- A detector that re-implements a rule's grouping will get the
grouping wrong. Derive the boundary from the same vocabulary the
rule reads, not from the half you happen to be thinking about:
walking a particle run over the NEVER-GIVEN set alone (the rule
chains through ANY particle) split "de la Vega" after "de la"
and reported 50 false movers for #364, where the true count is
one. Both wrong answers were plausible and printed cleanly.
- Guard the whole family, parametrize over it: a defect on one of
N parallel entry points hides behind a per-example test — three
times in one session (a guard on one class of two, a decode hint
Expand Down
87 changes: 75 additions & 12 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@ decisions.md. Cross-references use the anchor form `decisions.md#P2`
/ `mechanisms.md#SPANS`; a bare ID is never a citation. The
`interacts:` field on a pointer line is advisory — the
citation-integrity test checks the ID exists, not that the
interaction is real.
interaction is real. `implemented:` and `tracked:` are not
advisory: every rule carries exactly one. `implemented:` names the
modules honoring the rule, checked against the modules that cite
it; `tracked:` names the issues that would ship a rule nothing
implements yet, so a wholly-aspirational rule cannot sit here
untracked, and a shipped rule cannot keep a stale tracking pointer.

Every example line is EXECUTABLE. The grammar (its executable
definition is `tests/v2/rules_doc.py`; `tests/v2/test_rules_doc.py`
Expand Down Expand Up @@ -163,19 +168,33 @@ P1. Rationale: a never-given particle standing alone cannot be
one as the given name, is a surname written out in full.
A never-given particle standing alone where the given name would
go — or opening the name — marks the name as surname-only: the
given and middle words fold into the family. It needs another
name word to fold into. An ambiguous particle keeps whatever
reading its position gives it. Whether the fold should stop at
the particle group instead of taking everything is open (#364).
particle run and the one name word it attaches to are the
family, and any name words beyond that read by position. It
needs another name word to attach to. The run is every particle
in sequence, never-given and ambiguous alike ("de la Vega" is
one group, not "de" plus a separate "la Vega"). An ambiguous particle keeps
whatever reading its position gives it. The reading holds under
every declared name order: a never-given particle is evidence
about how the name is written, and a declared order governs only
what no vocabulary has claimed (O4) — the same precedence the
script license takes in W4.
"de la Vega" → family="de la Vega"
"Sir de Mesnil" → family="de Mesnil"
"Mesnil de" family-first → family="Mesnil de"
"Juan de la Vega" family-first → family="de la Vega" deviates: #368 (today: family="Juan")
"de Mesnil Juan" → family="de Mesnil" deviates: #364 (today: family="de Mesnil Juan")
"de Mesnil Juan" → given="Juan" deviates: #364 (today: given="")
"van Gogh" → given="van" · boundary
Accepted: a bare "de" stays the given name — there is nothing to
fold into, and inventing a surname would be worse.
"de" → given="de"
history: decisions.md#P1 · interacts: P2 · implemented: nameparser/_pipeline/_post_rules.py
Accepted: only the OPENING position is this rule's subject. A
particle chain standing inside the name is grouped normally (P2)
and positioned by the declared order, so a family-first reading
may report it as the given name; what the vocabulary forbids is
the bare particle reading as a given name, not any name part
that begins with one.
"Juan de la Vega" family-first → family="Juan"
history: decisions.md#P1 · interacts: P2, P4, P6 · implemented: nameparser/_pipeline/_post_rules.py

P2. Rationale: a particle is written as part of the surname it
precedes, and a title stands outside the name entirely.
Expand Down Expand Up @@ -234,6 +253,41 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
"mohamad ali smith" → given="mohamad" · boundary
history: decisions.md#P5 · implemented: nameparser/_pipeline/_group.py

P6. Rationale: a particle ending the name has nothing to link
forward to, so it is not doing a particle's work there. A
never-given particle in that position cannot be a name at all
and must belong to the family written beside it; an ambiguous
particle could genuinely be the name (Vietnamese "Van"), which
is what the words-to-spare test below is for, not an
afterthought to it. Dutch and Flemish names are listed exactly
this way ("Beethoven, Ludwig van"), the tussenvoegsel trailing
the given name but belonging to the surname.
Where a family comma has already named the family, a particle
ending the name attaches to that family name and is written
before it — provided at least one given word remains, so that a
name whose only given word is the particle keeps it (the
words-to-spare test S2 applies to ambiguous suffixes). Where
the word is BOTH a particle and suffix vocabulary, this
attachment outranks the suffix reading (S2): a trailing
abbreviation after a family comma is the tussenvoegsel far more
often than the decoration it collides with.
"Jong, Anke de" → family="de Jong" deviates: #379 (today: family="Jong")
"Beethoven, Ludwig van" → family="van Beethoven" deviates: #379 (today: family="Beethoven")
"Berg, Jan vd" → family="vd Berg" deviates: #380 (today: family="Berg")
"Nguyen, Van" → given="Van" · boundary
Accepted: without a family comma the name's written shape is not
settled — "Jong Anke de" may be a misformatted listing, and a
bare "Jong de" may be a given name beside a particle — so the
attachment is scoped to the comma form, and the comma-less
shapes keep their positional reading.
"Jong Anke de" → family="de"
Accepted: the precedence over S2 is stated for the shape, so it
sweeps in every word that is both particle and suffix
vocabulary — today vd, do and mc. Only vd's reading was
weighed; the other two inherit it, which is the shape's cost
and is tracked with the other contested memberships.
history: decisions.md#P6 · interacts: C1, P1, S2 · tracked: #379, #380

## Suffixes: generational & credentials (S)

Background: what follows a name is one of two different things —
Expand Down Expand Up @@ -709,19 +763,28 @@ R2. Rationale: callers need the surname with and without its
"Dr. Juan Q. Xavier de la Vega III" → family_base="Vega"
"Dr. Juan Q. Xavier de la Vega III" → family_particles="de la"
"Sean O'Connor" → family_base="O'Connor" · boundary
Accepted: an all-particle family reads an empty base today;
whether it should is #385.
"Anh Do" → family_base=""
history: decisions.md#R2 · implemented: nameparser/_types.py
A family name written wholly out of particle vocabulary still
has a base where one of those words is itself borne as an
ordinary surname: that word anchors the base, and only the words
that are never anyone's name stay particles.
"Anh Do" → family_base="Do" deviates: #385 (today: family_base="")
"Juan van der" → family_base=""
history: decisions.md#R2 · interacts: R3 · implemented: nameparser/_types.py

R3. Rationale: initials abbreviate the person's name words; titles,
suffixes, particles and nicknames are not name words.
Initials take the first letter of each given, middle, and base
family word; titles, suffixes, particles and nicknames
contribute nothing.
"Dr. Juan Q. Xavier de la Vega III" → initials="J. Q. X. V."
"Anh Do" → initials="A. D." deviates: #385 (today: initials="A.")
"Sean O'Connor" → initials="S. O." · boundary
implemented: nameparser/_render.py
Accepted: a family that is ALL particles contributes nothing,
so the initials are the given words alone — "van der" has no
borne name to anchor a base (R2), and initials of a bare
particle run would be nonsense.
"Juan van der" → initials="J."
history: decisions.md#R2 · interacts: R2 · implemented: nameparser/_render.py

R4. Rationale: case repair is a display concern, applied only on
request and never destructively.
Expand Down
15 changes: 12 additions & 3 deletions nameparser/_pipeline/_post_rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,18 @@ def post_rules(state: ParseState) -> ParseState:

# rules.md#P1: "a never-given particle standing alone where the
# given name would go — or opening the name — marks the name as
# surname-only: the given and middle words fold into the family.
# It needs another name word to fold into." (v1
# handle_non_first_name_prefix; history: decisions.md#P1)
# surname-only: the particle run and the one name word it
# attaches to are the family, and any name words beyond that
# read by position." (v1 handle_non_first_name_prefix; history:
# decisions.md#P1)
# DEVIATION #364: the fold below still takes every remaining name
# word, not just the particle run's own -- de Mesnil Juan gives
# family=de Mesnil Juan where the rule says family=de Mesnil plus
# given=Juan. Pinned by the deviates: markers on P1.
# Values written unquoted deliberately: this note sits INSIDE the
# citation block above (# decisions.md#P1) does not close it --
# _CITE_RE wants a colon after the ID), and the excerpt check
# takes the first quoted span in the block.
# Code-local: a lone PIECE is the test at both sites, so a
# particle group already chained forward is not a lone particle,
# and rule H1 above cannot be what produces the fold's family
Expand Down
15 changes: 13 additions & 2 deletions tests/v2/rules_doc.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@

plus per-rule ``no-boundary: reason`` lines and the trailing pointer
line ``history: ... · interacts: A1, B2 · implemented: path, path``.
A rule nothing implements yet carries ``tracked: #N, #M`` in place of
``implemented:`` — the issues that would ship it. Exactly one of the
two is required, so a normative rule always points either at the code
that honors it or at the work that will.

Inside a rule block, any line whose first non-space character is a
double quote (or an opening bracket, the D-section subject form) is an
Expand Down Expand Up @@ -35,8 +39,9 @@
rf"(?P<tfield>[a-z_]+)=(?P<today>{_VALUE})\))?"
r"\s*$")
_NO_BOUNDARY_RE = re.compile(r"^\s*no-boundary:\s+(?P<reason>\S.*)$")
_POINTER_RE = re.compile(r"^\s*(history|interacts|implemented):")
_POINTER_PART_RE = re.compile(r"(history|interacts|implemented):\s*([^·]+)")
_POINTER_RE = re.compile(r"^\s*(history|interacts|implemented|tracked):")
_POINTER_PART_RE = re.compile(
r"(history|interacts|implemented|tracked):\s*([^·]+)")

ASSERTABLE_FIELDS = frozenset({
"title", "given", "middle", "family", "suffix", "nickname", "maiden",
Expand Down Expand Up @@ -64,6 +69,10 @@ class Rule:
no_boundary: str | None = None
interacts: tuple[str, ...] = ()
implemented: tuple[str, ...] = ()
#: Issues that would ship a rule nothing implements yet. Mutually
#: exclusive with ``implemented:`` -- a rule points at code or at
#: the issues that will produce it, never at neither.
tracked: tuple[str, ...] = ()

def has_boundary_or_waiver(self) -> bool:
return self.no_boundary is not None or any(
Expand Down Expand Up @@ -184,4 +193,6 @@ def parse_rules_doc(text: str) -> list[Rule]:
current.interacts = items
elif key == "implemented":
current.implemented = items
elif key == "tracked":
current.tracked = items
return rules
12 changes: 11 additions & 1 deletion tests/v2/test_doc_citations.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,23 @@ def test_implemented_matches_citing_modules() -> None:
citing.setdefault(cid, set()).add(str(path.relative_to(REPO)))
problems = []
for rule in parse_rules_doc(RULES_DOC.read_text(encoding="utf-8")):
actual = citing.get(rule.rule_id, set())
if rule.implemented:
actual = citing.get(rule.rule_id, set())
declared = set(rule.implemented)
if actual != declared:
problems.append(
f"{rule.rule_id}: implemented: says {sorted(declared)} "
f"but citations found in {sorted(actual)}")
elif rule.tracked and actual:
# The other half of test_rules_doc.py's exactly-one-pointer
# rule: that test cannot see code, so a rule that SHIPPED
# while keeping tracked: would pass it. Without this branch
# the stale pointer is invisible -- the loop above skips
# any rule with no implemented: at all.
problems.append(
f"{rule.rule_id}: declares tracked: {sorted(rule.tracked)} "
f"but code cites it in {sorted(actual)}; swap tracked: for "
f"implemented: now that something implements it")
assert not problems, "\n".join(problems)


Expand Down
24 changes: 24 additions & 0 deletions tests/v2/test_rules_doc.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from __future__ import annotations

import importlib.util
import re

import pytest

Expand All @@ -32,6 +33,29 @@ def test_every_rule_has_examples_and_boundary(rule: Rule) -> None:
f"'no-boundary: <reason>'")


@pytest.mark.parametrize("rule", RULES, ids=lambda r: r.rule_id)
def test_every_rule_points_at_code_or_at_the_work(rule: Rule) -> None:
"""A normative rule names the code honoring it, or the issues that
would ship it -- never neither, so an unimplemented rule cannot sit
in the doc untracked, and never both.

Scope, precisely: this test reads the DOC only, so it cannot tell
that a tracked: rule has since been implemented -- adding the code
and its citation while leaving tracked: in place passes here. That
half is test_doc_citations.py::test_implemented_matches_citing_
modules, which sees the citing modules. Neither test alone makes a
stale pointer unrepresentable; the pair does.
"""
assert rule.implemented or rule.tracked, (
f"{rule.rule_id}: add 'implemented: <path>' or, if nothing "
f"implements it yet, 'tracked: #N' naming the issues that would")
assert not (rule.implemented and rule.tracked), (
f"{rule.rule_id}: has both implemented: and tracked:; drop "
f"tracked: once the rule ships")
bad = [t for t in rule.tracked if not re.fullmatch(r"#\d+", t)]
assert not bad, f"{rule.rule_id}: tracked: wants #N issue refs, got {bad}"


def _check_diagnostic(example: Example) -> None:
import re as _re
from collections.abc import Callable
Expand Down