Repository navigation
Skip C4's count for a run of initials behind a full name (#1435) - #1458
Merged
Merged
Conversation
rules/C16's run of initials reads as the given name only behind one word, so where it is the comma part's only word of the ambiguous class, _comma.decide now asks _lone_word first and, behind a head that is not one word, skips name_word_count and the C11 books. The run stays in `maybe`, so the part is read as before and never takes the listing form's early return. 'John Smith, M.D.' 209 -> 187 frames on 3.11 (191 -> 173 on 3.15); no reading moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t the log
The 'another word of the class' row used 'M.D. MA', where MA is the
run's company and the run never joins maybe; 'M.D. M.A' reaches the
branch. Pin the readings an over-wide gate moves ('Smith van, M.D.',
'John Smith, M.D. M.A') in test_comma.py, and correct the log bullet:
master's reading of 'Smith van, M.D.', the frame figures re-taken in
fresh processes with _frames_for, and 'only entry in maybe'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-take the grid's mutation counts (34,848 and 4,536 of 215,280 rows), name the one-case arm of the run's letter test, and say that C16's count is taken only where the head may be one word. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73
added a commit
that referenced
this pull request
Nov 19, 2026
Measured on the branch rebased onto 688991e (#1458's merge), each interpreter in an environment of its own: parse 285.72 -> 197.72 on 3.11 and 277.72 -> 188.72 on 3.12 to 3.15, so _CALL_BASELINE moves 285 -> 197 and 277 -> 188; the 64-link run 2,404 -> 1,716 and 2,395 -> 1,706, so _LINK_BASELINE moves 2,418 -> 1,716 and 2,398 -> 1,706. decisions/parse-cost records the move with per-commit and per-name figures, the clock, and the checks that no reading moved. Closes #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73
added a commit
that referenced
this pull request
Nov 19, 2026
…1459) * Test the emoji ranges in a loop rather than a generator _is_emoji ran any() over a generator, a frame per range tried, for every non-ASCII character under strip_emoji (the default). Frames per parse on py3.11: reference row 285.72 -> 285.72, 'Smith, John' 136 -> 136, the long row 473.86 -> 473.86, a Devanagari name 336.04 -> 271.04. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Scan a region for its separators in C, not character by character _tokenize_region asked _ignorable of every character of the name, a frame each. The ignorable set is now a regex class per setting of the two strip switches, built from the same parts (whitespace, the name dots, the invisible separators, _BIDI's class, the emoji ranges), and the region is scanned with finditer for the characters that may separate: those, the commas and the transcription dots, whose context tests are unchanged. _ignorable and _stripped match the same classes, so extract and the flank guard read the one definition; a new test holds each class to its parts over every code point, and _is_emoji goes, its ranges now read only through the class. Frames per parse on py3.11: reference row 285.72 -> 258.72, 'Smith, John' 136 -> 126, the long row 473.86 -> 421.86, a Devanagari name 271.04 -> 226.04. Tokenize output for every policy over a grid of the separator candidates in context, and _ignorable/_stripped over every code point, compared byte-identical with master in two trees. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Sort tokenize's tokens by an attrgetter key The lambda key was a frame per token. Frames per parse on py3.11: reference row 258.72 -> 252.72, 'Smith, John' 126 -> 124, the long row 421.86 -> 411.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Build classify's tokens from a list, not a generator tuple() over a generator resumed it once per token, a frame each. Frames per parse on py3.11: reference row 252.72 -> 246.72, 'Smith, John' 124 -> 122, the long row 411.86 -> 401.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Freeze group's pieces with map rather than generators Each segment's pieces and piece tags were frozen through a generator, a frame per piece twice over; map over the type does it in C. Frames per parse on py3.11: reference row 246.72 -> 236.72, 'Smith, John' 122 -> 114, the long row 401.86 -> 383.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read group's tail start by identity, not from an enum-keyed dict The dict was built on every parse, and Enum.__hash__ is a Python method: three frames a parse. Frames per parse on py3.11: reference row 236.72 -> 233.72, 'Smith, John' 114 -> 111, the long row 383.86 -> 380.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check a Token's span without a generator, and keep a Span as given Token.__post_init__ tested the two coordinates with all() over a generator, three frames a token, and rebuilt the span as a Span even when it was one, a frame for the NamedTuple's __new__. The checks and their messages are unchanged; a span that is not exactly a Span is still rebuilt as one. Frames per parse on py3.11: reference row 233.72 -> 209.72, 'Smith, John' 111 -> 103, the long row 380.86 -> 340.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Skip the str and Mapping refusals for a Token's frozenset tags A frozenset is neither, and isinstance against Mapping, an ABC, enters a Python frame; every token the parse returns carries a frozenset. Frames per parse on py3.11: reference row 209.72 -> 203.72, 'Smith, John' 103 -> 101, the long row 340.86 -> 330.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Coerce a Token's role only when it is not already a Role _coerce_enum hands a Role back unchanged, a frame a token for nothing on every token the parse builds; a string role is still coerced, and refused, by it. Frames per parse on py3.11: reference row 203.72 -> 197.72, 'Smith, John' 101 -> 99, the long row 330.86 -> 320.86. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bring the tokenize comments in line with the scan, and test its edges Review of the first nine commits: the _stripped and _ignorable docstrings and the module docstring still described the per-character call, and none named U+202F as the one ignorable character context decides. The every-code-point test now matches each class whole, in C (0.16s, was about 3.5s across four parameters), and says what it cannot guard. New rows hold what no test reached: the dot's flanks and U+202F's letters are read inside the region (each fails with the bound dropped), and a Token's span is checked coordinate by coordinate when it is already a Span (each fails with the check moved behind the Span test). Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check both class readers under every strip setting Review of 8392358: its reader rows ran under the all-on and all-off policies only, where a reader taking the two switches in the wrong order reads right; under every setting that mutation now fails. The module docstring's claim that extract and tokenize cannot disagree is narrowed to the characters that separate in every context, the two context decides being tokenize's alone. And a span with a bad start and a good end gets its rows. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Re-take the call baselines and the link pin after #1436's cuts Measured on the branch rebased onto 688991e (#1458's merge), each interpreter in an environment of its own: parse 285.72 -> 197.72 on 3.11 and 277.72 -> 188.72 on 3.12 to 3.15, so _CALL_BASELINE moves 285 -> 197 and 277 -> 188; the 64-link run 2,404 -> 1,716 and 2,395 -> 1,706, so _LINK_BASELINE moves 2,418 -> 1,716 and 2,398 -> 1,706. decisions/parse-cost records the move with per-commit and per-name figures, the clock, and the checks that no reading moved. Closes #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Date the link pin's old band arithmetic, and pin the recipe's base Review of f13950e: the comment on why the pin reads the long run mixed the 2026-09-22 clause-run figures, given a wrong date, with the name-link run's current one, and kept the old band in the present tense beside the new; it now states the old arithmetic as measured then and points at the pin for the current band. The decisions entry's recipe compared against origin/master, which reads 0 once this merges, and now names 688991e; and it names both pairs of trees the 114,647-reading grid compared, re-run after the rebase, identical. Refs #1436 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1435
In
_comma.decide, rules/C16's block put a dotted run of initials that spells a listed credential intomaybeeven behind a full name. SoJohn Smith, M.D.paid C4'sname_word_countand the C11 books, which only a one-word head reads the run by.Where the run is the part's only entry in
maybe(run_alone),decidenow asks_lone_wordfirst. Its answer is kept for the reading, so it is asked once. Behind a head that is not one word,decideskips the count and the books. The run stays inmaybe, so the part is still read as the count's part and never takes the listing form's early return. This is why I did not take the issue's proposed gate: it sends the part down the no-maybepath, which matches only where the listing return and C1's fallback agree, and nothing guarantees that. With nothing else inmaybe, the count licenses nothing,head_namedmeets no member, and the books find nothing. So the reading is the one the count would give.Frames
One parse after a warm-up, in a fresh process, with
test_benchmark._frames_for; master (b78d5af) → branch:John Smith, M.D.John Smith, MDSmith, JohnSmith, J.P.(still counts)The issue's 209 for
John Smith, M.D.on master was taken in a process that had parsed other names first. That changes the absolute count, not the difference (decisions/parse-cost, 2026-11-09 #1170). The reference band holds;_CALL_BASELINEis not edited.Evidence that no reading moves
tools/differential/snapshot.jsonlis byte-identical and untouched.str(),initials()).Freiherr St,St. Freiherr), one-word heads,Bin Zhang,de Waart,Smith van, one-case names and more.M.D.;, fullwidth.,,,,,, and a later, PhDpart) × the 3name_orders × 6 policies (default, strict comma suffixes, dotted switch off, caps EVERYWHERE and OFF, an extra delimiter)._lone_wordthat always answers true moves 34,848 rows. Skipping the count behind a one-word head too moves 4,536:Smith van, M.D.reads given 'M.', middle 'D.', family 'Smith van', where master reads given 'Smith', family 'van', suffix 'M.D.'. The issue's own gate also moved 0 rows on this grid.snapshot.py --local ctgov-officials --base <master>: 539,368 officials.snapshot.py --local ctgov-contacts --base <master>: 522,688 contacts.Tests
test_a_run_of_initials_behind_a_full_name_takes_no_count(tests/v2/test_benchmark.py) holds the saving with exact counts ofname_word_countand_lone_word, with reachability probes. Its negative control is recorded: with the gate'slone_head is not Falsearm deleted, no reading moves andJohn Smith, M.D.costs 23 more frames. It passes alone with-n0, and under 3.15 in its own environment.test_the_count_c16_skips_decides_no_reading(tests/v2/pipeline/test_comma.py) pins the readings a wrong gate moves. It recordsSmith van, M.D.as it reads today and does not endorse that reading.run_aloneforced true: both tests fail.docs/design/decisions/parse-cost.mdgets a dated Log bullet; the baseline entry is not touched. There is no release-log line, following the earlier cost-only PRs (#1170, #1089, #1036).Checks and reviews
uv run pytest(49,958 passed),uv run pytest -n0 -m serial,uv run mypyanduv run ruff checkall pass.M.D. MA) never reached the branch, becauseMAis the run's company. Replaced withM.D. M.A, and the reading pins were added.Follow-up worth a look, not touched here: master reads
Smith van, M.D.as given 'Smith', family 'van', suffix 'M.D.', andJohn Smith, M.A M.D.as given 'M.', middle 'A', family 'John Smith'.🤖 Generated with Claude Code