You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Parse cost could fall by about 18% with no reading moved #1436
The 3.1 simplify pass (#1434) measured scratch patches that cut frame counts without moving any reading. They were out of scope there because they need _CALL_BASELINE moved on purpose.
Name
master
with the patches
reference row
285
about 234
Smith, John
136
about 112
The largest pieces:
the per-character _ignorable call in _tokenize, which scales with every character of the name;
_is_emoji's generator over its ranges;
generator-to-list changes in _classify and _group.
The patches were not kept, so these figures are a reviewer's measurement to reproduce, not a result.
Done when:
each change holds snapshot.jsonl byte-identical and the scaling ratios in tests/v2/test_benchmark.py;
_CALL_BASELINE is re-taken once, with a dated entry in docs/design/decisions/parse-cost.md giving before and after;
wall-clock time for the thousand-names test is reported beside the frames, since a frame count cannot see C-level work.
Activity
Test the emoji ranges in a loop rather than a generator
Scan a region for its separators in C, not character by character
Sort tokenize's tokens by an attrgetter key
Build classify's tokens from a list, not a generator
Freeze group's pieces with map rather than generators
Read group's tail start by identity, not from an enum-keyed dict
Check a Token's span without a generator, and keep a Span as given
Skip the str and Mapping refusals for a Token's frozenset tags
Coerce a Token's role only when it is not already a Role
Bring the tokenize comments in line with the scan, and test its edges
Check both class readers under every strip setting
Re-take the call baselines and the link pin after #1436's cuts
Date the link pin's old band arithmetic, and pin the recipe's base