Skip to content

Parse cost could fall by about 18% with no reading moved #1436

Closed

Description

@derek73

The 3.1 simplify pass (#1434) measured scratch patches that cut frame counts without moving any reading. They were out of scope there because they need _CALL_BASELINE moved on purpose.

Name master with the patches
reference row 285 about 234
Smith, John 136 about 112

The largest pieces:

  • the per-character _ignorable call in _tokenize, which scales with every character of the name;
  • _is_emoji's generator over its ranges;
  • generator-to-list changes in _classify and _group.

The patches were not kept, so these figures are a reviewer's measurement to reproduce, not a result.

Done when:

  • each change holds snapshot.jsonl byte-identical and the scaling ratios in tests/v2/test_benchmark.py;
  • _CALL_BASELINE is re-taken once, with a dated entry in docs/design/decisions/parse-cost.md giving before and after;
  • wall-clock time for the thousand-names test is reported beside the frames, since a frame count cannot see C-level work.

Metadata

Metadata

Assignees

Labels

internalsRefactors and parse cost with no behavior change

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions