gh-153569: consolidate tokenizer input, layout, and diagnostics - #157055

Open

pablogsal wants to merge 6 commits into

python:mainfrom

pablogsal:gh-153569-tokenizer-input-diagnostics

Commits on Sep 12, 2026

gh-153569: move tokenizer input state and relocation into the reader
pablogsal
committed
abb1d45
gh-153569: borrow diagnostic lines through the source API
pablogsal
committed
8023fd8
gh-153569: group indentation and logical-line state
pablogsal
committed
9664ee8
gh-153569: remove unused tokenizer cursor and source lookup APIs
pablogsal
committed
65c1e22
gh-153569: report tokenizer diagnostics without rewinding the scanner
pablogsal
committed
029775f
gh-153569: use tokenizer views and remove obsolete API remnants
pablogsal
committed
d03c162