experimental-inspect: Iterator __next__ not optional - #6274
Open
jonasdedden wants to merge 4 commits into
Open
Conversation
PyO3 lets `__next__` be written as `-> Option<T>`, where `None` means "raise
`StopIteration`". Introspection resolved that through the ordinary return-type
path, so the stub said:
def __next__(self, /) -> int | None: ...
Which is wrong in a way that costs more than `Incomplete` would. mypy takes
`__next__`'s return type as the iteration item type, so every `for x in it`
binds `int | None`, and the class stops satisfying `Iterator[int]`. On a real
extension whose iterators are its main output type, exporting the iterator
classes with this bug produced 66 new `mypy --strict` errors, against 0 when
they were left unexported and typed `Any`.
`IterNextOutput<T>` strips the encoding for the two shapes that carry it,
`Option<T>` and `Result<Option<T>, E>`, using the inherent-const-beats-trait-
const trick PyO3 already uses for `Probe`. An iterator that always yields and
raises to stop — `PyClassIter` in `pytests` — matches neither and falls through
to `PyReturnType` unchanged.
`pytests` gains `PyClassOptionIter`, which had no counterpart: every existing
`__next__` there returns a plain value, so nothing covered this shape.
`PyClassOptionIter` returned `PyResult<Option<usize>>` and never errored, which trips `clippy::unnecessary_wraps` and broke the clippy jobs. Split the two shapes the introspection change strips instead of picking one: `PyClassOptionIter` now returns a plain `Option<usize>`, and the new `PyClassResultOptionIter` keeps `PyResult<Option<usize>>` — the wrapper is the point there, so the lint is `#[expect]`ed with a reason. Both stubs come out as `-> int`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BcrSbuvXqbWWormVZCv3Tq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PyO3 lets
__next__be written as-> Option<T>, whereNonemeans "raiseStopIteration". Introspection resolved that through the ordinary return-type path, so the stub said:Which is wrong in a way that costs more than
Incompletewould. mypy takes__next__'s return type as the iteration item type, so everyfor x in itbindsint | None, and the class stops satisfyingIterator[int].IterNextOutput<T>strips the encoding for the two shapes that carry it,Option<T>andResult<Option<T>, E>, using the inherent-const-beats-trait-const trick PyO3 already uses forProbe. An iterator that always yields and raises to stop -PyClassIterinpytests- matches neither and falls through toPyReturnTypeunchanged.pytestsgainsPyClassOptionIter, which had no counterpart: every existing__next__there returns a plain value, so nothing covered this shape.