Skip to content

fix: prevent false unreachable error when comparing generic callables#21188

Open
Bahtya wants to merge 2 commits intopython:masterfrom
Bahtya:fix/unreachable-generic-callable-comparison
Open

fix: prevent false unreachable error when comparing generic callables#21188
Bahtya wants to merge 2 commits intopython:masterfrom
Bahtya:fix/unreachable-generic-callable-comparison

Conversation

@Bahtya
Copy link
Copy Markdown

@Bahtya Bahtya commented Apr 8, 2026

Problem

Fixes #21182

When comparing a generic Callable parameter with a generic function using ==, mypy incorrectly reported "Statement is unreachable" with --warn-unreachable:

from typing import Callable, TypeVar

T = TypeVar("T")
S = TypeVar("S")

def identity(x: T) -> T:
    return x

def msg(cmp_property: Callable[[T], S]) -> None:
    if cmp_property == identity:
        return  # ERROR: Statement is unreachable [unreachable]

This is a regression — the code was accepted in earlier mypy versions.

Root Cause

In conditional_types with from_equality=True, mypy uses shallow_erase_type_for_equality to erase generic type parameters from the proposed type before checking overlap. However:

  1. shallow_erase_type_for_equality only handled Instance types, not CallableType. So generic callables like def [T] (x: T) -> T were never erased.

  2. Only the proposed type was erased, not the current type. When is_overlapping_types was called with an unerased generic callable as current_type (e.g., the identity function with .variables=(T,)), is_callable_compatible tried to unify its type variables against the other callable's free type variables and failed, concluding the types don't overlap.

At runtime, generic type parameters are erased, so identity and cmp_property can absolutely be equal — one could be passed as the other.

Solution

Two changes:

1. Extend shallow_erase_type_for_equality to CallableType (mypy/erasetype.py)

  • Non-type-object callables with .variables have their arg types and return type replaced with Any and variables cleared
  • Type objects (class constructors) are excluded to preserve their identity for type narrowing

2. Erase both sides in the overlap check (mypy/checker.py)

  • In conditional_types, apply shallow_erase_type_for_equality to both current_type and proposed_type before calling is_overlapping_types

Testing

  • ✅ Original reproduction case from [1.20 regression] unreachable regression with generic Callable #21182 passes (no more false unreachable)
  • ✅ All 146 existing unreachable tests pass
  • ✅ All 298 narrowing tests pass
  • ✅ All 73 equality tests pass
  • ✅ All 198 meet/overlap tests pass
  • ✅ All 10 erase tests pass
  • ✅ Comprehensive test cases: comparing two generic callables, generic vs non-generic, != operator — all work correctly

Bahtya and others added 2 commits April 9, 2026 02:56
When comparing a generic Callable parameter with a generic function
using ==, mypy incorrectly concluded the types could never overlap
and marked the if-body as unreachable.

Fix by extending shallow_erase_type_for_equality to handle CallableType
and using erased current_type in the equality overlap checks.

Fixes python#21182

Signed-off-by: bahtya <bahtyar153@qq.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

Diff from mypy_primer, showing the effect of this PR on open source code:

rotki (https://github.com/rotki/rotki)
- rotkehlchen/tests/integration/test_premium.py:844: error: Statement is unreachable  [unreachable]
+ rotkehlchen/tests/integration/test_premium.py:845: error: Statement is unreachable  [unreachable]
- rotkehlchen/tests/integration/test_premium.py:858: error: Statement is unreachable  [unreachable]
+ rotkehlchen/tests/integration/test_premium.py:859: error: Statement is unreachable  [unreachable]

Copy link
Copy Markdown
Collaborator

@hauntsaninja hauntsaninja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm having to erase current type points to a bug in is_overlapping_types

@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 9, 2026

Thanks for the review! You're right that needing to erase the current type is a signal of an underlying issue in is_overlapping_types.

Looking at the bug: when comparing Callable[[T], S] with def identity(x: T) -> T, the overlap check receives the original TypeVar-bound callable on the left, and is_overlapping_types doesn't account for the fact that TypeVar bounds can unify across both sides. The erasing is a workaround to avoid a false non-overlap, but ideally is_overlapping_types should treat TypeVar-bound callables as potentially overlapping with any compatible generic callable.

Would you prefer I investigate fixing is_overlapping_types directly? I could open a separate issue/PR for that, and keep this PR as the targeted regression fix in the meantime.

@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 9, 2026

The mypyc-compiled test failures (testAllBase64Features_librt with binascii.Error: Incorrect padding) also appear on #21190 and other recent PRs — this is a pre-existing flaky test unrelated to this change.

@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 9, 2026

Update on the mypyc-compiled test failures:

I've confirmed these are pre-existing and unrelated to this PR:

  1. Same failure on other PRs: PR Fix narrowing for match case with variadic tuples #21192 by @hauntsaninja has the exact same 3 failing checks (testAllBase64Features_librt with binascii.Error: Incorrect padding)

  2. Already tracked: This is filed as issue testAllBase64Features_librt fails with Python 3.14.4 #21120 — the root cause is a behavior change in Python 3.13.13+/3.14.4+ where binascii.a2b_base64 no longer ignores excess data after padding (CPython commit e31c551)

  3. Not flaky — consistently failing: The test fails deterministically on all mypyc-compiled CI jobs that use the latest Python patch versions (3.13.13, 3.14.4, 3.14.4t)

No action needed on this PR for those failures.

@Bahtya
Copy link
Copy Markdown
Author

Bahtya commented Apr 9, 2026

Hi @hauntsaninja — pinging on this since I haven't heard back after your is_overlapping_types comment.

To recap my earlier response: I agree the type erasure points to a deeper issue in is_overlapping_types. I'm happy to either:

  1. Keep this PR as a targeted regression fix, and open a separate issue/PR to investigate and fix is_overlapping_types properly, or
  2. Investigate the root cause directly in this PR if you'd prefer a more complete fix here.

On the CI failures: the 3 failing mypyc-compiled checks (py313, py314, py314t) are pre-existing and unrelated to this change — they're caused by a behavior change in Python 3.13.13+/3.14.4+ in binascii.a2b_base64, tracked in issue #21120. The same failures appear on other recent PRs.

What would you like me to do next?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[1.20 regression] unreachable regression with generic Callable

2 participants