Inconsistencies in __annotations__ when annotating with or without aliases

[harahu]

I just upgraded some code to python 3.10, and auto-upgraded type hints to use non-aliased types where possible. So, for instance typing.Dict became dict. This lead to an issue.

I have some code that does reflection on type annotations, through __annotations__ attributes, and it turns out that we get slightly different content in the __annotations__ dict of a type when annotating using aliases vs. annotating using types directly. This led to logic errors in my case.

This difference can be seen by executing the following code:

from typing import Dict
from typing import Generic
from typing import Tuple
from typing import TypeVar

T = TypeVar("T")


class SomeContainer(Generic[T]):
    v: T


class Foo:
    x: dict[str, "Baz"]
    y: tuple["Baz"]
    z: SomeContainer["Baz"]


class Bar:
    x: Dict[str, "Baz"]
    y: Tuple["Baz"]
    z: SomeContainer["Baz"]


class Baz:
    pass


print(Foo.__annotations__)
print(Bar.__annotations__)

For instance, the annotation dict[str, "Baz"] on Foo leads to dict[str, 'Baz'] in the __annotations__ dict, while Dict[str, "Baz"] on Bar recognizes the forward reference to Baz, and results in typing.Dict[str, ForwardRef('Baz')].

My user-defined generic SomeContainer also recognizes the forward-ref, resulting in __main__.SomeContainer[ForwardRef('Baz')]

In other words, it seems that the construction of __annotations__ usually understands forward references, but not for certain built in types, like dict or tuple explored here.

Is this intentional, or is this a bug?

[hmc-cs-mdrissi]

What does get_type_hints give for each case? I think get_type_hints is the public api way to examine annotations. Using __annotations__ works but I’m less sure how well defined behavior should be.

[harahu]

When I said I was reflecting using __annotations__, I lied. I was using get_type_hints, but the issue I encountered stems from __annotations__. Modified my example accordingly:

from typing import Dict
from typing import Generic
from typing import Tuple
from typing import TypeVar
from typing import get_type_hints

T = TypeVar("T")


class SomeContainer(Generic[T]):
    v: T


class Foo:
    x: dict[str, "Baz"]
    y: tuple["Baz"]
    z: SomeContainer["Baz"]


class Bar:
    x: Dict[str, "Baz"]
    y: Tuple["Baz"]
    z: SomeContainer["Baz"]


class Baz:
    pass


print(Foo.__annotations__)
print(get_type_hints(Foo))
print(Bar.__annotations__)
print(get_type_hints(Bar))

And we get:

{'x': dict[str, 'Baz'], 'y': tuple['Baz'], 'z': __main__.SomeContainer[ForwardRef('Baz')]}
{'x': dict[str, 'Baz'], 'y': tuple['Baz'], 'z': __main__.SomeContainer[__main__.Baz]}
{'x': typing.Dict[str, ForwardRef('Baz')], 'y': typing.Tuple[ForwardRef('Baz')], 'z': __main__.SomeContainer[ForwardRef('Baz')]}
{'x': typing.Dict[str, __main__.Baz], 'y': typing.Tuple[__main__.Baz], 'z': __main__.SomeContainer[__main__.Baz]}

So, due to the presence of a plain string instead of a ForwardRef, get_type_hints isn't able to properly evaluate the type.

[Gobot1234]

I think you might be interested in python/cpython#85542

[harahu]

I think you might be interested in python/cpython#85542

This seems to be exactly what I was looking for! I see it's been closed. Does that mean I'll have to wait for 3.11, or is this coming in a 3.10 patch release at some point?

[JelleZijlstra]

We didn't backport it into 3.10, so this change will be in 3.11+ only.