gh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF

[JelleZijlstra]

Fixes #132805. Closes #132806. cc @DavidCEllis.

• Issue: [3.14] annotationlib - Union '|' syntax and typing.Union[...] generate different forward references. #132805

[DavidCEllis]

This PR does mean that partial evaluations of the different ways of writing unions aren't quite equivalent. A union written with | with an element that can't be evaluated gives no information about the types, while a union written with Union[...] gives partial information¹.

• str | undefined -> ForwardRef('__annotationlib_name__1 | undefined', ...)
• Union[str, undefined] -> str | ForwardRef('undefined', ...)

This does make me think that perhaps the previous change to make a | b into Union[a, b] might still be worth doing? Arguably a and b in this case are forward references and | between forward references creates a union.

If not, then perhaps the __extra_names__ should be more visible to help distinguish between otherwise similar looking forward references with different contents.

Example:

from annotationlib import get_annotations, Format

class Ex1:
    a: str | undefined
    b: str | int | undefined

for k, v in get_annotations(Ex1, format=Format.FORWARDREF).items():
    print(f"{k}: {v}")

a: ForwardRef('__annotationlib_name_1__ | undefined', is_class=True, owner=<class '__main__.Ex1'>)
b: ForwardRef('__annotationlib_name_2__ | undefined', is_class=True, owner=<class '__main__.Ex1'>)

Footnotes

1. Side note: is there a helper function somewhere to try to evaluate any forwardrefs contained in a union or generic alias? ↩

[JelleZijlstra]

I sort of think that's correct: if we don't know what A and B are, we don't know what A | B will evaluate to. It might be a union, but it might be something else too if A happens to implement __or__ some other way.

We could perhaps hide the __annotation_name_1__ names in the repr() of ForwardRefs, though, do you think that would be better?

[DavidCEllis]

I sort of think that's correct: if we don't know what A and B are, we don't know what A | B will evaluate to. It might be a union, but it might be something else too if A happens to implement __or__ some other way.

My logic is more along the lines of - if a name can't be resolved it becomes a forward ref, anything that uses | with a forwardref becomes a union¹ thus it makes sense by that logic for this to become a union.

The other reasoning is that a runtime checker could use the partial information to know that an argument is valid in the case where you get a union, if the type is one of the elements that does resolve, but couldn't do so in the case where the entire statement is a forwardref. I'm not sure how common this case would be though.

We could perhaps hide the __annotation_name_1__ names in the repr() of ForwardRefs, though, do you think that would be better?

I'm not sure about this one, I know the repr already isn't exactly complete though with all the hidden internals.

Footnotes

1. Provided it doesn't have an __or__ that does something else with a forwardref. ↩

[JelleZijlstra]

It still feels wrong to me to special-case just __or__; your suggestion would mean this operator works differently than all the others.

I'm actually not convinced ForwardRef.__or__ should exist at all, as ForwardRefs shouldn't be created in a place where users create Unions out of them. It was added in https://bugs.python.org/issue45489 without much discussion. Removing it would be a compatibility issue at this point though, so I'm not proposing it.

[DavidCEllis]

That makes sense, it just bugs me that this means a | b is not the same as Union[a, b] if you're using type hints in this context (and one happens to be a forwardref). Unfortunately tools like pyupgrade will automatically convert Union syntax to | and now in this case they're not equivalent.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x (tier-3) has failed when building commit c8f233c.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/3537) and take a look at the build logs.
4. Check if the failure is related to this commit (c8f233c) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/3537

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/99D824FA-F7CC-45B8-8E9C-C9B3DCBBA39B/data/Containers/Bundle/Application/9DCD2A29-C3D4-4FA7-B324-25A9D853418C/iOSTestbed.app/python/lib/python3.14/threading.py", line 1079, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/99D824FA-F7CC-45B8-8E9C-C9B3DCBBA39B/data/Containers/Bundle/Application/9DCD2A29-C3D4-4FA7-B324-25A9D853418C/iOSTestbed.app/python/lib/python3.14/threading.py", line 1021, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/99D824FA-F7CC-45B8-8E9C-C9B3DCBBA39B/data/Containers/Bundle/Application/9DCD2A29-C3D4-4FA7-B324-25A9D853418C/iOSTestbed.app/python/lib/python3.14/test/test_interpreters/test_stress.py", line 30, in task
    interp = interpreters.create()
  File "/Users/buildbot/Library/Developer/XCTestDevices/99D824FA-F7CC-45B8-8E9C-C9B3DCBBA39B/data/Containers/Bundle/Application/9DCD2A29-C3D4-4FA7-B324-25A9D853418C/iOSTestbed.app/python/lib/python3.14/test/support/interpreters/__init__.py", line 76, in create
    id = _interpreters.create(reqrefs=True)
interpreters.InterpreterError: interpreter creation failed
k