[ty] fix more ecosystem/fuzzer panics with fixpoint

[carljm]

Summary

Add cycle handling for try_metaclass and pep695_generic_context queries, as well as adjusting the cycle handling for try_mro to ensure that it short-circuits on cycles and won't grow MROs indefinitely.

This reduces the number of failing fuzzer seeds from 68 to 17. The latter count includes fuzzer seeds 120, 160, and 335, all of which previously panicked but now either hang or are very slow; I've temporarily skipped those seeds in the fuzzer until I can dig into that slowness further.

This also allows us to move some more ecosystem projects from bad.txt to good.txt, which I've done in #17903

Test Plan

Added mdtests.

[github-actions]

mypy_primer results

No ecosystem changes detected ✅

[AlexWaygood]

I know I shouldn't be looking at draft PRs, but just to note... as well as finding a couple of new panics, the fuzz job appears to have taken 15 minutes to complete on this PR branch (much longer than it does on main). I can repro this locally running this branch of red-knot on the code generated by seed 120, which is:

-(name_0 < name_2)([], **name_1, name_3=name_1)
[name_3, (), name_2.name_5, name_4.name_2, *name_5], *(), (name_1 async for name_2 in name_1)[not name_0] = (lambda name_4, /, name_0, name_1, name_3, name_5: {*()})[-(name_1 if name_0 else name_4)] = name_1 = [name_0.name_5, {name_2: name_3 for name_0 in name_4}[f''], [name_0[name_2], [], name_4.name_1, name_2.name_5, []], *name_5[name_5], {name_3: name_3 for name_1 in name_3}[{name_2 for name_5 in name_5}]] = [(name_2 := name_5) > '' <= [name_4 for name_0 in name_5] is {} is not [] in {*()} for name_4.name_5 in f'{name_0!a:}3.2430476877164316' for [name_1, name_5] in 9 if f'{name_4:}\'"\'\'\'"""{{}}\\some const text' if (name_2 if name_3 else name_2) if name_1 or name_3 or name_3 or name_5 or name_0 if -name_4 for name_5.name_4 in False if {name_0 for name_3 in name_0 for name_2 in name_2 for name_0 in name_1} if f'' if name_3 @ name_2 if (name_3, name_0, name_1) for [name_1] in name_1.name_5 if (name_4 async for name_0 in name_5 async for name_1 in name_3 for name_5 in name_5 for name_0 in name_4 async for name_0 in name_3) if {name_4 for name_4 in name_1 for name_2 in name_0 for name_1 in name_0} if name_2 or name_3 or name_5 or name_5 or name_2 or name_0 for name_1.name_5 in name_1.name_2 if {name_3: name_0 for name_4 in name_0 for name_1 in name_5 for name_4 in name_0 for name_4 in name_3 for name_1 in name_3 for name_1 in name_2} if {name_1: name_3 for name_1 in name_2 for name_0 in name_0 for name_2 in name_2} if name_1 if [name_1 for name_3 in name_5 for name_1 in name_4 for name_2 in name_0 for name_0 in name_1] if 'some const text'] # type: ignoresome text
b'\'"\'\'\'"""{}\\'.name_5.name_4 = name_1 = [[name_1.name_2, *name_2, name_1.name_4], (name_3 if name_4 else name_0)[{}], (name_4 := name_5)[name_5[name_2]], name_0] = {name_4 or name_0 for name_4 in name_3 if name_0 if name_4 if name_5 for name_4 in name_2 for name_1 in name_4 if name_0 if name_1 if name_0 if name_4}.name_2 = (name_4 == name_0)(name_1 == name_0, {*()}, name_4, name_1[name_5], 'some const text', **name_1, **name_3, name_2=name_0, **name_4, name_4=name_4).name_1 = 'some const text'[f''].name_2 = {lambda name_3, name_2, /, name_5, name_1, *, name_0=name_0, name_4=name_0: lambda **name_5: name_0: name_2.name_0 and {} and (lambda: name_1) and [name_4 for name_3 in name_5] and name_0 and (name_2 or name_3) for name_3[name_1] in {name_4, name_3, name_5, name_0} if (name_4 async for name_1 in name_5) for name_0 in name_5 - name_2 if {name_1: name_3 for name_5 in name_4 for name_3 in name_5} if name_4 if name_5 + name_1 if {name_1 for name_3 in name_4 for name_2 in name_4 for name_5 in name_0 for name_0 in name_3 for name_5 in name_5 for name_2 in name_4} if f'2' for [] in name_5(name_5, name_2, name_2, name_5, name_1, name_1=name_3, **name_2, **name_4) if [name_4 for name_1 in name_1 for name_0 in name_4 for name_2 in name_0 for name_1 in name_5 for name_3 in name_1] if [name_5 for name_4 in name_0 for name_1 in name_4]} # type: ignoresome text
name_2
assert [], b'some bytes'.name_4
{{(name_3 async for name_5 in name_2): (name_4 async for name_4 in name_0), {name_5 for name_5 in name_4}: name_4 // name_4, {*()}: [name_2 for name_5 in name_3], []: name_3[name_0]} for name_3[name_2] in name_4.name_5 for name_4 in b'some bytes'}
pass
try:
    try:

        @[]
        @name_3 >= name_1
        class name_4[**name_4](name_5=name_4):
            import name_3
        {*()}
        (name_5 := name_3)

        class name_2[**name_0]({name_0: name_1 for name_5 in name_0}, name_5 and name_5, *name_0, {*()}, {name_4: name_5 for name_4 in name_3}, name_3=name_3):
            try:
                pass
            except* name_4:
                pass
            import name_5 as name_5
            name_4
            name_5
            name_5 = name_1 # type: ignoresome text
            name_2
    except {name_5 for name_4 in name_0} as name_5:
        name_4
        name_0
    else:
        del name_2[name_2]
    finally:
        f''
        name_5[name_1] //= name_5.name_2
        []
        name_2()

    @name_1
    class name_1[*name_4, *name_3, **name_0, *name_1](lambda name_0, /, name_4, name_5, name_3, name_1, **name_2: name_2 << name_3, [name_4 for name_2 in name_2 if name_0 if name_3 if name_0 if name_5 for name_1 in name_2 for name_1 in name_5 if name_0 for name_1 in name_0 if name_2 if name_0 if name_3 if name_1 if name_5], name_3, name_1=f'{name_0!s:}\'"\'\'\'"""{{}}\\{name_3}{name_2!a}', name_3=(name_5, name_1, name_5, name_1), **{name_0 for name_2 in name_3 for name_2 in name_0 for name_1 in name_0}, **name_3.name_1):
        f''
        name_3 & name_3
        {}
        for name_1 in -name_5: # type: ignoresome text
            pass
        else:
            name_5
    f'' and '' and (not name_4) and f'' and name_2[name_4] and (name_2 := name_1)

    @([] for name_1 in name_1 if name_4 if name_1 async for name_3 in name_4 async for name_2 in name_2 async for name_5 in name_5 if name_4 if name_5 if name_1 async for name_1 in name_2 if name_1 if name_5 for name_2 in name_1 if name_3 if name_4)
    @[name_3() for name_0 in name_0 for name_5 in name_4 if name_1 if name_1 if name_2 if name_4 if name_5 for name_5 in name_2 if name_1 for name_2 in name_0 if name_0]
    @lambda name_5, name_1, name_4, name_3: name_3.name_2
    def name_3[*name_1](name_4: name_4.name_2, name_2: [name_1 for name_0 in name_0]=True, name_5: name_5 is name_4=f'{name_0!a:}True', name_3=not name_1) -> {f'': (name_5 for name_3 in name_0) for name_5 in name_1 if name_0 if name_1 if name_3 if name_5}: # type: ignoresome text
        while lambda *name_3: name_2:
            name_5
        else:
            name_3
            name_1
            name_0

        @13.071148552803532
        @{name_5 for name_5 in name_3}
        @name_0 & name_3
        def name_3[name_0, **name_4](name_3, name_2, name_0, /, name_5: name_0, name_4, *, name_1: name_1=name_5) -> f'': # type: ignoresome text
            name_0
            name_3
            name_1
            name_0
        [name_2 for name_4 in name_1]
    {name_0: name_4 for name_3 in name_3}[lambda *name_1: name_0] = name_5 = [][[name_1 for name_2 in name_5]] = name_4.name_1.name_1 = [] = (name_1 == name_1)[-name_4] # type: ignoresome text

    @not name_2 // name_0
    @(name_3 if name_1 else name_2) ** [name_3 for name_5 in name_1]
    async def name_0[name_1](*, name_0: (name_2 async for name_4 in name_5)=(name_0,), name_3: name_5={name_0, name_1, name_5, name_2, name_5}, **name_4) -> {name_0: name_5 for name_1 in name_1}({name_3 for name_3 in name_0}, lambda *name_5, **name_1: name_2, f'', b'', ()): # type: ignoresome text
        name_4()
except* name_5 and name_2 if f'' else name_5 not in name_2 as name_2:
    (name_3 async for name_0 in name_2)
except* not {*()}:
    import name_1 as name_4, name_3, name_5, name_3, name_4 as name_1, name_1 as name_0
    name_1 < name_3

    @name_2.name_2
    class name_5[**name_5, name_3, *name_2](lambda *name_5, **name_1: name_2, *name_3, **name_0, **name_3, name_0=name_4):
        raise
    [name_1 for name_1 in name_4]
except* (f'', name_2.name_0, [name_1 for name_2 in name_5]) as name_1:
    (name_2 async for name_0 in name_1)
    name_4 == name_1
    for [] in name_5 if name_1 else name_2: # type: ignoresome text
        pass
        name_2 //= name_0
        name_2
    else:

        def name_3(*name_4: name_1, **name_1): # type: ignoresome text
            pass
        name_2
        name_1: name_0
        name_1
        name_5
    name_5[name_5]
    () = name_0.name_1 = name_2[name_4] = lambda: name_4 # type: ignoresome text
except* {{name_5 for name_5 in name_3} for name_3 in name_4 for name_4 in name_3 if name_0 if name_0 for name_5 in name_3 if name_0 if name_4 if name_5} as name_2:
    name_1[name_4]
    name_0
    name_3 or name_4
    []
except* (name_0(), {*()}):
    raise
else:
    if {name_4 // name_1: name_5 if name_2 else name_2 for name_1 in name_0 if name_0 if name_2 if name_3 if name_1 if name_3 for name_4 in name_2 if name_0 for name_2 in name_2 if name_3 for name_4 in name_2 if name_4 if name_2 if name_5 if name_4 if name_4}:
        (name_4 := name_5)
        if {*()}:
            name_2
            name_0
        name_5.name_5
    else:
        name_1 // name_0
    del (), name_4[name_4][name_4 <= name_1]
finally:
    del name_1, (name_3.name_2, ()), [name_2], ([], [], [], name_4, []), {name_3: name_3 for name_1 in name_1}[''], ([], (), (), name_1[name_0], [])
    name_2().name_3
    (name_2 := [name_0 for name_4 in name_3])
    if {} == (name_0 := name_2):
        (name_5 := name_2)
        name_0()
    match {name_5: name_4 for name_1 in name_5} and {name_0 for name_2 in name_2} and (lambda *name_2, **name_5: name_2):
        case name_4(name_0=name_2.name_2, name_5=name_2.name_5, name_3=name_3.name_3):
            type name_5[*name_2, name_0, name_4: name_3, name_1] = name_3
            name_2 = name_2 = name_5 = name_5 = name_0 = name_2 = name_2 # type: ignoresome text
            pass

            def name_3(**name_5: name_4): # type: ignoresome text
                pass
                pass
                pass
                pass
                pass
                pass
            name_3
            name_5
        case ['\'"\'\'\'"""{}\\', None, b'\'"\'\'\'"""{}\\'] if [name_3 for name_4 in name_0]:
            name_1
        case '' as name_3 if name_5[name_1]:
            name_0
            name_2
            pass
        case {b'\'"\'\'\'"""{}\\': name_2.name_3, b'': None, 0.6002726008169712: name_3.name_0, False: False} if [name_0 for name_3 in name_1 for name_4 in name_0 for name_0 in name_0 for name_4 in name_2 for name_3 in name_0]:
            name_1
            (name_4): name_0 = name_2
            name_5
            name_4 //= name_1
lambda name_2, name_5, name_1=name_0.name_2, *, name_3=name_1 // name_4, name_4=name_0(name_4, name_0, name_0, name_0, name_0, **name_5, name_2=name_4, **name_0, **name_1, **name_3): [[name_0 for name_2 in name_1] for name_4 in name_0 for name_2 in name_3 for name_1 in name_0 if name_1 for name_3 in name_0 if name_3]
{[], f'', {{name_5 for name_0 in name_4} for name_2 in name_1 if name_1 if name_1 if name_4 for name_0 in name_3 if name_2 if name_5 if name_4 for name_0 in name_3 if name_1 if name_2 if name_3 if name_0 for name_1 in name_0 if name_3 if name_4 if name_2 if name_3 if name_2 for name_2 in name_1 if name_3 if name_4 for name_3 in name_4 if name_5}, (name_2 != name_2)(name_4, [], name_3 > name_5, *name_3)}

@name_3.name_5[{*()}] and {{}, [name_0 for name_0 in name_2], lambda *name_0: name_1, {name_0: name_1 for name_5 in name_0}, []} and f'' and name_4 and {name_2: name_3 for name_2 in name_4}((name_4 async for name_4 in name_4), (), name_1[name_1], **name_4, **name_2, name_5=name_3) and {~name_3 for name_3 in name_1 if name_5 if name_5 for name_4 in name_1 if name_0 if name_0 if name_2} and (f'', {*()}, name_2[name_0], (), name_2)
@().name_5 ^ ((name_1 async for name_1 in name_2) is not (name_0 async for name_4 in name_3) in (name_5 is not name_5) is name_5() > [] >= [name_0 for name_5 in name_1])
@not not [name_2 for name_0 in name_5]
@({name_2, {}}, (name_0 := name_1)({name_1: name_0 for name_3 in name_3}, name_4=name_1, name_5=name_0, **name_3), ~(lambda **name_4: name_4))
async def name_4[name_0, *name_5](name_1: {name_3 - name_2: (name_0 async for name_0 in name_2) for name_2 in name_4}, name_4: f'NoneFalse'={lambda **name_4: name_4 for [] in () if name_2 if [] if () if [name_0 for name_5 in name_2] for () in () for name_2[name_4] in (lambda *name_4: name_0) for [] in name_4[name_1]}, name_3: [].name_5={name_2 @ name_4, name_4 or name_1 or name_3 or name_1 or name_3}, /): # type: ignoresome text
    [{}, {*()}]
    lambda name_1, name_4, name_2, name_5, name_0, /: (name_4 := name_3)
[{name_5: name_3 for name_5 in name_2} for name_1, name_5, name_5, name_2 in {name_2 for name_2 in name_4 for name_4 in name_5} if (lambda **name_2: name_3) if {name_5: name_3 for name_1 in name_4 for name_5 in name_5 for name_2 in name_3} for name_4 in -name_1 if [name_2 for name_2 in name_0 for name_3 in name_5] if (name_1 if name_2 else name_1) if [] if name_5[name_1] for [] in name_2.name_5 if name_5 >> name_3 for name_4[name_1] in name_5 or name_4 or name_3 if name_5 and name_3 and name_3 and name_1 and name_3 if name_0 < name_4 >= name_2 is not name_5 < name_0 < name_1 if {name_3, name_2, name_5, name_3} for name_2[name_5] in name_2 if [name_4 for name_4 in name_0 for name_0 in name_2 for name_4 in name_1 for name_3 in name_5 for name_2 in name_0 for name_3 in name_2] for () in name_4 / name_3]
if b'some bytes'(-name_0, **name_4, **name_1, name_1=name_0, name_3=name_1, **name_2) and {*()} is not +name_1 and (lambda name_3, name_2, /, name_4, name_1, *, name_0=name_1: name_0()) and {f'', (name_4 := name_5)}:
    [~name_3 for name_5 in name_2 if name_0 for name_4 in name_3 if name_4 for name_3 in name_4 if name_4 if name_5 for name_4 in name_3 if name_3 if name_1 if name_4 if name_4 for name_1 in name_3 if name_0 if name_2 if name_5 for name_1 in name_2 if name_4 if name_2 if name_1]
    {name_4 for name_2 in name_2} if '' else ~name_4
    True
    assert f'True'
else:
    assert lambda name_3, /, name_4, name_1, **name_0: {name_1: name_4 for name_4 in name_2}
    match [name_4 if name_0 else name_2, not name_5]:
        case name_0(name_4='\'"\'\'\'"""{}\\', name_5=b'some bytes'):
            name_1
            name_5
            name_4
            name_3
        case -1 | name_0.name_4 | name_4.name_1 | 'some const text' | -1.9200319491979312 | 'some const text' if name_0.name_3:
            from name_5 import name_2, name_4
            name_2
            while name_0:
                pass
                pass
            else:
                pass
                pass
    (f'' for name_5 in name_5 if name_0 if name_4 async for name_1 in name_4 if name_4 if name_3 if name_1 if name_5)
    (name_4 if name_0 else name_4).name_1
import name_5, name_4 as name_5, name_1
name_1 >>= ([name_3 for name_5 in name_4] async for name_1 in name_0 if name_4 if name_3 if name_0 if name_5 if name_3 for name_5 in name_4 if name_4 if name_3 if name_3 async for name_2 in name_3 if name_1 if name_0 if name_2 if name_2 for name_0 in name_2 if name_1 if name_0 async for name_3 in name_4 if name_4 async for name_1 in name_4) - [[name_5 for name_1 in name_2] for name_2 in name_0 if name_1 if name_1 if name_5 for name_1 in name_1 for name_1 in name_2 if name_5]
[f''[name_0()] for name_4.name_3 in name_4 or name_3 or name_2 or name_3 or name_5 if (lambda **name_1: name_3) for [name_3, name_3] in (name_1 for name_2 in name_5 async for name_5 in name_0 for name_5 in name_2 async for name_5 in name_1)]
(name_1 := (name_3 := (name_0 if name_0 else name_1)))
name_3().name_1()
()
(name_4 if name_3 else name_1).name_1 >= name_2 * (name_4 := name_1)

I've been waiting several minutes for this branch of red-knot to finish checking that file, but no joy yet.

[carljm]

I know I shouldn't be looking at draft PRs

One of the reasons for which, is that whatever time you spent narrowing that down to seed 120 is duplication of the time I spent doing the same thing last night 😆

[carljm]

I will note this branch fixes a lot of currently panicking fuzzer seeds (something the current structure of the fuzzer job doesn't inform us), but yeah it does add two new ones, and make one much slower; the former might be acceptable to defer to a follow up, the latter needs to be fixed in some way just so we don't slow down the fuzzer job.

[MichaReiser]

I'm slightly scared from adding new fixpoint features before we fixed all upstream issues in salsa.

[AlexWaygood]

One of the reasons for which, is that whatever time you spent narrowing that down to seed 120 is duplication of the time I spent doing the same thing last night 😆

didn't take that long 😛 I just looked to see what the final reported seed was in CI, saw it was 120, and then ran this in the terminal to get the code generated from seed 120:

% uv run --no-project --with=pysource-codegen python
Python 3.13.1 (main, Jan  3 2025, 12:04:03) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pysource_codegen import generate
>>> repro = generate(120)
>>> with open("repro.py", "w") as f:
...     f.write(repro)

[carljm]

I'm slightly scared from adding new fixpoint features before we fixed all upstream issues in salsa.

That's reasonable. I was aiming to knock down more of the easier panics to eliminate that are maybe more likely to hit user projects in alpha testing, but if at this point the upstream bugs are hitting us commonly enough for alpha use cases, I can put this on ice and work on the Salsa issues instead.

[carljm]

I just looked to see what the final reported seed was in CI

Nice! It took me slightly longer since I hadn't pushed the PR yet and didn't have the completed run from CI to look at, so I had to parse the output, extract the seeds that did complete, sort them, and then find the missing one :)

then ran this in the terminal

And I didn't know how to do all this, so instead I just modified fuzz.py to print out its initial source snippet before testing it.

[carljm]

To be clear, the motivation of this PR is to fix cycle panics in the ecosystem and the fuzzer (and it does that effectively); we're not just using fixpoint iteration to simplify code here.

We do now rely on fixpoint iteration of try_mro and try_metaclass to detect cyclic inheritance instead of the inheritance_cycle query, but this doesn't really increase the overall use of fixpoint, since the removed inheritance_cycle query itself had fixpoint iteration support too! The problem is just that inheritance_cycle was not actually sufficient to catch all inheritance cycles, so we still need the fixpoint iteration support in try_mro and try_metaclass -- at which point it doesn't really make sense to keep around a separate query that just catches some cycles, which we'll catch in the other queries anyway.

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

ℹ️ ecosystem check encountered format errors. (no format changes; 1 project error)

mesonbuild/meson-python (error)

warning: Detected debug build without --no-cache.
error: Failed to read tests/packages/symlinks/baz.py: No such file or directory (os error 2)
error: Failed to read tests/packages/symlinks/qux.py: No such file or directory (os error 2)

Formatter (preview)

ℹ️ ecosystem check encountered format errors. (no format changes; 1 project error)

mesonbuild/meson-python (error)

ruff format --preview

warning: Detected debug build without --no-cache.
error: Failed to read tests/packages/symlinks/baz.py: No such file or directory (os error 2)
error: Failed to read tests/packages/symlinks/qux.py: No such file or directory (os error 2)

[AlexWaygood]

nit: this is sort-of buried in the middle of the script here. Maybe add a SKIPPED_SEEDS: Final = frozenset({120}) global constant?

[carljm]

Oops, I missed this suggestion... probably won't do a separate follow-up just for this but will keep it in mind when I come back to this TODO.

[AlexWaygood]

Oh -- it looks like the "fuzz for new ty panics" CI job is still taking significantly longer on this branch? The latest run here finished in 16 minutes, but it takes around 1 minute on main

[AlexWaygood]

Oh -- it looks like the "fuzz for new ty panics" CI job is still taking significantly longer on this branch? The latest run here finished in 16 minutes, but it takes around 1 minute on main

But I can't reproduce this locally. Not sure what's going on here. Could you try rebasing the PR on main and see if it still occurs?

[AlexWaygood]

FWIW, I applied these changes locally the script, and planned to CTRL-C when I hit the hang so that the script would print out the list of seeds that it never finished running... but I never hit the hang locally.

diff --git a/python/py-fuzzer/fuzz.py b/python/py-fuzzer/fuzz.py
index bae2e027f1..eb57ee3819 100644
--- a/python/py-fuzzer/fuzz.py
+++ b/python/py-fuzzer/fuzz.py
@@ -215,6 +215,7 @@ def run_fuzzer_concurrently(args: ResolvedCliArgs) -> list[FuzzResult]:
         f"file{'s' if num_seeds != 1 else ''}..."
     )
     bugs: list[FuzzResult] = []
+    successes: list[FuzzResult] = []
     with concurrent.futures.ProcessPoolExecutor() as executor:
         fuzz_result_futures = [
             executor.submit(fuzz_code, seed, args) for seed in args.seeds
@@ -228,10 +229,14 @@ def run_fuzzer_concurrently(args: ResolvedCliArgs) -> list[FuzzResult]:
                     fuzz_result.print_description(i, num_seeds)
                 if fuzz_result.maybe_bug:
                     bugs.append(fuzz_result)
+                else:
+                    successes.append(fuzz_result)
         except KeyboardInterrupt:
             print("\nShutting down the ProcessPoolExecutor due to KeyboardInterrupt...")
             print("(This might take a few seconds)")
             executor.shutdown(cancel_futures=True)
+            hanging_seeds = set(args.seeds) - {result.seed for result in (bugs + successes)}
+            print(f'hanging seeds are: {hanging_seeds}')
             raise
     return bugs
 

[carljm]

There were two more fuzzer seeds that were hanging or taking a long time (one did finish after a long time, the other didn't finish within the 20 minute timeout). Both are also seeds that were previously just cycle-panicking, so I don't have any reason to think at this point that this PR causes the hang/slowness on those seeds (it doesn't seem to cause any new hangs on currently-good ecosystem projects); it might just expose it by getting us past the panic on those seeds. For now I've skipped those seeds on the fuzzer as well.

I still think this is worth landing as-is to reduce cycle panics; I'll look at the hanging fuzzer seeds when I can, though I'm more motivated to look first at hangs occurring on ecosystem projects.