"transitive-package-only-prereleases-in-range" doesn't follow the spec

[notatallshaw]

The scanrio is the following currently:

{
    "name": "transitive-package-only-prereleases-in-range",
    "description": "The user requires package `a` which has a dependency on a package which only matches prerelease versions but they did not include a prerelease specifier.",
    "root": {
        "requires": [
            "a"
        ]
    },
    "packages": {
        "a": {
            "versions": {
                "0.1.0": {
                    "requires": [
                        "b>0.1"
                    ]
                }
            }
        },
        "b": {
            "versions": {
                "0.1.0": {},
                "1.0.0a1": {}
            }
        }
    },
    "expected": {
        "satisfiable": false,
        "explanation": "Since there are stable versions of `b` available, the prerelease version should not be selected without explicit opt-in. The available version is excluded by the range requested by the user."
    }
},

In short, the idea is because a prerelease requirement is not opted into then even though a prerelease version is available it is not selected.

However, this is explictly again the following line in the spec (https://packaging.python.org/en/latest/specifications/version-specifiers/#handling-of-pre-releases):

By default, dependency resolution tools SHOULD:

• ...
• accept remotely available pre-releases for version specifiers where there is no final or post release that satisfies the version specifier
• ...

In this case there is no final or post release that satisfies the version specifier, and therefore the dependency resolution SHOULD accept the removetely available pre-release.

Currently neither uv nor pip follow the spec here, pip needs an overhaul on prereleases and uv has deicded not to follow the spec due to design choices, but poetry does follow the spec and does find this satisfiable.

If packse is attempting to follow the spec, and not be a catalogue of uv's design choices, this should be set to "satisfiable": true and uv should mark this as XFAIL. I'll be happy to make a PR.

[konstin]

The exact meaning of this paragraph is hard to interpret because it was written before backtracking resolver were a thing in python and the main concern was backwards compat to pre-PEP 440 versions. From Donald Stufft:

As the one of the original author of PEP 440, I can say pretty definitively that the reason that the PEP says this:

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

and not this

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system or explicitly requested by the user.

Is because there were widely used packages at the time that had versions which didn’t technically comply with PEP 440 (because they predated PEP440) and got interpreted as prerelease versions, and we were trying to prevent breakages (a lot of effort in PEP 440 and a lot of weird edge cases came from trying to gain another 1%-2% of compatibility with existing versions on PyPI). One such package at the time was pytz which used versions like 2005a.

There were also a number of packages that were only available as pre-releases (for one reason or another) and we wanted to avoid additional breakages.

At the time of PEP 440, none of the Python installers had any sort of real dependency resolving or backtracking or anything of that nature, so the implications of that statement, wrt to how it impacted backtracking/resolving simply wasn’t a thing that existed to be considered. I suspect this is why (1) from above is built into the Finder instead of into the resolver as well.

From a user perspective, i think the current behaviour makes sense, at least in uv i wouldn't want to fix it. (Not to say that uv's prerelease behaviour doesn't need it improving)

[notatallshaw]

Yes, I started that thread, and I don't think it's relevant here because no backtracking is occuring.

This is a direct application of the specifier to the versions available, I don't believe there is any ambiguity.

Users also seem happy with poetry's behavior, and it complies with the spec here.

I will say, pip maintainers have said it's not required because of the word SHOULD, but I assume packse doesn't want to add scenarios that contradict the SHOULDs in the spec.

[notatallshaw]

This is a more fundamental issue with packse's current scenarios, that they are aligned closely with uv's design goals and not the actual spec.

I think for other tools to adopt packse there should be a clear seperation of spec scenarios, the enforcement level of the spec, and design scenarios, perhaps by including a flag.

But if that isn't itself a design goal of packse, I understand. I think discussion on that would require it's own issue, I don't think leaving this issue open is helpful.