Python: Support PEP 735 [dependency-groups] in pyproject.toml

[vrslev]

Is there an existing issue for this?

• I have searched the existing issues

Feature description

uv recently released new version that supports now standardized [dependency-groups] table. See uv 0.4.27 and PEP 735.

[KyeRussell]

Additional context to hopefully nudge this one along: this is a dependency declaration standard being used right now by at least one Python package / project manager with a significant userbase (uv via uv add --dev ...). Parsing this is essentially the same as parsing [project.optional-dependencies], which dependabot can already handle. It's be great to see this included!

[dpgraham4401]

It's also used by PDM. Our team encountered error from dependabot if there were multiple groups listed under the dependency-groups section.

Dependabot can't parse your pyproject.toml
Dependabot failed to update your dependencies because there was an error parsing the pyproject.toml found at /pyproject.toml.

Dependabot encountered the following error:

/pyproject.toml not parseable

The logs do not include additional information about the parsing error. We were able to troubleshoot by placing all dependency-groups into one dev group.

EDIT: after placing all development dependencies into one dev group under the [dependency-groups], dependabot stopped erroring but actually didn't successfully start creating PRs. We ended up moving to the uv pip interface and optional-dependencies for the time being.

[evanbernstein]

I really hope that Dependabot is updated to support dev dependencies for modern python tooling. My codebases currently use two package managers, one for dependabot and the real one, which I have to update manually when dependabot pings us to update the old one. It is gross. I'm so glad that the python community is landing on a standard, and I hope it gets implemented soon!

[AA-Turner]

The pip project has just merged support for the PEP 735 [dependency-groups] table (pypa/pip#13065), which will likely be released in April.

[markhallen]

We have raised an issue for the toml-rb gem.

If they don't have a quick fix for this we have the option to use tomlrb or perfect_toml:

# minimal.toml
[dependency-groups]
dev = [
    {"include-group" = "docs"},
    "deptry>=0.15.0",
]
docs = [
    "furo>=2024.5.6",
]

irb(main):005:0> Tomlrb.load_file("minimal.toml")
=> {"dependency-groups"=>{"dev"=>[{"include-group"=>"docs"}, "deptry>=0.15.0"], "docs"=>["furo>=2024.5.6"]}}
irb(main):006:0> PerfectTOML.load_file("minimal.toml")
=> {"dependency-groups"=>{"dev"=>[{"include-group"=>"docs"}, "deptry>=0.15.0"], "docs"=>["furo>=2024.5.6"]}}

[markhallen]

This is closed for uv, but I need to test the handling for other python package managers before closing.

[evanbernstein]

@markhallen Thanks for doing this. If I'm reading this correctly, if I switch from rye to uv then Dependabot will work correctly for all packages? If so, I've found a new project for today.

[rslinckx]

I can't find the code that looks for dependency-groups in dependabot, according to the comment above it seems to have been fixed, but what was fixed is only the parser understanding the toml, not the actual extraction of dependency-groups ?

[abdulapopoola]

@rslinckx , here is the PR: #11796

[rslinckx]

As you can see here: https://github.com/dependabot/dependabot-core/blob/main/uv/helpers/lib/parser.py#L58 the only sections that are parsed are dependencies and optional-dependencies. I'm not even sure why the ruby gem toml has any relevance since it's python code reading and parsing the pyproject.toml anyway? (the call site is around here: https://github.com/dependabot/dependabot-core/blob/main/uv/lib/dependabot/uv/file_parser/pyproject_files_parser.rb#L93 )

[MusicalNinjaDad]

The issue fixed by #11796 was that the updater, which uses rb-toml, errored out as described in this comment

[rslinckx]

Unfortunately the uv ecosystem still doesn't look for the legacy [tool.uv].dev-dependencies or the new [dependency-groups].dev keys, so the UV updater cannot possibly update dev dependencies nor present them as dev dependencies according to dependabot filtering/grouping rules