Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for uv.lock #841

Open
hynek opened this issue Aug 25, 2024 · 30 comments
Open

Support for uv.lock #841

hynek opened this issue Aug 25, 2024 · 30 comments

Comments

@hynek
Copy link
Contributor

hynek commented Aug 25, 2024

How would this feature be useful?

uv added a cross-platform lock format in 0.3.0. I can see this to become a standard for packages to keep CI stable.

Given that uv sync doesn't have target-installations (yet), it's tricky

Describe the solution you'd like

I think it would be really cool if session.install(".[tests]") automatically discovered a uv.lock file and ran uv sync --extra tests but I could live with session.sync("extras") too.

Either would improve the overall Python DX tremendously.

Describe alternatives you've considered

Hand-craft it, I guess? One has to do manual change directories etc tho.

Anything else?

No response

@jamesharris-garmin
Copy link

Would astral-sh/uv#6834 support this use case? It adds UV_PROJECT_ENVIRONMENT, I guess, once that lands, assuming nox provides a way to access the virtualenv path, you could implement a session that sets the uv environment to the specified nox location.

One thing to note about this is they expect to add a warning when VIRTUAL_ENV is set in a follow up PR: astral-sh/uv#6864. But even this only warns if UV_PROJECT_ENVRIONMENT and VIRTUAL_ENV don't match. I'll be looking forward to that support happening.

@hynek
Copy link
Contributor Author

hynek commented Sep 4, 2024

JFTR UV_PROJECT_ENVIRONMENT has shipped.

@jamesharris-garmin
Copy link

seems like we could just make sure to export UV_PROJECT_ENVRIONMENT and set it equal to the virtualenv then you could call uv sync as a run_install call.

@jamesharris-garmin
Copy link

After experimenting with this it kind of breaks to do this via uv sync and this envrionment variable. uv sync destroys and recreates the environmetn every time if nox modifies it. so we can't just "reuse' the build environment. Maybe a better aproach would be to automatically generate a requirements file from the lock file and re-install it.

@hynek
Copy link
Contributor Author

hynek commented Sep 6, 2024

That seems a bit convoluted, it would make more sense to write up a proper bug report to Astral and ask them – they're usually quite receptive.

What exactly is nox doing there? Could it already be fixed using uv sync --inexact? Also, uv is so fast, venv reuse doesn't matter at all.

@jamesharris-garmin
Copy link

I haven't had a chance to dig too far into the behavior to see how this is supposed to work. I don't consider it a bug just yet. maybe nox is using a different python version than I expect?

@edgarrmondragon
Copy link
Contributor

edgarrmondragon commented Sep 9, 2024

nox-poetry does that something similar for poetry.lock, so maybe what we need is an extension like that for uv?

cc @cjolowicz

@henryiii
Copy link
Collaborator

henryiii commented Sep 9, 2024

Tangent: We don't really support plugins here - ideally, you nox would be able to install it's own plugins, rather than assuming people can inject an extra plugin into wherever nox is installed. I'd like to eventually support these better.

@Wurstnase
Copy link
Contributor

I would like to contribute uv sync. Should I simple create a PR with my ideas, @henryiii?

@henryiii
Copy link
Collaborator

henryiii commented Oct 6, 2024

Could you describe your ideas first? You can start with a PR if that's easier. I was sort of waiting until the dependency groups and standardize lock file PEPs get resolution.

@Wurstnase
Copy link
Contributor

Uh, I didn't realize that there is a new PEP1 for lock files.
I just played around on my local project and this PR is what I've ended with: #857

I'm using the workspace feature from uv. So I need the --package option a lot.

Footnotes

  1. PEP 751

@dariocurr
Copy link

Any news on this?

@Wurstnase
Copy link
Contributor

My current prefered way is to use something like from the cookbook: https://nox.thea.codes/en/stable/cookbook.html#using-a-lockfile

@henryiii
Copy link
Collaborator

I think the environment feature will likely be pushed to the next release, but it still is the next major thing I want to work on.

Recipes like the cookbook one should be fine until then.

@johnthagen
Copy link
Contributor

Could you describe your ideas first?

As @edgarrmondragon mentioned above, for me the use case I'm looking for is similar to what nox-poetry provides

Specifically, any time Session.install() is called, the versions to be installed will be constrained to whatever is listed within uv.lock. Often times, we only want/need to install a subset of the dependencies (think, for unit testing we only want to install the main + test dependencies).

This is also where supporting dependency groups would come in

So that we could do something like:

def test(session: Session):
    session.install(".", groups=["test"])
    ...

And this would install the main dependencies and the test dependency group that lists pytest, pytest-cov, etc all with locked versions from uv.lock so everything is reproducible and there is no DRY issues (e.g., having to list dependency lists twice between pyproject.toml and noxfile.py).

@jamesharris-garmin
Copy link

I mean this is certainly more ergonomic than what I do but I just wrote an installer method that runs uv sync --frozen --inexact with the --extras and --groups flags required to install the combination of dependency groups/extra requirements I need to support the task I need at any given time.

@johnthagen
Copy link
Contributor

As existing work, tox-uv supports this concept for tox

@edgarrmondragon
Copy link
Contributor

Tangent: We don't really support plugins here - ideally, you nox would be able to install it's own plugins, rather than assuming people can inject an extra plugin into wherever nox is installed. I'd like to eventually support these better.

@henryiii is there an issue to track plugin support?

@henryiii
Copy link
Collaborator

Plugin support is implemented in #881 but it's been stuck waiting for a review since October.

@johnthagen
Copy link
Contributor

Plugin support is implemented in #881 but it's been stuck waiting for a review since October.

@henryiii Now that the reference MR is merged

Do you have recommendations for how can move tighter uv integration for Nox forward? Is a separate nox-uv package similar to nox-poety the preferred way, or, given the growing popularity of uv, would Nox upstream more specific support for uv.lock files?

@henryiii
Copy link
Collaborator

henryiii commented Mar 3, 2025

I think there are two possibilities. One would be to do a plugin, much like nox-poetry. (And that plugin should update examples to show how to use with the current system). That could be done at any time.

The other is work I have planned (but haven't/wont start for a while), which would provide a new "environment" concept, where you could make an environment then define sessions on it. Combining that with lock support makes sense, IMO, and since uv is an integrated backend, it could be natively supported there. What I was thinking was something like this:

@nox.env(path=".venv")
def dev(env: nox.Env) -> None:
    env.install("some", "deps")

@dev.session
def test(session: nox.Session) -> None:
    session.run("pytest")

And native locking support could be something like this:

@nox.env(path=".venv")
def dev(env: nox.Env) -> None:
    env.sync()

...

@d-miketa
Copy link

d-miketa commented Mar 6, 2025

Given uv's native support for transient environments is there even any need for sessions at all? I've been able to run Nox with a disabled Python installation using just uv run. You get full venv isolation without any redundant packages. As far as I can tell it all just works out of the box.

@johnthagen
Copy link
Contributor

johnthagen commented Mar 21, 2025

@henryiii I've been working on porting my projects from nox-poetry to nox with uv. I wanted to suggest that it could be nice in the Nox docs to have a consolidated "uv Guide". Currently I had to hunt for various bits in formation in a bunch of locations. I'm trying to figure out what the total current "best practices" are for Nox + uv. So far what I've been able to piece together:

nox.options.default_venv_backend = "uv|virtualenv"

I'll try to update this post for others as I go in case it's helpful.

@henryiii
Copy link
Collaborator

The only other one I can think of directly is sometimes you need session.venv_backend == "uv" to check. Most of my noxfiles (see my projects at iscinumpy.dev or on my GitHub profile readme, about 90% use nox) use uv. I mostly avoid lock files, since I mostly work on libraries.

@johnthagen
Copy link
Contributor

johnthagen commented Mar 24, 2025

Another nox uv feature that I discovered:

You cannot yet configure this (e.g., cannot opt out), however, and that is tracked in

@dantebben
Copy link

I have created a small tool that may be useful to others wanting to facilitate using nox with uv. https://pypi.org/project/nox-uv/

@henryiii
Copy link
Collaborator

henryiii commented Apr 7, 2025

Reminder, you can include the new PEP 723 support in your docs on how to use your plugin. If you have

/// script
dependencies = ["nox>=2025", "nox-uv"]
///

The nox will download your plugin automatically before running!

@jamesharris-garmin
Copy link

I noticed that the new PEP 723 support creates the python environment from whichever python is python in your path. This is a problem for a system where the default python is something like 3.8 which unfortunately is true for my work machine. Do we have an issue to track supporting the requires-python directive in a metadata script block?

@henryiii
Copy link
Collaborator

henryiii commented Apr 8, 2025

We need to support the Python range statement first (then it would be easy). #814

@jamesharris-garmin
Copy link

alright i'll keep an eye on that ticket then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

9 participants