Skip to content

Using uv run as a task runner #5903

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
my1e5 opened this issue Aug 8, 2024 · 172 comments
Open

Using uv run as a task runner #5903

my1e5 opened this issue Aug 8, 2024 · 172 comments
Labels
enhancement New feature or improvement to existing functionality

Comments

@my1e5
Copy link
Contributor

my1e5 commented Aug 8, 2024

For those of us migrating over from Rye, one of its nice features is the built-in task runner using rye run and [tool.rye.scripts]. For example:

[tool.rye.scripts]
hello = "echo Hello from Rye!"
$ rye run hello
Hello from Rye!

It could have some more features - here is a selection of feature requests from the community:

A lot of these requested features are things that other 3rd party tools currently offer. I thought it might be useful to highlight a few other tools here, in particular because they also integrate with the pyproject.toml ecosystem and can be used with uv today.

Perhaps these can serve as some inspiration for a future uv run task runner and also in the meantime offer a solution for people coming over from Rye looking for a way to run tasks.

@chrisrodrigue
Copy link

Relevant comment from another issue: #5632 (comment)

@chrisrodrigue
Copy link

PDM supports this: https://pdm-project.org/latest/usage/scripts/

@charliermarsh charliermarsh added enhancement New feature or improvement to existing functionality preview Experimental behavior labels Aug 9, 2024
@charliermarsh
Copy link
Member

Yeah we plan to support something like this! We haven't spent time on the design yet.

@nikhilweee
Copy link

The pyproject standard already supports [project.scripts], so uv may not need to use its own table.
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#creating-executable-scripts

@charliermarsh
Copy link
Member

[project.scripts] is a little different -- that's used to expose specific Python functions as executable scripts, and we do support that already.

@chrisrodrigue
Copy link

Perhaps naming this section tool.uv.tasks or tool.uv.aliases could help disambiguate that.

@nikhilweee
Copy link

nikhilweee commented Aug 10, 2024

Or maybe [tool.uv.run] to be consistent with the command uv run. Or we could even think about [tool.uv.commands].
I'm not a big fan of [tool.uv.scripts] since it conflicts with [project.scripts] and I myself got confused before.

@cdwilson
Copy link

This is the main thing I missed coming from hatch: https://hatch.pypa.io/dev/config/environment/overview/#scripts

@chrisrodrigue
Copy link

chrisrodrigue commented Aug 23, 2024

+1 to @nikhilweee suggestions. I think “command” reflects the intent/concept.

Hatch has an “environment” concept and supports running commands namespaced to an environment like so

hatch run test:cov

where “test” is a user-defined environment (with a dependency group) and “cov” is a user-defined command for that environment.

[tool.hatch.envs.test]
dependencies = [
 "pytest",
 "pytest-cov",
 "pytest-mock",
 "freezegun",
]

[tool.hatch.envs.test.scripts]
cov = 'pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=src'

[[tool.hatch.envs.test.matrix]]
python = ["3.8", "3.9", "3.10", "3.11", "3.12"]

I would be curious to hear the use cases of nesting dependency groups and commands into “environments” like this rather than defining them at the top-level (i.e. [tool.uv.commands]/[tool.uv.dev-dependencies]).

@pietroppeter
Copy link

since it has not been mentioned yet, adding as a possible inspiration for design of tasks also pixi: https://pixi.sh/latest/features/advanced_tasks/

@metaist
Copy link

metaist commented Aug 28, 2024

I happen to be writing a cross-project task runner that supports a bunch of formats (e.g. rye, pdm, package.json, Cargo.toml; even uv's workspace config).

For what it's worth, almost all python runners use tool.<name>.scripts for task config (presumably inspired by npm's package.json format), so it's somewhat of an easier upgrade path for people coming from other tools.

https://github.com/metaist/ds

@inoa-jboliveira
Copy link

Also related to this thread, I wish uvx was uv run instead of uv tool run when inside a project.

uv tool run seems like something you would do to play with a tool, like ruff. But then you will eventually uv add ruff --dev and forever keep writing uv run ruff check instead of uvx ruff check which won't respect the locked version and will be in different virtualenv. It also means the tool could be running on a different Python (does not apply to ruff, but any other python lib) and all sorts of weird stuff can happen.

I know uv run stuff is still short, but it could be 4 keystrokes shorter.

Regarding uv run being a task runner it means people will type it waaaaay more often than uv tool run.
I would appreciate a dedicated command like uvr

@zanieb
Copy link
Member

zanieb commented Sep 8, 2024

@inoa-jboliveira I opened a dedicated issue for that #7186

@nikhilweee
Copy link

Putting together some thoughts about semantics. This issue is about adding support for running arbitrary instructions specified in pyproject.toml. I deliberately use the term instruction to avoid using any of the other terms under consideration (command, tool, script, etc).

What do we call these instructions?

Lots of existing tools refer to them as "scripts".

  1. npm and yarn have first class support for scripts. Users can define them in package.json
  2. composer also uses the same term. Users can define them in composer.json
  3. pdm takes inspiration from npm and also uses the term scripts. Users can define them in [tool.pdm.scripts]
  4. rye follows suit. Custom scripts are defined in [tool.rye.scripts]
  5. hatch also uses the term scripts, although they are tied to environments. Defined in [tool.hatch.envs.<env>.scripts]

It seems advantageous to just go with the term "scripts" because it is the de-facto standard. As noted by another user #5903 (comment), this would also reduce friction for users coming to uv from other package managers. That said, this approach has a major flaw because it overlaps with the concept of entry points defined in [project.scripts]. Entry points expose certain python functions as executable scripts, but do not allow arbitrary commands. Furthermore, [project.scripts] has already been established in PEP-0621, as the official spec. So what about other terms?

Another option is to use the term "tasks"

  1. pixi uses the term "tasks". Users can define them in the [tasks] table in pixi.toml
  2. bundler uses rake tasks. Although it resembles entry points, sh is supported.
  3. grunt and gulp also use the term "tasks". Although they are task runners, not package managers.
  4. gradle uses the term "tasks", defined in build.gradle

Another option is to call them "executables". dart uses this term in pubspec.yaml

We could also use "commands", although I wasn't able to find existing tools which use this term.

After settling on a name, an obvious thing to do is to let users define instructions in the [tool.uv.<name>] table.

How do we invoke these instructions?

There are two options here.

  1. Overload existing uv run <instruction> (follows from npm run <script>)
  2. Add a new subcommand uv invoke / uv command / uv task

How should these instructions be specified?

PDM's documentation around user scripts is pretty evolved, with support for a bunch of features.

  1. cmd mode, shell mode, composite mode
  2. Specify env vars and env files for each script
  3. Specify working dir and site packages for each script
  4. Specify order of arguments
  5. Specify pre and post scripts
  6. call a function from a python script (entry point)

Rye has its own format, which is a subset of PDM features.

  1. Specify env vars and env files for each script
  2. chain multiple scripts one after the other
  3. call a function from a python script (entry point)

I hope this serves as a starter for discussing additional details for this feature.

@charliermarsh
Copy link
Member

(Nice comment, thank you!)

@Xdynix
Copy link

Xdynix commented Sep 10, 2024

One nice thing about PDM is that if a command is not recognized as a built-in, it is treated as pdm run. Thus, pdm foobar would be shorthand for pdm run foobar, which executes the command defined in [tool.pdm.scripts.foobar].

@gdamjan
Copy link

gdamjan commented Sep 10, 2024

IMHO, pdm has the most extensive support for these scripts and I would personally like to see the same support in uv too. And if so, best to support the same pyproject section too 😱. It's especially nice when one doesn't have to use any other tools like Makefiles (ugh).

Alas, one can argue that the pdm support for scripts is feature-creep for uv, in which case the rye model works too :)

@patrick91
Copy link

Furthermore, [project.scripts] has already been established in PEP-0621, as the official spec. So what about other terms?

I wonder if would be a good time to maybe standardise this? I don't know if a pep is required, but since we have some many package managers for python it would be nice if we can have one way to define these instructions

@humanzz
Copy link

humanzz commented Mar 11, 2025

Late to this thread, but I really like the idea of tasks.py from #5903 (comment) as it's similar to my current usage of https://github.com/pyinvoke/invoke which also uses a tasks.py where I run all that I need via

  • inv <task name> if I wanna run a specific task
  • inv to run the default task which I typically configure to run the several tasks e.g. installing dependencies (e.g. uv pip install as am still using the pip interface), linting checks, running the tests, etc.

It'd be great if uv can provide some similar functionality, and a command - as suggested in other comments e.g. uvr or even uv run to allow for similar functionality.

Example snippets from tasks.py

...

@task
def lint(ctx):
    """
    Run linting checks using ruff and mypy
    """
    ctx.run("ruff format --check src test tasks.py", echo=True)
    ctx.run("ruff check src test tasks.py", echo=True, pty=True)
    ctx.run("mypy src test tasks.py", echo=True, pty=True)


@task(name="lint:fix")
def lint_fix(ctx):
    """
    Fix linting issues using ruff
    """
    ctx.run("ruff check --fix src test tasks.py", echo=True, pty=True)
    ctx.run("ruff format src test tasks.py", echo=True)

@task
def develop(ctx):
    ctx.run(f"uv pip install -r {LOCK_FILE} -e .", echo=True)

...

@task(default=True, pre=[develop, lint, test, build_docs, publish])
def release(ctx):
    """
    Release the package
    """
    pass

@carlosferreyra
Copy link

I've been following this thread, but now I wanted to give a newbie perspective to the community.

I've been playing with npm and having a "scripts" section as an out-of-the-box feature, it's a massive gain in terms of maintainability and ease of development.

analog to this feature, I would agree that uv can add support to something similar to that.

I would follow this example

[tool.uv.run|aliases|commands]
default = "uvx --from=rust-just just"         # uvr
lint = "uv run scripts/lint.py"               # uvr lint
test = "echo 'hello from uv'"                 # uvr test

or maybe even not add a uvr but make uvx to search aliases first, before fetching from the online packages.

@Quidge
Copy link

Quidge commented Mar 19, 2025

I've also been subscribed/following this feature for awhile. I ended up discovering how to install a project as a package, which gives you [scripts] support and that solves my primary use case.

But WOW for someone new learning how to set that up it would be rough. There have been so many iterations on python packaging that you really need to understand a lot to get it working. It also seems uncommon to set up python application projects (not distributable libraries) up like this in the first place, so you're not going to see mention of it in various tutorials.

For some skill context, I'd consider myself an intermediate Python developer who has spent the last several years only in web dev projects. No distributable libraries. It's been bizarre to me that I got this far without knowing about packaging, but I wouldn't be surprised if the large majority of Python devs are in the same boat.

The package.json scripts type approach is far easier for a newbie to understand IMO. Not sure if it's worth the complexity/scope, but I definitely understand the want.

(side note, I cannot emphasize enough how grateful I am to the work being done on uv -- bundling Python installation and dependency management into one tool is an absolute godsend for reducing surface area of issues across teammates)

@AtticusZeller
Copy link

AtticusZeller commented Mar 20, 2025

i find this feature request rather misguided. UV is an excellent environment and project management tool, but introducing such trivial functionality is unnecessary and counterproductive.

as people mentioned before, the request aims to address cross-platform issues, but handling these collaboration challenges is a fundamental skill any competent developer should possess. Features that don't bring substantial productivity improvements only burden the maintainers while encouraging poor development practices.

Python is renowned for its philosophy of explicitness—In keeping with Python's commitment to clarity, developers should understand the specific scripts they're working with instead of hiding them behind excessive layers of abstraction.

I'm commenting because I can't stand seeing discussions about features that won't bring actual benefits and only create more work for maintainers. Does this issue really affect your actual efficiency when using the uv tool?

btw, I love uvx ,it does be useful and clean,

uxr is the most stupid thought I've ever seen.sorry bro

uvx is enough man

be appreciate on the current UV and focus on the actual and practical features please

@Lordfirespeed
Copy link

Lordfirespeed commented Mar 20, 2025

i find this feature request rather misguided

@AtticusZeller you aren't a member of astral-sh nor have you contributed to uv.
Your comment, in summary: "I don't want this feature, so it would be a pointless burden on the maintainers."

Other users of uv have expressed they want this feature.
Other ecosystems have analogous features.
Maintainers have expressed they are interested in supporting this feature. (ref)

[this feature] won't bring actual benefits

No benefits for you does not imply no benefits for anyone/everyone. That is a fallacy.

Your comment does not meaningfully contribute to the discussion at hand. Please refrain from commenting like this on GitHub.

@dandavison
Copy link

dandavison commented Mar 20, 2025

I'd like to add shell completion to this design discussion. There was a quite a bit of discussion earlier in the thread about minimizing typing via aliases and the uvr short form, but good shell completion would have an even bigger impact on typing convenience and discoverability.

I'm not sure how it should work, but if it were possible to do something like eval "$(uv generate-shell-completion --include-uv-run-commands)" I think that could be very beneficial. Perhaps it would involve having a way for uv run command definitions to explicitly declare their own shell completions, or perhaps the contract would be that if a uv run command itself supports generate-shell-completion then those get incorporated into the tree of completions rooted at uv run?

In general I agree that having a task runner built into uv (like poe etc, and like make when used as a simple task runner via .PHONY, but with cross-platform support) seems very attractive.

@paulovcmedeiros
Copy link

paulovcmedeiros commented Apr 3, 2025

In case other people may find it useful, this is how I do it with uv and poe:

[project]
name = "myproject"
requires-python = ">=3.12"
authors = [{name = "Author Name", email = "[email protected]"}]
description = "Example package config file"

[project.scripts]
my_package_entrypoint = "myproject.__main__:main"
task-runner = "poethepoet:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[dependency-groups]
task_runner = ["poethepoet>=0.33.1"]
linting = ["black>=25.1.0", "isort>=6.0.1", "ruff>=0.11.2"]

[tool.uv]
package = true
default-groups = "all"

[tool.poe.tasks]
black = "black ."
isort = "isort ."
ruff = "ruff check -q myproject/"
linters = ["black", "isort", "ruff"]

[tool.ruff]
select = ["ALL"]

With this you can run uv run task-runner linters (or whatever name you choose instead of task-runner). I find this to be enough for what I've needed so far.

@pot-code
Copy link

pot-code commented Apr 6, 2025

any updates?

@kevlarr
Copy link

kevlarr commented Apr 11, 2025

That's a great and well organized workaround @paulovcmedeiros. Especially if uv could be updated in the future to natively support defining tasks in the same way that poe does and tool.poe.tasks could simply be relabeled as tool.uv.tasks. (And then invoked like uv run task linters instead of needing to define a separate task-runner, etc. script.)

@amd-isaac
Copy link

I also appreciated @paulovcmedeiros' solution, but I hit a snag: it only works for packages, as they are the only types of project that support entrypoints. If you're wanting to use tasks with a non-packaged application, you can't redefine the name of the entrypoint from poe to task-runner or task (which is what I prefer) and need to instead use uv run poe format which is less "obvious" to me.

My solution to this was to use the taskfile.dev project, which is a standalone go executable which can be installed as a Python package (uv add --dev go-task-bin). This conveniently exposes a task executable in the venv without requiring an entrypoint in the pyproject.toml file. You can then run uv run task format which to me looks better than uv run poe format when running in a non-packaged application. The only downside is that you can't define the tasks in the pyproject.toml file, but need to instead use a taskfile.yml file.

So the pyproject.toml file would look like:

[project]
name = "myproject"
description = "Clever description here"
requires-python = ">=3.10"

[dependency-groups]
dev = [
    "go-task-bin>=3.42.1",
    "ruff>=0.11.5",
]

And the taskfile.yml file would look like:

version: '3'

tasks:
  format:
    desc: Format all files
    cmds:
      - ruff format

@JBloss1517
Copy link

I have been using poe with uv for a while now without much friction. Instead of using taskrunner like @paulovcmedeiros had, I just codify the commands that I would normally run via uv into my [tool.poe.tasks] section.

For example:

[tool.poe.tasks]
dev = "uv run src/main.py"
build = "uv run pyinstaller --clean ./Server.spec"
build-debug = "uv run pyinstaller --clean ./Server-debug.spec"

and use it like poe dev or poe build no problem.

As strictly a matter of preference for the api for uv, I would rather type uv run dev than uv run task dev. Where the run looks for available commands from your defined tasks in something like [tool.uv.tasks] and if it doesn't find it there, it looks into your scripts folder for other files for a the same name.

@Speedlulu
Copy link

Speedlulu commented Apr 14, 2025

and use it like poe dev or poe build no problem.

@JBloss1517 you must have poe installed globally for that to work, right ?

@Lordfirespeed
Copy link

Lordfirespeed commented Apr 14, 2025

@JBloss1517 you must have poe installed globally for that to work, right ?

$ uv init
$ uv add --dev poethepoet
$ source .venv/bin/activate
$ poe --version
Poe the Poet - A task runner that works well with poetry.
version 0.33.1

No, once you have activated the virtual environment.

Also, the uv run prefix in task should be unnecessary: poe docs. This should be sufficient:

[tool.uv]  # ensure this table exists somewhere in your pyproject.toml. It can be empty.

[tool.poe.tasks]
dev = "src/main.py"
build = "pyinstaller --clean ./Server.spec"
build-debug = "pyinstaller --clean ./Server-debug.spec"

@JBloss1517
Copy link

@Lordfirespeed great to know! Thanks for the info.

@AKuederle
Copy link

Just as a note, recent versions of poe have first class support for uv. This means poe will run uv run under the hood for any command. With that you don't need to activate the venv and you will get automatic uv sync when running the command

@phitoduck
Copy link

phitoduck commented Apr 17, 2025

Re: design

Comparison of a few task runners from the perspective of people who know bash and use uv.

I left blank areas that I was unsure. Feel free to correct me.

Feature bash script (example I like) Justfile Makefile Taskfile Poethepoet
Near-zero learning curve
Tab autocomplete commands
cwd does not matter
True bash (functions, env vars) ❌❌❌ ❌ (mvdan/sh)
Bash syntax highlighting in editors
No extra install ✅ (via uv run poe ...)
Can apt-get install if needed n/a already everywhere n/a
Cmds can accept arguments
^^^ named arguments
Tab autocomplete arguments

Some notes

  • "cwd does not matter" -- refers to how it doesn't matter which current working directory you're in, e.g. when you run uv add .... uv is smart enough to figure out where the pyproject.toml file is and goes from there. Similarly, you can run make ... in any subdirectory of a dir containing Makefile and the targets will be discovered. Really handy.

@my1e5
Copy link
Contributor Author

my1e5 commented Apr 17, 2025

@phitoduck For "No extra install" - Just and Task are as easy to install as Poe the Poet as the binaries are avaliable on PyPi. So they can be used with uv run automatically.

  • Task

    https://taskfile.dev/

    $ uv add --dev go-task-bin
    
    $ uv run task --init
    Taskfile created: Taskfile.yml
    
    $ uv run task
    Hello, World!
    
  • Just

    https://github.com/casey/just

    $ uv add --dev just-bin
    

    Create a justfile

    hello:
      echo 'Hello from just!'
    
    $ uv run just hello
    echo 'Hello from just!'
    Hello from just!
    

@phitoduck
Copy link

phitoduck commented Apr 18, 2025

Whoooooa 🤯. I had no idea! Sadly, you do lose tab-completion if you invoke just through uvx --from just-bin just or uv add just-bin; uv run just. Similar to poethepoet. But still, being able pip-install those other ones is huge.

Bringing this back to uv's design--If uv could figure out tab completion for their solution when they build it, that'd be sick. It's one of the surprisingly strong reasons my last 2 teams have stuck with Just/Make over pure bash.

Edit: wow, I feel like I have to report back what I found since it relates to some of the previous comments about poethepoet (sorry if this is off topic). Look at this!

If you use poethepoet in a project to define your tasks in pyproject.toml, a newcomer could run:

# install `poe` globally
uv tool install poethepoet

# make sure ~/.local/bin is in your $PATH
uv tool update-shell
# Output for me: Executable directory /Users/ericriddoch/.local/bin is already in PATH (but if it weren't it'd be added, so this is idempotent!)

# enable tab completion of poe tasks in pyproject.toml; there's a similar cmd for those that don't use OhMyZSH--an onboarding script could just run all of them
# https://poethepoet.natn.io/installation.html#shell-completion
mkdir -p ~/.oh-my-zsh/completions
poe _zsh_completion > ~/.oh-my-zsh/completions/_poe

# reset the shell to reflect all these $PATH changes
eval $SHELL

Now typing poe and then pressing Tab will autocomplete the tasks. 🎉

As others have pointed out, poe automatically uses uv run to run tasks so you get all the benefits of uv. 🎉

This is exactly what I was looking for as a ML Platform engineer trying to make onboarding to our DS repos simple/consistent. As far as I'm concerned, I'd be thrilled if uv essentially gave you all this without requiring all the setup commands first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement to existing functionality
Projects
None yet
Development

No branches or pull requests