Enhance pdm run to support specifying Python version

[laike9m]

Is your feature request related to a problem? Please describe.

N/A

Describe the solution you'd like

As of now, pdm run inherits from the current Python version specified by pdm use. I find that I often need to run pdm use just to be able to run pdm run with a specific Python version. It would be convenient if pdm run supports something like:

pdm run --version=3.7 pytest

Ideally, it shouldn't change the currently selected Python version, but only applies to the command being run.

[pawamoy]

Nice idea! That could be very useful to me too 🙂

I currently use a multirun.sh script that iterates on multiple Python version to run the same command (typically pytest).
So for each version, it does pdm use -f python$version, then pdm run pytest .... At the end, the selected Python version for the project is the last I tested, i.e. Python 3.11, which is not always what I want, and can be problematic if the next task to run does not work on Python 3.11 (I don't have a way to specify an exact version yet for a task, so I have to run pdm use again, manually, or change the order of tasks).

With what you suggest I'd just have to run pdm run --version=$version pytest ... and be sure that the selected Python version (for example 3.10) is left intact at the end of the run.

Note: rather than --version, I'd suggest to use the same logic from use, something like --interpreter, or --python maybe.

[laike9m]

Thanks for seconding the idea. --use does feel better as it's the same with pdm use so people know what it's doing.

[frostming]

While the feature is mostly good, I have some concerns:

• pdm use shows a list of matching versions and prompts users to choose, but we can't do the same for run --use, can we? I am not sure if users care about the selection and are happy with what is chosen by PDM.
• In fact, the interpreters behave differently even if they have the same version. When use_venv is true, venv-python corresponds to a virtual environment(which installs packages into venv/lib/site-packages) while others correspond to a PEP 582 environment.

So I doubt if we can do the interpreter selection for users.

[pawamoy]

Indeed that's something to consider. I myself am using pyenv, so I have only one interpreter of each minor version available in my PATH, allowing me to always rely on pdm use -f. But that will not be the case for everyone.

[laike9m]

but we can't do the same for run --use, can we?

Personally I'm fine with having the same prompt when using pdm run --use.

Since we're discussing this, I've had a question for a while. Is it possible that pdm caches the interpreters selected before, or at least provide a way to configure it? Having to do a selection each time feels tedious, and I assume when working on a specific project, it's very rare that people need to select a different interpreter path for the same Python version.

[pawamoy]

Personally I'm fine with having the same prompt when using pdm run --use.

I'd definitely not want that 😅 Unless of course there's a -f switch as well 🙂
It's basically merging use in run without saving the selected interpreter to .pdm.toml.

[frostming]

--use with -f looks weird and is repeating pdm use. Instead of integrating into PDM, I'd wonder if @pawamoy can make multirun.sh a PDM plugin.

However, IMO all you need here is no more than a task runner like tox and nox. In fact, they can be installed globally onto your system so you don't need to include them in every project's dependencies. What I want to emphasize is PDM is never to be a swiss knife toolset, but is trying to stay as a core component of Python packaging and connect with various tools in the eco-system which are already doing a great job in their fields.

Let the specialized tool do the specialized work

Lastly, thank you for understanding.

[laike9m]

Sure, I'm fine with keeping this low priority. Let's keep the discussion open so that others can chime in.

[laike9m]

I'll create another FR for configuring interpreter paths for different versions.

[pawamoy]

Nice plugin idea indeed @frostming 🙂
Restoring the initial Python version would solve it for me (be it in a shell script or plugin).
About tox/nox: I don't like any of those 🤣 Nor having to install them globally (that's one more setup step instead of having them listed as dependencies).

[ElijahSink]

Is there a reason that using the PDM_PYTHON envvar isn't sufficient?

> pyenv local 3.8.13 && PDM_PYTHON=$(pyenv which python) pdm run python --version
Python 3.8.13
> pyenv local 3.9.12 && PDM_PYTHON=$(pyenv which python) pdm run python --version
Python 3.9.12
> pdm run python --version # python version saved in .pdm.toml is unaffected
Python 3.10.4

[sarayourfriend]

Sorry for reviving an old issue, but I wanted to share a PDM plugin I created (though haven't yet published).

I've recently started using PDM after we switched over WordPress/openverse. I've used Hatch in the past for some projects, and really missed its virtual environment matrix feature. I cobbled together a PDM plugin that does more or less what I was hoping for when I came across this issue: https://git.sr.ht/~sara/pdm-run-matrix.

The plugin allows you to specify a matrix of values, including specific Python versions, then auto-manages installing the interpreter, creating and updating virtual environments for each variable combination, and running the script in each venv. This only approximates the features that Hatch has, but so far does what I need it to do. The main thing the plugin is definitely missing is good error handling and reporting (fast fail vs try everything) and probably some additional configuration options for flexibility.

I'm not familiar with PDMs internals, so I am almost certain it has severe problems (hence why I haven't published it), and I had to rely on a set of rather unfortunate closures to be able to accomplish some things. @frostming if something along these lines could be added to PDM, and you'd like to collaborate on it, please let me know. I'd be keen to learn more about how PDM works and to contribute something along these lines.

Of course, an idea version of this would allow sharing the matrix-generated virtual environments between multiple tests, rather than tests that otherwise share identical matrix configurations having completely isolated environments. I'll play around with that idea in the plugin format as well.

[pawamoy]

Amazing @sarayourfriend, thanks for sharing! If you publish it to PyPI.org at some point, do also add it to https://github.com/pdm-project/awesome-pdm 😄

I'll also share that I ended up writing a PDM plugin too to replace my multirun.sh script: https://github.com/pawamoy/pdm-multirun/ 🙂

[frostming]

@sarayourfriend nice try, the implementation looks neat.

and I had to rely on a set of rather unfortunate closures to be able to accomplish some things.

If you mean the runner_cls subclass, see this: #2872

I am interested in it and maybe you can publish it as a plugin first and then we will see if there is any possibility building it into PDM.