allow running non-default Python interpreters directly via uvx

[oconnor663]

Previously if you wanted to run e.g. PyPy via uvx, you had to spell it like uvx -p pypy python. Now we reuse the PythonRequest::parse machinery to handle the executable, so all of the following examples work:

• uvx python
• uvx python3.8
• uvx 38
• uvx pypy
• uvx [email protected]
• uvx graalpy
• uvx [email protected]

The python (and on Windows only, pythonw) special cases are retained, which normally aren't allowed values of -p/--python.

Closes #13536.

---

The initial version of this PR has 3 failing snapshot tests. The common issue is that this @ syntax used to work:

$ uvx [email protected]
Python 3.8.20 (default, Oct  2 2024, 16:34:12) 
[Clang 18.1.8 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

But now it does this:

$ uvx [email protected]
× No solution found when resolving tool dependencies:
  ╰─▶ Because python was not found in the package registry and you require python==3.8, we can conclude that your requirements are unsatisfiable.

The underlying reason for the change is that we used to parse the executable with Target::parse, but in this PR we parse the executable with PythonRequest::parse. The latter doesn't recognize [email protected] at all, and it returns the catch-all PythonRequest::ExecutableName variant, which we take to mean that the executable is a package and not an interpreter.

There are several different ways we could handle this, and I'm not sure what's best. Reviewer feedback needed :)

• We could break back compat and stop supporting uvx [email protected]. Note that either way, uvx python3.8, uvx python38, and even uvx 38 all work under this PR. (The last one there might be even more flexible than we want? That's another potential reviewer question.) I'm not sure how widely the current syntax is used, but I'd guess it's more widely used than uvx -p python3.8 python?
• We could expand the special case in ToolRequest::parse (previously is_python) to support [email protected] in addition to bare python. (Update: We ended up expanding ToolRequest::parse and refactoring some of the PythonRequest::parse machinery to minimize duplication.)
• We could expand PythonRequest::parse to support [email protected], possibly by adding "python" to ImplementationName::long_names as an aslias for "cpython". This would allow e.g. uvx --python [email protected] ruff, which isn't currently allowed in main or in this PR, and which might not be desirable?

[Gankra]

We could expand the special case in ToolRequest::parse (previously is_python) to support [email protected] in addition to bare python.

After some checking this feels like the best option of the 3 you laid out. I don't want to break compat without a really good reason, and PythonRequest::parse is called pervasively throughout our CLI, so changing that would have widespread ramifications.

[Gankra]

The implementation seems reasonable. In an ideal world we could fail one of the parsers and simply fallback to the other to preserve the old behaviour, but I guess the parsers are too flexible for that.

[zanieb]

We could expand the special case in ToolRequest::parse (previously is_python) to support [email protected] in addition to bare python.

After some checking this feels like the best option of the 3 you laid out. I don't want to break compat without a really good reason, and PythonRequest::parse is called pervasively throughout our CLI, so changing that would have widespread ramifications.

I actually just suggested that we want consistent Python parsing throughout and python is a reasonable alias for cpython, so uv run -p [email protected] seems okay and I think the last option is what I'd lean towards unless there are other tool-specific special cases we need to implement anyway.

[zanieb]

I don't think a bare version (e.g., 312 or 3.12) should work, but I do think python3 and python3.12 should work.

[zanieb]

This suggests that maybe PythonRequest isn't quite right since it drops that information? You could add variants like Interpreter (i.e., bare python) and InterpreterVersion (i.e., python3.13) (names questionable), but that's a fair amount of work since you'd need to handle them everywhere.

[oconnor663]

We could expand the special case in ToolRequest::parse (previously is_python) to support [email protected] in addition to bare python.

After some checking this feels like the best option of the 3 you laid out. I don't want to break compat without a really good reason, and PythonRequest::parse is called pervasively throughout our CLI, so changing that would have widespread ramifications.

I chatted with @zanieb just now, and they were leaning towards the third option of allowing e.g. --python [email protected] everywhere (maybe by adding it to long_names and calling it an alias for "cpython"), which would fix my broken tests as a side effect. At the same time, uvx 38 seems like too much to allow, because we'd be implicitly shadowing a large class of possible package names. I'm going to noodle on whether I can factor out what we want from PythonRequest::parse, and/or whether we might want something like uv run's --script flag that callers with "interesting" internal package names can use to prevent shadowing. (The natural choice might be --package but alas that's already taken.)

[zanieb]

On

whether we might want something like uv run's --script flag that callers with "interesting" internal package names can use to prevent shadowing.

I mostly brought that up to explain our general thinking and existing patterns for that problem. I probably wouldn't pursue adding that here.

[oconnor663]

Second pass: I've factored out an alternate constructor for PythonRequest called try_parse_tool_executable. This shares parsing logic for cases that are (now) common between --python and uvx (python3.8, [email protected], pypy38) but allows for cases that are unique to --python (38, cp@38) or to uvx (python, pythonw on Windows). If the differences get any more complicated than this, it might be better to give up the sharing and just copy-paste the parser, but as-is it doesn't feel too bad. The match statement where I was seeing which variant of PythonRequest we got is thankfully gone.

There is one remaining failing snapshot test: tool_run_python_from

uv/crates/uv/tests/it/tool_run.rs

Lines 1956 to 2004 in 30be27b

	#[test]
	fn tool_run_python_from() {
	let context = TestContext::new_with_versions(&["3.12", "3.11"])
	.with_filtered_counts()
	.with_filtered_python_sources();
	uv_snapshot!(context.filters(), context.tool_run()
	.arg("--from")
	.arg("python")
	.arg("python")
	.arg("--version"), @r###"
	success: true
	exit_code: 0
	----- stdout -----
	Python 3.12.[X]
	----- stderr -----
	Resolved in [TIME]
	Audited in [TIME]
	"###);
	uv_snapshot!(context.filters(), context.tool_run()
	.arg("--from")
	.arg("[email protected]")
	.arg("python")
	.arg("--version"), @r###"
	success: true
	exit_code: 0
	----- stdout -----
	Python 3.11.[X]
	----- stderr -----
	Resolved in [TIME]
	Audited in [TIME]
	"###);
	uv_snapshot!(context.filters(), context.tool_run()
	.arg("--from")
	.arg("python>=3.12")
	.arg("python")
	.arg("--version"), @r###"
	success: false
	exit_code: 2
	----- stdout -----
	----- stderr -----
	error: Using `--from python<specifier>` is not supported. Use `python@<version>` instead.
	"###);
	}

This tests cases like uvx --from [email protected] python. That's similar to uvx --python python3.8 python (which is still supported and now allows the @ syntax too), but it actually ignores the executable argument entirely. For example with the currently released version:

$ uvx --from python@38 blargblargblarg
Python 3.8.20 (default, Oct  2 2024, 16:34:12) 
[Clang 18.1.8 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

I'm not sure what to do with this. We could keep supporting it, but it seems kind of bizarre. Does anyone use this? (If it was important enough to be in the snapshot tests I fear the answer is yes.) Feedback needed.

Update: Zanie clarified that --from python is important for use cases like uvx --from python bash, which runs bash from the PATH with Python available. I added support below.

[oconnor663]

Drive by: this is what was intended right?

[Gankra]

I'm not sure what to do with this. We could keep supporting it, but it seems kind of bizarre. Does anyone use this? (If it was important enough to be in the snapshot tests I fear the answer is yes.) Feedback needed.

This was added in #11076 but the PR/issue focuses on non-from usage. That said, zanie added explicit snapshots for it, and one error for "hey what are you doing", so pretty clear they were going eyes wide-open into it.

The fact that the second argument gets ignored seems like a bug though... at very least we should probably error if it's not python?

[konstin]

Just some drive-by comments

[konstin]

Does this snapshot break when someone publishes cp311 to PyPI?

[zanieb]

Yes, but we should squat or advocate for those names to be banned.

[oconnor663]

This is my first time using snapshot testing, so a broad philosophy question: How do we feel about "brittleness" in snapshots? Is it just as bad as brittleness in regular tests, or is it more acceptable?

[zanieb]

It's just as bad, really. We try to make them reproducible wherever possible. That's mostly w.r.t. the external world though, internal changes causing snapshot changes are fine.

However, I think this specific case is fine.

[zanieb]

This seems like a regression, and would be confusing since we do support, e.g., ruff@latest.

[oconnor663]

Could you say more here? To me it looks like something that was an error before is still an error, with a different message. PythonRequest doesn't currently have a Latest variant, so that would need to be implemented if we wanted to support it in this PR.

[zanieb]

The change in message here is a regression because it moves from a clear error message to a misleading one. Previously, we explained that latest was not supported for python requests. Here, we say latest is an invalid version request, but... that's not true in the general case because we support it for non-python requests, like ruff@latest.

[oconnor663]

753329f

[zanieb]

Separate, but we should open an issue to follow-up on this. uvx -p >=3.12 python is valid and probably a better recommendation? Also I'm not sure why we wouldn't support python>=3.12 here.

[oconnor663]

I'm not sure exactly how, but I'd gotten the impression that we didn't want to support this. If we do, I think I can just remove this check:

uv/crates/uv/src/commands/tool/run.rs

Lines 763 to 771 in 3b9358e

	if matches!(request, PythonRequest::Version(VersionRequest::Range(..))) {
	return Err(anyhow::anyhow!(
	"Using `{}` is not supported. Use `{}` or `{}` instead.",
	"--from python<range>".cyan(),
	"python<version>".cyan(),
	"python@<version>".cyan(),
	)
	.into());
	}

[oconnor663]

5ac5190

[zanieb]

(Presuming this regression is fixed https://github.com/astral-sh/uv/pull/13583/files#r2112604209)

[oconnor663]

Also, @Gankra @konstin any more comments?

[konstin]

I'll leave the design review to zanie, code LGTM.