Make uv’s first-index strategy more secure by default by failing early on authentication failure

[jtfmumm]

uv’s default index strategy was designed with dependency confusion attacks in mind. According to the docs, “if a package exists on an internal index, it should always be installed from the internal index, and never from PyPI”. Unfortunately, this is not true in the case where authentication fails on that internal index. In that case, uv will simply try the next index (even on the first-index strategy). This means that uv is not secure by default in this common scenario.

This PR causes uv to stop searching for a package if it encounters an authentication failure at an index. It is possible to opt out of this behavior for an index with a new pyproject.toml option ignore-error-codes. For example:

[[tool.uv.index]]
name = "my-index"
url = "<index-url>"
ignore-error-codes = [401, 403]

This will also enable users to handle idiosyncratic registries in a more fine-grained way. For example, PyTorch registries return a 403 when a package is not found. In this PR, we special-case PyTorch registries to ignore 403s, but users can use ignore-error-codes to handle similar behaviors if they encounter them on internal registries.

Depends on #12651

Closes #9429
Closes #12362

[charliermarsh]

Rustdoc please for the variants.

[jtfmumm]

Added

[charliermarsh]

Nit: Self instead of FetchUrl

[jtfmumm]

Updated

[charliermarsh]

Nit: Rustdoc please with an example of when this returns true vs. false.

[jtfmumm]

Because of the timing of the release, can we defer this to a PR afterwards? I just moved this logic from a function to a method in this PR.

[charliermarsh]

I'm still struggling a bit. Why do we break when (e.g.) we hit a 401, but 401 isn't included in the list of ignored errors? Why is it not something like this?

diff --git a/crates/uv-client/src/registry_client.rs b/crates/uv-client/src/registry_client.rs
index 8bdf2c065..3cb6b4978 100644
--- a/crates/uv-client/src/registry_client.rs
+++ b/crates/uv-client/src/registry_client.rs
@@ -348,12 +348,6 @@ impl RegistryClient {
                                 }
                                 // Package not found, so we will continue on to the next index (if there is one)
                                 SimpleMetadataSearchOutcome::NotFound => {}
-                                // The search failed because of an HTTP status code that we don't ignore for
-                                // this index. We end our search here.
-                                SimpleMetadataSearchOutcome::StatusCodeFailure(status_code) => {
-                                    debug!("Indexes search failed because of status code failure: {status_code}");
-                                    break;
-                                }
                             }
                         }
                         IndexFormat::Flat => {
@@ -522,17 +516,15 @@ impl RegistryClient {
                     let Some(status_code) = err.status() else {
                         return Err(ErrorKind::WrappedReqwestError(url, err).into());
                     };
-                    let decision =
-                        status_code_strategy.handle_status_code(status_code, index, capabilities);
-                    if let IndexStatusCodeDecision::Fail(status_code) = decision {
-                        if !matches!(
-                            status_code,
-                            StatusCode::UNAUTHORIZED | StatusCode::FORBIDDEN
-                        ) {
-                            return Err(ErrorKind::WrappedReqwestError(url, err).into());
+                    match status_code_strategy.handle_status_code(status_code, index, capabilities)
+                    {
+                        IndexStatusCodeDecision::Ignore => {
+                            Ok(SimpleMetadataSearchOutcome::NotFound)
+                        }
+                        IndexStatusCodeDecision::Fail(..) => {
+                            Err(ErrorKind::WrappedReqwestError(url, err).into())
                         }
                     }
-                    Ok(SimpleMetadataSearchOutcome::from(decision))
                 }
 
                 // The package is unavailable due to a lack of connectivity.
@@ -998,18 +990,6 @@ pub(crate) enum SimpleMetadataSearchOutcome {
     Found(OwnedArchive<SimpleMetadata>),
     /// Simple metadata was not found
     NotFound,
-    /// A status code failure was encountered when searching for
-    /// simple metadata and our strategy did not ignore it
-    StatusCodeFailure(StatusCode),
-}
-
-impl From<IndexStatusCodeDecision> for SimpleMetadataSearchOutcome {
-    fn from(item: IndexStatusCodeDecision) -> Self {
-        match item {
-            IndexStatusCodeDecision::Ignore => Self::NotFound,
-            IndexStatusCodeDecision::Fail(status_code) => Self::StatusCodeFailure(status_code),
-        }
-    }
 }
 
 /// A map from [`IndexUrl`] to [`FlatIndexEntry`] entries found at the given URL, indexed by

[charliermarsh]

I think this should mention the assumption that the status code failure will be tracked by IndexCapabilities and surfaced via hints.

[charliermarsh]

I think this should mention that these specific codes are tracked via IndexCapabilities and surfaced via hints; otherwise, it's not really clear why these specific codes are ignored.

[charliermarsh]

I think I would find it clearer if we actually just made individual enum variants for Unauthorized and Forbidden, rather than allowing an arbitrary status code here. That way the expectations are encoded in the type system. We should make the invalid state (e.g., StatusCodeFailure(400)) unrepresentable.

[zanieb]

I'm going to merge and kick the tires on the release.

[juhaszp95]

Hi There,

Just a quick question on the above - I have an internal package index which also returns 403 for missing packages. Is it possible to make the ignore-error-codes a user-wide setting by somehow incorporating it in uv.toml? I.e. to declare custom exceptions user-wide.

Furthermore, it seems to me that the workaround in pyproject.toml only works if I include the authentication token as well in the url, which is obviously undesirable.

[jtfmumm]

Hi There,

Just a quick question on the above - I have an internal package index which also returns 403 for missing packages. Is it possible to make the ignore-error-codes a user-wide setting by somehow incorporating it in uv.toml? I.e. to declare custom exceptions user-wide.

Furthermore, it seems to me that the workaround in pyproject.toml only works if I include the authentication token as well in the url, which is obviously undesirable.

You shouldn't need to include a token in the URL (and I agree you shouldn't include one in pyproject.toml). Did you run into an issue when excluding it?

[juhaszp95]

Yes - interestingly, I get the error An index URL (https://myorg.jfrog.io/artifactory/api/pypi/pypi/simple) could not be queried due to a lack of valid authentication credentials (403 Forbidden)., even though I set ignore-error-codes = [401, 403]. This index is set as an extra-index-url in uv.toml. This doesn't seem to be an issue for the other internal index I use, which is set as the index-url in uv.toml.

[jtfmumm]

Ok, would you mind opening an issue and including a reproduction and trace (-vv) logs? I can investigate this.

[juhaszp95]

Should I put it here as it seems closely related, or would you rather a separate issue?

[jtfmumm]

I think a separate issue for now would be helpful. It's not clear to me these are the same problem.

[juhaszp95]

I've realised what the problem is just now (and it's sort of OK): as I was debugging, I left the url in the format USER@ORG@repository-url which works for extra-index-url but strangely does not for index-url. When I removed the USER@ORG part it worked again, so that's all solved I believe! I have no idea why it behaves differently for index-url and extra-index-url, but at least it works for both when there aren't really any credentials there (i.e. not just the token deleted, but everything). Also, interestingly, I need to put the index-url in pyproject.toml (without any ignore-error-codes) for it to work - strange. It's like as if the extra-index-url is treated differently than the index-url, or perhaps if indexes are declared in pyproject.toml, the contents of uv.toml are ignored, hence the need to declare the index in pyproject.toml as well.

This still leaves the outstanding question whether all this config could be saved to a uv.toml for user-level configuration so I don't need to do this at every single project.

[jtfmumm]

Is this problem unique to 0.7.0 or do you also see it on 0.6.17? The ignore-error-codes feature doesn't exist on 0.6.17, but I'm trying to understand if something broke with 0.7.0 or if this is just an enhancement you'd like to see to the new error code behavior. Or if the main concern is the change that we are no longer ignoring 403s by default (and you can't find a good way to configure uv to ignore them for your situation).

[juhaszp95]

The problem and enhancement request is both unique to 0.7.0. Before, I had all index and authentication info saved in uv.toml with no issues. Now, due to how 0.7.0 handles error codes, I have to copy ignores to all projects as well. It'd be good if the ignores could be configured centrally rather than at a project level (for my use-case). I've checked 0.7.1 doesn't solve this (as expected).