403 errors on custom repositories

[vikigenius]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• Linux 5.15.7_1:
• Poetry version 1.1.12:
• Link of a Gist with the contents of your pyproject.toml file:

Issue

Added a custom repository like this:

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"

So that I can install torch from here

Tried installing to a venv with a clean install of poetry. I am using pyenv to maintain python versions so I do:

pyenv local 3.9.7
poetry env use 3.9.7
poetry install 

And then doing poetry add torch==1.10.0 leads to error

Updating dependencies
Resolving dependencies... (0.5s)

  RepositoryError

  403 Client Error: Forbidden for url: https://download.pytorch.org/whl/cu113/python-lsp-server/

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/legacy_repository.py:393 in _get
      389│             if response.status_code == 404:
      390│                 return
      391│             response.raise_for_status()
      392│         except requests.HTTPError as e:
    → 393│             raise RepositoryError(e)
      394│
      395│         if response.status_code in (401, 403):
      396│             self._log(
      397│                 "Authorization error accessing {url}".format(url=response.url),

If packages are not found in custom repository, why is poetry not falling back to PyPI. This is incredibly frustrating. All these workarounds for installing torch and none of them work because of various issues.

Edit

Changing the custom repo to secondary

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"

and doing poetry add torch --source torch still throws the same error? Why is poetry checking a seoncary repo?

[severinsimmler]

Duplicate of #4902.

[vikigenius]

@severinsimmler But that issue describes a private repo with authentication, this is a public repository of packages, there should be no authentication needed.

[cyd3r]

A quick and hacky workaround would be to modify your file at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/legacy_repository.py and insert a check against a 403 status code. Right before the error is raised, you can change it to:

try:
    response = self.session.get(url)
    if response.status_code == 404:
        return
    # add these two lines below
    if self.url == "https://download.pytorch.org/whl/cu113" and response.status_code == 403:
        return
    response.raise_for_status()
except requests.HTTPError as e:
    raise RepositoryError(e)

Note that this probably does not solve all your problems though (e.g. you won't be able to install torchvision).

[GuillaumeDesforges]

My current workaround was to add in pyproject.toml :

[[tool.poetry.source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu113"
secondary = true

and then I can do

$ poetry add torch==1.11.0+cu113

but if I need something from pypi I need to do

$ poetry add python-dotenv --source pypi

EDIT: did not work for requests

[ralbertazzi]

This will be solved in the next 1.5 release with the introduction of explicit repositories, that allow invoking a repository only for dependencies that explicitly require it. As the initial request came from PyTorch, I suggest you have a look at #6409 (comment) (and the whole thread in general).

@radoering I suggest closing this issue

[radoering]

Closing since all described issues should be resolved in Poetry 1.5. There will be explicit and supplemental package sources, which should do what was expected from secondary here.

Further, as it seems Poetry does not raise an error on 403 anymore for some time now:

poetry/src/poetry/repositories/http_repository.py

Lines 266 to 280 in 0af3f1e

	try:
	response: requests.Response = self.session.get(
	url, raise_for_status=False, timeout=REQUESTS_TIMEOUT
	)
	if response.status_code in (401, 403):
	self._log(
	f"Authorization error accessing {url}",
	level="warning",
	)
	return None
	if response.status_code == 404:
	return None
	response.raise_for_status()
	except requests.exceptions.HTTPError as e:
	raise RepositoryError(e)

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.