Poetry install is stuck on specific dependency with "pending..."

[cpvandehey]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: 10.15.7
• Poetry version: 1.1.4, 1.1.3, 1.0.9

Issue

Hello poetry crew. Loving the work that yall do!

I am having issues with poetry while running poetry install in one of my projects on one specific machine.

It appears to get stuck on one dependency during the install and it will hang forever. When I kill the process, I see this in my shell (hinting at a race condtion/locking problem):

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/concurrent/futures/thread.py", line 40, in _python_exit
    t.join()
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

I am on the latest poetry (1.1.4) and have tried 1.1.3 as well with the same results (all of which while running python 3.6.8 (I did use 3.6.5 as well)).

After downgrade to 1.0.9, I no longer faced this issue. I am assuming that there was an update to the way poetry can download multiple deps at the same time. Anyone else facing locking issues? At the very least, poetry should recognize a pending download with no progress being made to exit and fail early, but it simply hung and would never exit (left it going for at least 30 mins).

[sinoroc]

Maybe it would be helpful to see the pyproject.toml. Is that on macos? Were you able to identify if it hangs always on the same dependency or not?

[cpvandehey]

Unfortunately, I cannot share the pyproject.toml as it contains a lot of proprietary names.

Is that on macos?

As noted above, mac OS 10.15.7 (catalina)

Were you able to identify if it hangs always on the same dependency or not?

It does indeed hang on the same dependency every time.

---

One other interesting thing to note:
I can manually go to the venv created by poetry and go to the pip that lives in its bin directory. When I pip install the problematic dependency (that hangs in the poetry install execution), it does install correctly. So it appears to focus the problem even further on the locking mechanism being used.

[DeepDarkOdyssey]

Same issue here. I just upgrade from v1.0.0, but got pending on some packages during installing , typically numpy , sklearn... The error message shows it got a read time out from my nexus repo, but it never happened before. And after I try the poetry install several times, it installed all package successfully. Does the parallel installation trigger this issue?

[sinoroc]

I do not know how to help you all here...

My advice would be to try one of those (maybe in that order) if you have not yet:

• delete the poetry.lock file and retry
• delete the virtual environment and retry
• deactivate the new (parallel) installer with poetry config experimental.new-installer false and retry (do not forget to reactivate it afterwards if it proves to have no influence)
• or any combination of those

Also maybe provide some verbose output (-vvv), that might be helpful for when one of the maintainers gets the time to handle this ticket. Of course, the best as always is if you can provide a minimal reproducible example that triggers the failure.

[arnegroskurth]

I do experience the same problem with poetry 1.1.4 within a python 3.9 docker container. The error goes away by following the suggestion of @sinoroc to deactivate the experimental installer.

pyproject.toml:

[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.8"
watchdog = "^1.0.1"

[tool.poetry.dev-dependencies]
docker = "^4.4.0"
pytest = "^6.2.1"

poetry.lock.txt

[vaidik]

I can confirm that disabling the experimental parallel installer worked. My environment, although, was Docker, particularly the python:3.8-slim image.

[ohthehugemanatee]

Same problem here, and disabling the experimental installer fixed it for me, too. My environment is this docker image: mcr.microsoft.com/vscode/devcontainers/python:0-3.8 with the following commands:

apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends python3-venv gnupg2
pip install tox
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
source $HOME/.poetry/env
poetry --version
poetry install -v

[jlucas91]

Could someone on the Poetry team provide a perspective on prioritization for this? It seems like there's meaningful evidence that this is an issue independent of environment or dependencies. Turning off the installer is a good workaround in the interim, but it significantly increases install times. I'd love to know I have some hope of being able to turn it back on at some point.

[dbowring]

Disabling experimental installer also fixes this for me, but also able to fix it using the experimental installer by adding a timeout for the requests to pypi, E.g.

diff --git a/poetry/repositories/pypi_repository.py b/poetry/repositories/pypi_repository.py
index 1c7b7be3..a1f0922a 100644
--- a/poetry/repositories/pypi_repository.py
+++ b/poetry/repositories/pypi_repository.py
@@ -321,14 +321,14 @@ class PyPiRepository(RemoteRepository):
 
         return data.asdict()
 
-    def _get(self, endpoint: str) -> Union[dict, None]:
+    def _get(self, endpoint: str, timeout: int = 5) -> Union[dict, None]:
         try:
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
         except requests.exceptions.TooManyRedirects:
             # Cache control redirect loop.
             # We try to remove the cache and try again
             self._cache_control_cache.delete(self._base_url + endpoint)
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
 
         if json_response.status_code == 404:
             return None

[GerevSec]

Happens on WSL2 ubuntu 20.04 with python3.10

[schlich]

Recent issues possibly related to discussion happening in #2094

[epomatti]

On Ubuntu 20.04 WSL2 here. Only disabling IPv6 did the trick:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1