poetry install does not honor `build-system.requires`

[sharvil]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 20.04.4
• Poetry version: 1.2.0b2
• Link of a Gist with the contents of your pyproject.toml file:

pyproject.toml

[tool.poetry]
name = "meow"
version = "0.1.0"
description = ""
authors = [""]
build = "build.py"

[tool.poetry.dependencies]
python = "^3.9"
SoundFile = "0.10.*"
crcmod = "^1.7"
librosa = ">=0.8.0"
numba = "^0.56.0"
protobuf = "3.14.0"
tensorboard = "^2.10.0"
torchaudio = {version = "=0.12.1+cu113", source="torch"}
torch = {version = "=1.12.1+cu113", source="torch"}

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"
secondary = true

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0", "setuptools>=40.6.0", "wheel", "numpy", "pybind11"]
build-backend = "poetry.core.masonry.api"

Issue

Running poetry build with the above pyproject.toml does the right thing:

Preparing build environment with build-system requirements poetry-core>=1.0.0, setuptools>=40.6.0, wheel, numpy, pybind11
[...snip...]

Running poetry install fails because it attempts to build the project without installing build-system.requires packages.

I would have expected the install command to create an isolated environment, install the build requirements and perform the build in that isolated environment, and then install the produced artifact in my current environment.

[bilelomrani1]

Same issue on macOS 12.5. A dirty hack is to add the following at the top of build.py:

import subprocess

try:
    import pybind11
except ImportError:
    subprocess.run(["pip", "install", "pybind11"])

The exact same issue was described in #2789, which was marked as solved, but the issue is still present on Poetry 1.2.0b3. I didn't check if this is a regression from when this PR was merged.

[neersighted]

This is currently intentional -- Poetry has never respected the build-system entry for a Poetry-managed project. We might want to establish consensus on this being a desired behavior, since currently you can install any needed build dependencies using a Poetry group or similar (but this will result in duplication of deps in pyproject.toml).

[neersighted]

Re: this being a regression, the changes in that PR were explicitly scoped to use of build scripts.

Also, Poetry does not perform builds in an isolated/pep 517 environment, but instead uses the 'main' environment poetry install uses, as this is the only way to use lockfile versions during the build with the current architecture. It is possible to re-architect this so that a clean build environment is used and the build-system deps are only installed there (as opposed to polluting the dev group implicitly or similar) -- however that will require the implementation of a lockfile-aware poetry-core first (which we have already discussed/planned).

Is this the same issue as [tool.poetry.dependencies] deps being unavailable to tool.poetry.build.script in 1.2.0 despite being installed?

[arcbtc]

+1, having exact same issue tool.poetry.build.script is not using dependencies from [tool.poetry.dependencies]