ModuleNotFound for editable install of namespace package with two dots

[mauritsvanrees]

setuptools version

68.1.2, same with main

Python version

3.11.5

OS

MacOS

Additional environment information

Also happening on GitHub-actions on a Linux env.
See also my comment here: plone/meta#172 (comment)

Locally, on Python 3.10 all goes well, on 3.11 not.

Description

I maintain various setuptools/pkg_resources style namespace packages with two dots, for example: plone.app.uuid. Until the beginning of August, an editable install worked fine. Now it fails, because setuptools 68.1.0+ is used:
ModuleNotFoundError: No module named 'plone.app.uuid'

Packages with just one dot seem fine, for example plone.session.

Expected behavior

After pip install -e . of the plone.app.uuid package, I should be able to do import plone.app.uuid.

How to Reproduce

I tried to create a stripped-down copy of plone.app.uuid, but there an import works fine. I suspect it only goes wrong if another plone.app.* package is installed next to it. So use the original repo, but use a branch I created for this issue with minor changes (mainly a much smaller tox.ini):

git clone -b maurits-setuptools-namespace git@github.com:plone/plone.app.uuid.git
cd plone.app.uuid
tox -e test

This can take a few minutes, because it needs lots of packages.

Output

$ git clone -b maurits-setuptools-namespace git@github.com:plone/plone.app.uuid.git
Cloning into 'plone.app.uuid'...
remote: Enumerating objects: 698, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 698 (delta 0), reused 0 (delta 0), pack-reused 696
Receiving objects: 100% (698/698), 134.94 KiB | 746.00 KiB/s, done.
Resolving deltas: 100% (353/353), done.
$ cd plone.app.uuid
$ tox -e test
test: install_deps> python -I -m pip install zope.testrunner -c https://dist.plone.org/release/6.0-dev/constraints.txt
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /Users/maurits/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /Users/maurits/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_editable> python /Users/maurits/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_editable> python -I -m pip install wheel
.pkg: build_editable> python /Users/maurits/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
test: install_package_deps> python -I -m pip install Products.CMFCore Products.ZCatalog plone.app.testing plone.dexterity plone.indexer plone.testing plone.uuid setuptools zope.interface zope.publisher -c/Users/maurits/foo/plone.app.uuid/.tox/test/constraints.txt
test: install_package> python -I -m pip install --force-reinstall --no-deps /Users/maurits/foo/plone.app.uuid/.tox/.tmp/package/1/plone.app.uuid-2.2.3.dev0-0.editable-py3-none-any.whl
test: commands[0]> zope-testrunner --all --test-path=/Users/maurits/foo/plone.app.uuid -s plone.app.uuid
Traceback (most recent call last):
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/bin/zope-testrunner", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/__init__.py", line 31, in run
    failed = run_internal(defaults, args, script_parts=script_parts, cwd=cwd,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/__init__.py", line 55, in run_internal
    runner.run()
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/runner.py", line 179, in run
    feature.global_setup()
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 505, in global_setup
    tests = find_tests(self.runner.options, self.runner.found_suites)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 178, in find_tests
    for suite in found_suites:
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 199, in find_suites
    for fpath, package in find_test_files(options):
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 265, in find_test_files
    for f, package in find_test_files_(options):
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 293, in find_test_files_
    for (p, package) in test_dirs(options, {}):
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 348, in test_dirs
    p = import_name(p)
        ^^^^^^^^^^^^^^
  File "/Users/maurits/foo/plone.app.uuid/.tox/test/lib/python3.11/site-packages/zope/testrunner/find.py", line 423, in import_name
    __import__(name)
ModuleNotFoundError: No module named 'plone.app.uuid'
test: exit 1 (6.56 seconds) /Users/maurits/foo/plone.app.uuid> zope-testrunner --all --test-path=/Users/maurits/foo/plone.app.uuid -s plone.app.uuid pid=38144
.pkg: _exit> python /Users/maurits/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  test: FAIL code 1 (76.97=setup[70.41]+cmd[6.56] seconds)
  evaluation failed :( (77.12 seconds)

The editable install generated this file from a template in setuptools:

$ cd .tox/test
$ cat lib/python3.11/site-packages/__editable___plone_app_uuid_2_2_3_dev0_finder.py 
import sys
from importlib.machinery import ModuleSpec, PathFinder
from importlib.machinery import all_suffixes as module_suffixes
from importlib.util import spec_from_file_location
from itertools import chain
from pathlib import Path

MAPPING = {'plone': '/Users/maurits/foo/plone.app.uuid/plone'}
NAMESPACES = {}
PATH_PLACEHOLDER = '__editable__.plone.app.uuid-2.2.3.dev0.finder' + ".__path_hook__"
...

When I edit MAPPING to the following, and directly call the test command, instead of running tox again, it works:

MAPPING = {
    'plone': '/Users/maurits/foo/plone.app.uuid/plone',
    'plone.app': '/Users/maurits/foo/plone.app.uuid/plone/app',
}

I don't know if that makes any sense, or if the NAMESPACES should be filled.

Workaround: force using an older setuptools, by adding this in pyproject.toml:

[build-system]
# See https://snarky.ca/what-the-heck-is-pyproject-toml/
requires = ["setuptools<68.1.0", "wheel"]
build-backend = "setuptools.build_meta"

I suspect this is a regression caused by PR #3995, so cc @aganders3.
It may need a fix similar to PR #4020.

[abravalheri]

Thank you very much @mauritsvanrees for opening the issue.

This is my take on the minimal reproducer:

> docker run --rm -it python:3.11-bullseye /bin/bash

mkdir -p /tmp/pkg1/plone/app/uuid
cd /tmp/pkg1    
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > plone/app/uuid/__init__.py
cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.uuid",
    version="42",
    packages=find_packages(),
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools==68.1.2", "wheel"]
build-backend = "setuptools.build_meta"
EOF

mkdir -p /tmp/pkg2/plone/app/other
cd /tmp/pkg2
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > plone/app/other/__init__.py

cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.other",
    version="42",
    packages=find_packages(),
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools==68.1.2", "wheel"]
build-backend = "setuptools.build_meta"
EOF


cd /tmp
python3.11 -m venv .venv
.venv/bin/python -m pip install -U 'pip==23.2'
.venv/bin/python -m pip install -e pkg1
.venv/bin/python -m pip install -e pkg2
.venv/bin/python -c 'from plone.app import uuid, other; print(uuid, other)'

I think this is related to the fact that it is very hard to support namespace packages with PEP 660.

I have the impression that the -nspkg.pth¹ was already broken before 68.1.0 but it was masked by the specific implementation of the meta path finder².

I think a fix similar to #4020 would not be possible². Instead maybe the best is to fix the -nspkg.pth file that is generated³.

Footnotes

1. pkg_resources.declare_namespace requires a special -nspkg.pth file to be produced. ↩

2. Changing the finder again would not be a good option. Personally, I don't know how to solve this problem without using the logic that was previously implemented in the Finder, but that was problematic from the point of view of case sensitivity.
   I also would not like to add the churn of handling file system case sensitivity on setuptools itself as initially suggested in Make _EditableFinder case-sensitive on case-insensitive filesystems (for certain editable installations) #3995 (it not only hinders maintenance but also decrease performance), so I still believe the best is to delegate to the stdlib's import machinery. ↩ ↩²

3. I suspected the problem is that it is getting rewritten every time a new package using the same namespace is installed. My guess is that each package should generate a new one, with a different name. The -nspkg.pth file only get rewritten if its name contain .. ↩

[mauritsvanrees]

I may have found a way around this. You mentioned PEP 660. Quoting from the part about what to put in the wheel (emphasis mine):

Build backends may choose to place a .pth file at the root of the .whl file, containing the root directory of the source tree. This approach is simple but not very precise, although it may be considered good enough (especially when using the src layout) and is similar to what setup.py develop currently does.

This got me thinking: would moving to a src layout help? And it seems it does!
I created a PR for plone.app.uuid that does this, and there the tests pass.

Ah, but maybe this only works for one package at a time. I adapted your minimal example (thanks!), and there plone.app.uuid cannot be imported, but plone.app.other can:

mkdir -p /tmp/pkg1/src/plone/app/uuid
cd /tmp/pkg1
echo '__import__("pkg_resources").declare_namespace(__name__)' > src/plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > src/plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > src/plone/app/uuid/__init__.py
cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.uuid",
    version="42",
    packages=find_packages(where="src"),
    package_dir={"": "src"},
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools==68.1.2", "wheel"]
build-backend = "setuptools.build_meta"
EOF

mkdir -p /tmp/pkg2/src/plone/app/other
cd /tmp/pkg2
echo '__import__("pkg_resources").declare_namespace(__name__)' > src/plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > src/plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > src/plone/app/other/__init__.py

cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.other",
    version="42",
    packages=find_packages(where="src"),
    package_dir={"": "src"},
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools==68.1.2", "wheel"]
build-backend = "setuptools.build_meta"
EOF


cd /tmp
python3.11 -m venv .venv
.venv/bin/python -m pip install -U 'pip==23.2'
.venv/bin/python -m pip install -e pkg1
.venv/bin/python -m pip install -e pkg2
.venv/bin/python -c 'from plone.app import uuid; print(uuid)'
.venv/bin/python -c 'from plone.app import other; print(other)'

Output of the last two commands:

# .venv/bin/python -c 'from plone.app import uuid; print(uuid)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name 'uuid' from 'plone.app' (/tmp/pkg2/src/plone/app/__init__.py)

# .venv/bin/python -c 'from plone.app import other; print(other)'
hello world plone.app.other
<module 'plone.app.other' from '/tmp/pkg2/src/plone/app/other/__init__.py'>

[mauritsvanrees]

I created a PR for plone.app.event that also switches to a src-layout. In tox.ini this does an editable install of the src-layout branch of plone.app.uuid. The tests pass, and both packages can be imported.
So this is different than in my copy of your minimal example.

[abravalheri]

Hi @mauritsvanrees, does #4041 work for your original problem?

You can test by first installing it (together with wheel) in your venv:

pip install wheel https://github.com/abravalheri/setuptools/archive/refs/heads/issue-4039.zip

and then installing your project with --no-build-isolation in pip.

[abravalheri]

This got me thinking: would moving to a src layout help? And it seems it does!

Yeah, the src layout definitely simplifies things and make supporting namespaces easier.

There is still a problem with the naming of the *-nspkg.pth file that I solved in #4041.
If you are using the src layout that should be enough to get things right...

[abravalheri]

I still had to modify a bit the logic of editable_wheel in #4041 to make it work on pkg_resources-style namespaces... and it likely does not work for pkgutil. I will try to explain the reason below.

I was a bit disappointed because I was expecting the *-nspkg.pth trick to work even when mixing editable and regular installations for packages that share the same namespace.

---

If I understood correctly the way the *-nspkg.pth files are supposed to work, they "fix" the __path__ of the packages, even before you run your first import. But for some reason, the importlib machinery is surprisingly "undoing" the work of *-nspkg.pth half-way through...

For example, if I get the part of #4041 that fix the name uniqueness of *-nspkg.pth (but leave out the part that I changed the values passed to the meta path finder), and try to run a test that mixes editable installs and conventional installs, it will fail:

> docker run --rm -it python:3.11-bullseye /bin/bash

mkdir -p /tmp/pkg1/plone/app/uuid
cd /tmp/pkg1    
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > plone/app/uuid/__init__.py
cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.uuid",
    version="42",
    packages=find_packages(),
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
EOF

mkdir -p /tmp/pkg2/plone/app/other
cd /tmp/pkg2
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/__init__.py
echo '__import__("pkg_resources").declare_namespace(__name__)' > plone/app/__init__.py
echo 'print(f"hello world {__name__}")' > plone/app/other/__init__.py

cat <<EOF > setup.py
from setuptools import setup, find_packages

setup(
    name="plone.app.other",
    version="42",
    packages=find_packages(),
    namespace_packages=[
        "plone",
        "plone.app",
    ]
)
EOF
cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
EOF


cd /tmp
python3.11 -m venv .venv
.venv/bin/python -m pip install -U 'pip==23.2'
.venv/bin/python -m pip install https://github.com/abravalheri/setuptools/archive/39245fc7503a82d7ba7f1706b2e599c5aecb1cdf.zip
.venv/bin/python -m pip install 'wheel==0.41.2'
.venv/bin/python -m pip install -e ./pkg1 --no-build-isolation
.venv/bin/python -m pip install ./pkg2 --no-build-isolation
.venv/bin/python -c 'from plone.app import uuid, other; print(uuid, other)'
# hello world plone.app.other
# Traceback (most recent call last):
#   File "<string>", line 1, in <module>
# ImportError: cannot import name 'uuid' from 'plone.app' (unknown location)

Investigating further we can see that the importlib machinery actually removes the entry in __path__ that the *-nspkg.pth adds, after you import the first package:

.venv/bin/python
Python 3.11.4 (main, Jul 28 2023, 05:20:14) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.modules["plone.app"].__path__
_NamespacePath(['/tmp/.venv/lib/python3.11/site-packages/plone/app', '/tmp/pkg1/plone/app'])
>>> import plone.app.other
hello world plone.app.other
>>> sys.modules["plone.app"].__path__
_NamespacePath(['/tmp/.venv/lib/python3.11/site-packages/plone/app'])

So the only thing I can do is to modify the meta path finder again to cover for this case as well...
But that is increasingly difficult, specially on the light of #3994.

(With #3994 in mind, any attempt to "guess" file location for modules during import time becomes very un-maintainable/difficult-to-do/un-performant because Python does not have good tools to deal with case-insensitive-yet-case-preserving file systems in the way PEP 235 requires - at least not exposed as part of the public API).

At the end of the day, I settled for using the information explicitly given via setup(namespace_packages=...). However that does not work for pkgutil-style namespaces.

[mauritsvanrees]

Hi @mauritsvanrees, does #4041 work for your original problem?

You can test by first installing it (together with wheel) in your venv:

pip install wheel https://github.com/abravalheri/setuptools/archive/refs/heads/issue-4039.zip

and then installing your project with --no-build-isolation in pip.

Hi @abravalheri.
Yes, I confirm this works for my original problem, so without the switch to src-layouts.

Checking with src-layout... yes, this works as well.

Thank you!

I am not familiar with the code, and can't really judge whether the fix is good, except that it works for me.

I suppose if the choice is to support either pkg_resources or pkgutil namespaces, it makes more sense to support pkg_resources, as that is part of setuptools.

Note that in Plone we do want to move to native namespaces, but that will have to wait till a next major release, which I expect at the earliest in 2025.

[abravalheri]

Thank you @mauritsvanrees for testing.

I plan to release a beta version soon that will allow users to experiment with the accumulated changes. If everything goes well we can release a final version next week.

I suppose if the choice is to support either pkg_resources or pkgutil namespaces, it makes more sense to support pkg_resources, as that is part of setuptools.

I don't think we have this choice right now (unless we decide to reimplement the convoluted way how stdlib's import machinery deals with case-insensitive-but-preserving file systems).

[abravalheri]

I plan to release a beta version soon that will allow users to experiment with the accumulated changes. If everything goes well we can release a final version next week.

Well, I tried to create a beta version... but the CI process automatically converted the beta into a full blown version, so I had no choice there and the new version is out 😅.

[mauritsvanrees]

I tried my original problem repo again, without any special setuptools settings. This failed.
Then I required the new setuptools version in pyproject.toml and then it worked:

[build-system]
requires = ["setuptools>=68.2", "wheel"]

So the new release works. Thanks!

[mauritsvanrees]

Out of scope for this issue, but I wonder why setuptools 68.1.2 is still used if I don't explicitly ask for a newer version in pyproject.toml. Warning: long comment ahead.

I don't have a question here, just sharing some notes in case someone else comes across this issue and wonders why a certain setuptools version is used. And this "someone else" can be me in a few months. ;-)

Because if I undo the pyproject.toml change, an import fails again. I see an environment is created (by tox) to build the package, and it uses the previous setuptools version:

$ .tox/.pkg/bin/pip freeze --all
pip==23.2.1
setuptools==68.1.2
wheel==0.41.2

With my pyproject.toml change, the setuptools version in this build env becomes 68.2.

I am not sure why it picks 68.1.2 when I do not explicitly require a minimum version. The tox env where the editable package is used, has an older version:

$ .tox/test/bin/pip freeze --all | grep setuptools
setuptools==68.0.0

So where does the 68.1.2 version come from? Ah, this comes from the environment where the tox command is installed:

$ /Users/maurits/.local/pipx/venvs/tox/bin/python -m pip freeze --all | grep setuptools
setuptools==68.1.2

Well, not entirely. When I downgrade or upgrade setuptools in that env, still 68.1.2 is used.

Ah, the python in there is really a Python installed by Mac homebrew . I thought I was using pyenv everywhere, but maybe pipx (which I used to install tox) does this differently, because pipx is installed by homebrew. This Python has setuptools 68.1.2:

$ /usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m pip freeze --all | grep setuptools
setuptools @ file:///usr/local/Cellar/python%403.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ensurepip/_bundled/setuptools-68.1.2-py3-none-any.whl#sha256=ccf32376ed5c23b0f24a76dd4c7123394820f0b7c5ca43f4e67eee465cfaee94

Interestingly, the pipx command self is installed by homebrew in an env with a much older setuptools:

$ /usr/local/Cellar/pipx/1.2.0/libexec/bin/python3.11 -m pip freeze --all | grep setuptools
setuptools==65.6.3

When I call python, I get version 3.11.5 from pyenv, and it has these packages:

$ python --version
Python 3.11.5
$ which python
/Users/maurits/.pyenv/shims/python
$ python -mpip freeze --all
pip==23.2.1
setuptools==65.5.0

Does it help if I let pipx use a fresh tox with explicitly this python?

$ pipx run --python python tox -e test
⚠️  tox is already on your PATH and installed at /Users/maurits/.local/bin/tox. Downloading and running anyway.
test: recreate env because python changed executable='/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/bin/python3.11'->'/Users/maurits/.pyenv/versions/3.11.5/bin/python3.11'
test: remove tox env folder /Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/.tox/test
test: install_deps> python -I -m pip install zope.testrunner -c https://dist.plone.org/release/6.0-dev/constraints.txt
.pkg: recreate env because python changed executable='/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/bin/python3.11'->'/Users/maurits/.pyenv/versions/3.11.5/bin/python3.11'
.pkg: remove tox env folder /Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/.tox/.pkg
.pkg: install_requires> python -I -m pip install setuptools wheel
.pkg: _optional_hooks> python /Users/maurits/.local/pipx/.cache/f34b78656886670/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_editable> python /Users/maurits/.local/pipx/.cache/f34b78656886670/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: install_requires_for_build_editable> python -I -m pip install wheel
.pkg: build_editable> python /Users/maurits/.local/pipx/.cache/f34b78656886670/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
test: install_package_deps> python -I -m pip install Products.CMFCore Products.ZCatalog plone.app.testing plone.dexterity plone.indexer plone.testing plone.uuid setuptools zope.interface zope.publisher -c/Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/.tox/test/constraints.txt
test: install_package> python -I -m pip install --force-reinstall --no-deps /Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/.tox/.tmp/package/4/plone.app.uuid-2.2.3.dev0-0.editable-py3-none-any.whl
test: commands[0]> zope-testrunner --all --test-path=/Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid -s plone.app.uuid
Traceback (most recent call last):
...
ModuleNotFoundError: No module named 'plone.app.uuid'
$ .tox/.pkg/bin/pip freeze --all | grep setuptools
setuptools==68.1.2

So: no, this does not help. It may be specific for how tox installs the development package. Manually it is fine:

$ cd .tox/test/
$ bin/python -m pip freeze --all | grep setuptools
setuptools==68.0.0
$ bin/python -m pip install --no-deps -e ../../
...
$ bin/python -c "import plone.app.uuid; print(plone.app.uuid)"
<module 'plone.app.uuid' from '/Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/plone/app/uuid/__init__.py'>

And the tests pass.

I can install the previous setuptools version in the env and it still works:

$ bin/python -m pip install setuptools==68.1.2
$ bin/python -m pip install --no-deps -e ../../
...
 bin/python -c "import plone.app.uuid; print(plone.app.uuid)"
<module 'plone.app.uuid' from '/Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/plone/app/uuid/__init__.py'>

If I then try without build isolation, then I expect it to fail:

$ bin/python -m pip install --no-deps --no-build-isolation -e ../../
...
$ bin/python -c "import plone.app.uuid; print(plone.app.uuid)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'plone.app.uuid'

Update to latest setuptools and it should work again:

$ bin/python -m pip install -U setuptools
...
Successfully installed setuptools-68.2.0
$ bin/python -m pip install --no-deps --no-build-isolation -e ../../
...
$ bin/python -c "import plone.app.uuid; print(plone.app.uuid)"
<module 'plone.app.uuid' from '/Users/maurits/community/plone-coredev/6.0/src/plone.app.uuid/plone/app/uuid/__init__.py'>

Summary: it can be tricky to predict which setuptools version will be used to build a package.

• If you need a minimum/maximum version, add this in the pyproject.toml [build-system].
• If you want to use the setuptools version from your current environment, use the --no-build-isolation option.
• If the required version from the build-system differs from what is installed in your env, and you use --no-build-isolation, then the version from the build-system wins.

For tox, you can influence the packaging environment and force a specific setuptools version for building packages:

[pkgenv]
deps = setuptools==67.0.0

I tried the following for my use case, but the -c gives an error, although it works fine in a normal env:

[pkgenv]
deps =
    pip
    setuptools
    wheel
    -c https://dist.plone.org/release/6.0-dev/constraints.txt

Other ways to get tox to behave how you want, would be let it not install the package itself, but use the exact command that you want, something like this:

[testenv:test]
...
use_develop = false
skip_install = true
commands_pre =
    pip install --no-build-isolation -e {toxinidir}[test]

Anyway, best seems to be to specify the build-system in pyproject.toml, and use a minimum and/or maximum version for setuptools if you know you need it.