setup.py generated by `poetry build` does not have included file in package_data

[bdoms]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• OS version and name: Ubuntu 19.04

• Poetry version: 0.12.17

Issue

I have a web project that builds a static JS file, which I have explicitly ignored in my .gitignore file, but that I do want to include in builds of my project. Adding that file to the include directive in my pyproject.toml works correctly in that the file is included in the sdist tar, HOWEVER, it's missing from the package_data entry in the generated setup.py, so when I install the package, the file is not included and the installed package is broken.

Here's the setup. I have two files in a package sub-directory:

web/static
|-- app.js
|-- utils.js

Then in .gitignore this line:

web/static/app.js

In pyproject.toml:

packages = [
    {include = "web"}
]

include = ["web/static/app.js"]

Running poetry build -f sdist succeeds. But then setup.py looks like this:

packages = \
['web']

package_data = \
{'': ['*'],
 'web': ['static/utils.js']}

I would expect that a file specified in include would have an entry there, or be covered by another rule, but that's not the case and it breaks things for me.

Let me know if you need more info or how I can help. Poetry is awesome.

[bdoms]

Wanted to throw out a potential fix. I don't have time to test this myself right now, but I believe that putting something in the builder around https://github.com/sdispater/poetry/blob/master/poetry/masonry/builders/builder.py#L63 like this would correct the issue:

for included_glob in self._package.include:
    for included in glob(
        os.path.join(self._path.as_posix(), str(included_glob)), recursive=True
    ):
        vcs_ignored_files.discard(
            Path(included).relative_to(self._path).as_posix()
        )

[con-f-use]

see: pypa/setuptools#1650

[funkyfuture]

i hit the same when i try to include a marker according to PEP561:

[tool.poetry]
# …
include = ["mypackage/py.typed"]
# …

which results in

# …
package_data = \
{'': ['*']}
# …

but the file is not excluded in any vcs-related context.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bdoms]

Thanks for the warning stale bot. I just tried this again after a poetry self:update and it is definitely still happening, and still an issue if poetry wants to support sdist builds.

[sdispater]

@bdoms Can you try with the latest beta version of the 1.0.0 release?

[gijzelaerr]

This seems to work correctly for version 1.0.0b4 for me.

[bdoms]

@sdispater thanks for checking in! I upgraded to 1.0.0b5 and this is still a problem: a file I had excluded in .gitignore but explicitly included in pyproject.toml is there in the tar but not in setup.py.

The code has moved around a bit since I last checked, but looking at it now the problem appears to be this line: https://github.com/sdispater/poetry/blob/master/poetry/masonry/builders/sdist.py#L137

It's the else case for the if isinstance(include, PackageInclude) check when creating the list of files that becomes the package_data. I have no idea why the files I need fail that check, but all of the files I've been having problems with fail there, so they hit the pass line, and thus are not included in the list.

[ErikBjare]

I'm not sure how related my problem is to this issue and if it warrants a separate issue, but include = ['package/file.py'] doesn't seem to override my .gitignore as I expected it to. It does not end up in my installed package in site-packages when I run pip install ..

.gitignore:

package/file.py

pyproject.toml:

[tool.poetry]
include = ["package/file.py"]
...

$ pip install .
(installs successfully)
$ ls .../site-packages/package/
...
(no file.py)

Removing the entry from .gitignore works fine, however.

Running poetry build seems to correctly include the file in the wheel and sdist (and doesn't if I don't include the tool.poetry.include entry, as expected).

I'm working on it in ActivityWatch/aw-qt#51

Edit: I made a dirty workaround that temporarily comments out the file from .gitignore before building (and uncomments when build is done), but would appreciate a clean fix.

[fcomabella]

i hit the same when i try to include a marker according to PEP561:

[tool.poetry]
# …
include = ["mypackage/py.typed"]
# …

which results in

# …
package_data = \
{'': ['*']}
# …

but the file is not excluded in any vcs-related context.

I'm having the same issue.

[yanyang729]

I am on Poetry version 1.0.0b4 and having the exact same issue

[KaiserKarel]

Trying to include some API definitions like so

include = ["../api/*.capnp"]

Am I having problems due to this issue; or are relative paths outside the pyproject.toml directory not supported?