sdist should follow symlinks

[sjakobi]

Summary: instead of going with the originally proposed design we decided that cabal sdist should be changed to follow symlinks. Symlinks are not portable anyway, so the generated tarballs shouldn't include them. Original ticket description below.

---

For multi-package projects it would be nice if one could include e.g. common READMEs or CONTRIBUTING files in each package.

Currently, when I try this with stack sdist using Cabal-1.24.0.0, I get the following message:

'extra-src-files: ../README.md' is a relative path outside of the source tree. This will not work when generating a tarball with 'sdist'.

[23Skidoo]

What do you think sdist should do in this case?

[sjakobi]

It should include the README.md in the base directory of the tarball.

[ezyang]

What if it's ../../README.md? Then there isn't a "correct" directory?

[sjakobi]

How about a heuristic like this:

Reference	Path in tarball
../file1	file1
../../file2	file2
../../dir3/file3	dir3/file3

I think that should be predictable and sufficient for most cases?

[sjakobi]

When two referenced files would be located at the same path in the tarball, I think cabal should simply raise an error.

[ezyang]

It feels like a bad idea. Let's see if we can come up with more corner cases ;)

If I cabal unpack a package, I generally expect to cd in and then build it. How are .. references handled in this case?

[23Skidoo]

@sjakobi Perhaps your problem could be also solved by making sdist follow symlinks?

[sjakobi]

@ezyang:

Let's see if we can come up with more corner cases ;)

Please do! I wouldn't mind if cabal would error out for all the corner cases. I think simple cases like ../README.md would cover 99% of real world cases.

If I cabal unpack a package, I generally expect to cd in and then build it. How are .. references handled in this case?

Ah, I didn't consider that. I'd expect unpack to keep the directory structure of the tarball but that would break file references from the code.

@23Skidoo:

Perhaps your problem could be also solved by making sdist follow symlinks?

That sounds like a better idea than mine! Does that sound better to you too, @ezyang?

[ezyang]

Yes, that sounds a lot better, esp. since symlinks aren't cross platform anyway, so we should be manually breaking them regardless.

[23Skidoo]

Updated the issue title and and added a summary of the discussion.

[robinp]

Just want to check the status of this. On Linux, doing stack sdist with Cabal 1.24.2 it seems to put the content of the file pointed by the symlink into the tarball. Which is fine for me - but what is this bug about?