Fix and document --allow-paths

[cameel]

Fixes #4623.
Fixes #10059.
Depends on #11686 (draft until that one's merged). Merged.

Cases that this PR fixes:

• Neither allowed paths nor imports were being normalized before being compared. This broke:
  • Even very simple cases like whitelisting ../dir/ or ./dir/.
  • Non-normalized paths like dir1/../dir2/ dir1/../dir2/, dir1///dir2/.
  • Paths going beyond root like /../../dir/.
  • Symbolic links inside allowed paths were not resolved (only the ones in the imports were).
  • Whitelisted relative paths could only match imports using relative paths as well. Same for absolute paths.
  • All of this applied also to implicit whitelisting of directories due to being remapping targets or base path.
• Base path was not being automatically whitelisted.
• Empty paths would just whitelist the whole filesystem. There were several ways they could appear:
  • Given explicitly (e.g. --allow-paths=a/,,b/ or --allow-paths ""). These are now ignored.
  • Resulting from a remapping with an empty target (e.g. x=). These are now ignored. After Fix remappings with empty targets #11685 they will not be possible at all.
  • Resulting from a remapping to a file located at the VFS root (x=contract.sol). These now whitelist the working dir.
• When a remapping target ended with /.., wrong directory was whitelisted (e.g. for x=/a/b/.. the whitelisted dir was /a/b/, not /a/).
  • Note that this is just a special case. In cases like x=/a/b/c.sol the right directory is actually /a/b/.

[cameel]

I resolved all the CI problems I had here except for one that's happening on Windows and this one is actually something that I cannot really fix on our side. I suspect it's a bug in boost::filesystem. I reported it in boostorg/filesystem#201.

The issue is basically that some paths that have redundant .. segments (e.g. /a/../a/b/) make weakly_canonical() throw on Windows. This was not a problem with --base-path because it requires the directory to actually exist. --allow-paths does not and I don't think it can - if it did, the targets of remappings would have to always exist on disk.

One workaround I can think of (other than waiting for this to be fixed in boost) would be to just require these paths not to have such redundant segments. Possibly only on Windows. The user can always avoid it by using a normalized path anyway.

[cameel]

This now passes all tests. I disabled tests that trigger the bug in boost (but only on Windows and on boost <= 1.76). The problem is still there until boost 1.77 gets released and we start using it for release binaries. And even then anyone who builds with older boost will run into it.

I think that's better than waiting until 1.77 is released and increasing the required version. Increasing it might cause problems for people who build on distros that do not have cutting-edge libraries and the bug is Windows-only anyway. To trigger it someone has to specifically use --allow-paths with a path to a directory that does not exist and is unnormalized in a very specific way - either enters and exits a directory (e.g. a/../a/b/) or goes beyond FS root and ends with .. (e.g. /../a/..). It also does not affect --base-path even though we normalize it before checking if the dir exists - that's because the bug is triggered only when we use weakly_canonical() to get symlinks resolved.

[cameel]

#11686 approved, which now unblocks this PR.

[cameel]

Commits squashed.

[christianparpart]

@cameel It's a huge PR, took a while, sorry. But I think that's all "right" :)

[cameel]

Thanks!

Well, it's big but it's either that or splitting it into multiple PRs that depend on each other and in the past I've seen that long chains of dependent PRs get pretty confusing. Still, not sure if this turned out better. I'll probably go back to splitting them next time because that way the parts are smaller and easier to review.