Supporting builddir in cabal.project and global config

[merijn]

To the best of my knowledge, new-X don't currently obey builddir entries in cabal.project. There was some speculation that there was a technical reason for this, but at first glance of the code it just seems the problem is laziness. That is, all code paths dealing with computing the DistDirLayout try and resolve the cabal.project path and builddir in a single step.

It seems fairly straightforward* to split this into two steps, first looking up cabal.project and the project path, then checking establishing the DistDirLayout after that. I would assume that the logical priority for determining builddir would be:

1. commandline --builddir flag
2. builddir in cabal.project
3. default to dist-newstyle

If no one has any theoretical objections I'll try and see if I can get it working as straightforwardly as I hope it to be :p

[*] - Famous last words.

[alexbiehl]

I think it won't be that easy. There is a knot:

• Parsing of configuration (basically rebuildProjectConfig) uses file monitors to check for changes in config files
• File monitors use a cache to be able to tell file changes over multiple runs of cabal
• These caches have to be stored somewhere... and of course its the dist directory

So rebuildProjectConfig, the function that reads and assembles our project configuration, actually writes to dist dir on a cold start. Not sure how you would work around that.

[alexbiehl]

Maybe we should bite the bullet and stop caching of ProjectConfig stuff?

[merijn]

Well, we can just parse the cabal.project, check what the builddir is, THEN check with the cache if anything changed and things need to be rerun. Sure, that introduces a redundant read/parse of cabal.project if the config hasn't changed, but I doubt reading and parsing cabal.project is so slow that it will noticeably impact the speed of any cabal command.

[hvr]

@merijn the problem is that you're assuming that we've already fully implemented all features; but there's more planned, like file-globbing includes for cabal project files. And then I'm afraid the induced latency would be a lot more noticeable, and it'd also risk destroying the latency savings that caching would give us (I care to keep the no-op detection latency way below 100ms, as otherwise cabal new-run becomes useless for my scripting)

[merijn]

I think the ability to specify a builddir loses most of it's value if you can't do it per-project and you have to always specify it on the commandline for each command. At that point, why even support specifying a custom builddir at all?

[23Skidoo]

Looks like it's not certain we actually want to implement this, so labeled as "discussion".

[AndreasPK]

Maybe we should bite the bullet and stop caching of ProjectConfig stuff?

I've also had issues with the stored (outdated) config overwriting command line settings for store-dir.
Might also also be a result of the cache not being recognized properly as outdated.

I have not opened a Ticket on that yet as it's not quite clear to me when that happens.

[hvr]

I had a brief discussion with @alexbiehl for two potential pragmatic workarounds; either,

• Introduce a new file cabal.builddir which is located the same way cabal.project is, but merely contains a string denoting the --builddir argument value; this is basically a textual symlink (Git uses something like this too inside its .git folder), or

• Allow builddir: ... inside cabal.project files but in a restricted way that can be scanned for by a trivial dumb parser, and w/o needing to read the full cabal.project file; this should allow to keep the latency overhead well in the 1ms order of magnitude.

The latter approach could be implemented as follows:

The key/value pair

builddir: <path>

• must occur (completely, i.e. including a terminating LF) within the first 4096 bytes;
• it must start in column one, and
• must be a single line, and it
• must be lower-case and there
• must be no whitespace between builddir and :
• there must at most be a single occurrence of builddir (as we only scan for the first one)

we can validate this easily when doing a full parse of the cabal.project by doing a consistency check (the full parser must report the same builddir as the lightweight parser; this is a similiar scheme to the one we already used for the cabal-version scanning)

Then scanning is very cheap, assuming that projfrag contains the prefix of a cabal.project file:

• Try stripPrefix "builddir:" projfrag, if successful take rest of line
• Otherwise, breakSubstring "\nbuilddir:" projfrag, if found, take rest of the line after the :
• Otherwise, there was no (valid) --builddir

[23Skidoo]

I lean towards the second solution.