Unnecessary rebuilding of dependencies

[sjakobi]

I'm seeing this strange behavior at sjakobi/store@1a4c632. Every time I run a benchmark, some dependencies are unregistered and rebuilt, even when there is no change on any file that stack should care about.

The most interesting line from the verbose logs appears to be this one:

2017-03-01 15:59:36.313604: [debug] Ignoring package distributive, from (InstalledTo Local,"/home/simon/src/store/.stack-work/install/x86_64-linux/nightly-2017-02-28/8.0.2/pkgdb/"), due to wrong location: (Just (InstalledTo Local),Snap)

(It would be nice if the log said what the expected location was.)

distributive and a few (related?) packages are rebuilt again and again and apparently registered in the wrong package db. I think this might be related to containers being a local extra-dep in this configuration.

I haven't properly minimized this yet, just wanted to report it in case someone else could reproduce it.

Full logs – with and without -v – here.

$ stack --version
Version 1.3.3, Git revision 8b0ab0bf687a225b2d724f743db50a20ddcc303c (dirty) (4613 commits) x86_64 hpack-0.17.0

[mgsloan]

Whew, this was tricky to track down.

Minimal repro is to create a new project with the default template, add distributive to build-depends and have the following in your stack.yaml:

packages:
- '.'
- location:
    git: https://github.com/sjakobi/containers
    commit: 9612b20
  extra-dep: true
# - location:
#     git: https://github.com/ekmett/tagged
#     commit: 298215b
#   extra-dep: true

Uncommenting the tagged repo causes things to work properly.

I figured out that the issue is that loadSourceMapFull, for distributive, is yielding PSUpstream with Snap. Later during the build process it figures out that it needs to be Local, due to it depending on things in the local db, causing reinstallation.

Here's why this is happening:

• stack dot --external shows that distributive has a direct dependency on Cabal, despite it not being in build-depends. Cabal depends on containers, and so when a local version of containers is used, the Cabal version must also be local. The dep on Cabal comes from custom-setup deps.

• shadowMiniBuildPlan is used to handle the logic of figuring out which packages in the snapshot instead need to be installed locally. The MiniBuildPlan didn't treat Cabal as a dep of containers.

• The BuildPlan loading code used a function called packageDependencies, which doesn't take into consideration custom-setup deps.

I've opened a PR against the release candidate, since it added support for custom-setup: #3039

[mgsloan]

Fixed! (or at least it will be once the release candidate branch is merged back to master)

[borsboom]

The merge back to master is done.