dependency not getting installed in .stack-work with stack 1.3.0

[wolftune]

related to #2845 this is specifically occurring with https://github.com/snowdriftcoop/snowdrift

Tested on multiple systems (all Linux-based)

With Stack 1.2.0, everything is fine. With Stack 1.3.0, the yesod binary is not found. It's not getting put into .stack-work correctly.

If I build successfully with stack 1.2.0, I can keep the stack-work, then build with stack 1.3.0 and still have things work because the binary built before is still present.

This probably relates to snowdrift using an alternate forked version of yesod-bin with a git address. It seems stack 1.3.0 has a bug where it isn't installing binaries in stack-work correctly when the binary comes from an extra dependency…

[mitchellwrosen]

Hmm... I filed #2845, but I'm not sure it's related (or even a bug!). I was using 1.2.1, as noted in the ticket.

[mitchellwrosen]

However, I can reproduce this bug using 1.3; here's how:

$ git clone https://github.com/snowdriftcoop/snowdrift
$ cd snowdrift
$ stack build yesod-bin
$ stack exec -- which yesod

Expected output: yesod installed somewhere in .stack-work/
Actual output: nothing

(I also tried stack build yesod-bin:exe:yesod - no dice)

[mitchellwrosen]

It also doesn't have to do with the forked yesod-bin package, as pointing at the actual repo at the proper tag (as below) exhibits the same behavior.

- location:
    git: https://github.com/yesodweb/yesod
    commit: d1495bad85565f75ce0238545dbec1d4257aba24
  subdirs:
    - yesod-bin
  extra-dep: true

[mitchellwrosen]

Here's a log of stack build -v yesod-bin:exe:yesod --no-time-in-log:

Version 1.3.0 x86_64 hpack-0.15.0
[debug] Checking for project config at: /home/mitchell/snowdrift/stack.yaml
@(Stack/Config.hs:863:9)
[debug] Loading project config file stack.yaml
@(Stack/Config.hs:881:13)
[debug] Trying to decode /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-6.17.cache
@(Data/Store/VersionTagged.hs:68:5)
[debug] Success decoding /home/mitchell/.stack/build-plan-cache/x86_64-linux/lts-6.17.cache
@(Data/Store/VersionTagged.hs:72:13)
[debug] Run process: /sbin/ldconfig -p
@(System/Process/Read.hs:306:3)
[debug] Process finished in 0ms: /sbin/ldconfig -p
@(System/Process/Read.hs:306:3)
[debug] Run process: /usr/bin/gcc -v
@(System/Process/Read.hs:306:3)
[debug] Process finished in 0ms: /usr/bin/gcc -v
@(System/Process/Read.hs:306:3)
[debug] PIE enabled
@(Stack/Setup.hs:574:17)
[debug] Found shared library libtinfo.so.5 in /usr/lib
@(Stack/Setup.hs:563:38)
[debug] Found shared library libtinfo.so.6 in /usr/lib
@(Stack/Setup.hs:563:38)
[debug] Found shared library libncursesw.so.6 in 'ldconfig -p' output
@(Stack/Setup.hs:550:29)
[debug] Found shared library libgmp.so.10 in 'ldconfig -p' output
@(Stack/Setup.hs:550:29)
[debug] Did not find shared library libgmp.so.3
@(Stack/Setup.hs:564:38)
[debug] Using standard GHC build
@(Stack/Setup.hs:597:9)
[debug] Getting global package database location
@(Stack/GhcPkg.hs:55:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:103:13)
[debug] Getting Cabal package version
@(Stack/GhcPkg.hs:170:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:306:3)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:306:3)
[debug] Process finished in 13ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Process finished in 12ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Read.hs:306:3)
[debug] Process finished in 23ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Read.hs:306:3)
[debug] Resolving package entries
@(Stack/Setup.hs:252:5)
[debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:163:18)
[debug] Parsing the cabal files of the local packages
@(Stack/Build/Source.hs:298:5)
[debug] Parsing the targets
@(Stack/Build/Source.hs:235:5)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/website/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/website/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/website/Snowdrift.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 6ms: getPackageFiles /home/mitchell/snowdrift/website/Snowdrift.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/crowdmatch/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/crowdmatch/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/crowdmatch/crowdmatch.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 1ms: getPackageFiles /home/mitchell/snowdrift/crowdmatch/crowdmatch.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/persistent.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 4ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/6nk1N5bfncwY/persistent/persistent.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/postgresql-simple-migration.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 4ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/Nl5BUGxz7nps/postgresql-simple-migration.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/run-persist/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/run-persist/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/run-persist/run-persist.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 0ms: getPackageFiles /home/mitchell/snowdrift/run-persist/run-persist.cabal
@(Stack/Package.hs:250:21)
[debug] Exception ignored when attempting to load /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/stack-build-cache: openBinaryFile: does not exist (No such file or directory)
@(Data/Store/VersionTagged.hs:86:9)
[debug] Start: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/yesod-bin.cabal
@(Stack/Package.hs:250:21)
[debug] Finished in 2ms: getPackageFiles /home/mitchell/snowdrift/.stack-work/downloaded/bnIHMPumBlP2/yesod-bin/yesod-bin.cabal
@(Stack/Package.hs:250:21)
[debug] Finding out which packages are already installed
@(Stack/Build/Installed.hs:68:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 15ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Ignoring package haskeline due to wanting version 0.7.2.3 instead of 0.7.2.1
@(Stack/Build/Installed.hs:191:5)
[debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1
@(Stack/Build/Installed.hs:191:5)
[debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0
@(Stack/Build/Installed.hs:191:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/.stack/snapshots/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 46ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/.stack/snapshots/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/snowdrift/.stack-work/install/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Process finished in 11ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /home/mitchell/snowdrift/.stack-work/install/x86_64-linux/lts-6.17/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Read.hs:306:3)
[debug] Trying to decode /home/mitchell/.stack/indices/Hackage/00-index.cache
@(Data/Store/VersionTagged.hs:68:5)
[debug] Success decoding /home/mitchell/.stack/indices/Hackage/00-index.cache
@(Data/Store/VersionTagged.hs:72:13)
[debug] Constructing the build plan
@(Stack/Build/ConstructPlan.hs:159:5)
[debug] Checking if we are going to build multiple executables with the same name
@(Stack/Build.hs:196:5)
[debug] Executing the build plan
@(Stack/Build/Execute.hs:454:5)
[debug] Getting global package database location
@(Stack/GhcPkg.hs:55:5)
[debug] Run process: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)
[debug] Process finished in 11ms: /home/mitchell/.stack/programs/x86_64-linux/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Read.hs:306:3)

[mitchellwrosen]

I think this behavior changed in 58d4f42, and it was intentional.

The solution for the snowdrift guys (cc @wolftune) is to simply set extra-dep: false for the yesod-bin local package.

[mgsloan]

Alternatively, require that stack build yesod-bin is run. Sounds like this has been resolved, thanks for investigating @mitchellwrosen !

[mitchellwrosen]

@mgsloan No problem, but can you clarify what you mean by that last comment?

[wolftune]

@mgsloan the source of the issue has been identified, but I'm not sure that we know that the way the behavior happens is indeed the desired result. At any rate, your understanding is not correct because stack build yesod-bin does not succeed in our case in getting past this issue!

I'm testing right now to see if workaround from @mitchellwrosen succeeds here. Again, not sure that it's the correct answer or a workaround for something that isn't right in stack.

[mitchellwrosen]

@wolftune It's not a workaround; as I said the change to not treat packages with extra-dep: true as targets was intentional =)

[wolftune]

Confirmed resolution from @mitchellwrosen — but I don't myself understand the meaning here of extra-dep: false versus extra-dep: true and why it would (or wouldn't?) make sense to include a location in stack.yaml that isn't an extra-dep. Clearly, I did not follow the recent changes to stack and the .yaml syntax.

I agree this is thus solved, as far as I can tell. Perhaps the erroneous suggestion from @mgsloan is just a red herring.

[mitchellwrosen]

Well, looking over the commit message I don't actually know for sure if @snoyberg realized he was changing the behavior of stack build <local dep with executable component>.

@wolftune as far as I know, you have a forked version of yesod-bin for the sole purpose of manually typing stack build yesod-bin, no? It's a little odd that it can't be marked as extra-dep: true so that it does't (for example) squelch the build output of the package we really care about... the very reason the commit was made in the first place!

[wolftune]

@mitchellwrosen no, that wasn't a reason at all (as far as I thought / know). The reason for the fork was to work with some new TH we added.

See:
https://git.snowdrift.coop/sd/yesod/commit/fa15b526dcc4acfdb4ec4536d5edadbcfcd6391d
and
https://git.snowdrift.coop/sd/yesod/commit/e6af409c68d217b8e33ef9f4e267935e07b25f65

[wolftune]

And, in conjunction with that, I don't find it transparent (to me at least) whether the extra-dep true or false variable would be a concern for this case (or similar cases)