Support sub-library dependencies

[kk-hainq]

Drunk attempt to address #5318. Would probably need #5558 first, and a lot of refactoring for dependency and sub-library code in general.

Note: Documentation fixes for https://docs.haskellstack.org/en/stable/ should target the "stable" branch, not master.

Please include the following checklist in your PR:

• Any changes that could be relevant to users have been recorded in the ChangeLog.md
• The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

[hasufell]

@mpilgrem this seems like a major issue. Has anyone looked into it?

[mpilgrem]

@hasufell, I think the short answer is No. I think the issue falls outside the scope of my own capabilities.

My understanding of the issue is that the Cabal library is experimenting with supporting multiple public libraries in a single package (haskell/cabal#5526 and haskell/cabal#5660, the latter still being an 'open' issue).

The Stack issue #5318 gives the example of a Cabal package pkg-a that has a second, visibility: public, library pkg-a2; and a package pkg-b that seeks to depend both on pkg-a and pkg-a:pkg-a2.

As regards a version of stack built with Cabal >= 3.4, if I try to stack build pkg-b, with a version of stack built with Cabal-3.4.1.0, I get a couple of Cabal 'experimental feature' warnings followed by a couple of Stack 'not supported' warnings and then, ultimately, what I think is a Cabal error message 'Encountered missing or private dependencies: ...':

Cabal file warning inD:\Users\mike\Code\GitHub\demo233\pkg-a\pkg-a.cabal@47:21: visibility is experimental feature (issue #5660)
Cabal file warning inD:\Users\mike\Code\GitHub\demo233\pkg-b\pkg-b.cabal@31:13: colon specifier is experimental feature (issue #5660)
SubLibrary dependency is not supported, this will almost certainly fail
SubLibrary dependency is not supported, this will almost certainly fail
pkg-a> configure (lib + internal-lib)
pkg-a> Warning: pkg-a.cabal:47:21: visibility is experimental feature (issue #5660)
pkg-a> Configuring pkg-a-0.1.0.0...
pkg-a> build (lib + internal-lib)
pkg-a> Preprocessing library for pkg-a-0.1.0.0..
pkg-a> Building library for pkg-a-0.1.0.0..
pkg-a> [1 of 2] Compiling Paths_pkg_a
pkg-a> [2 of 2] Compiling SrcA
pkg-a> Preprocessing library 'pkg-a2' for pkg-a-0.1.0.0..
pkg-a> Building library 'pkg-a2' for pkg-a-0.1.0.0..
pkg-a> [1 of 2] Compiling Paths_pkg_a
pkg-a> [2 of 2] Compiling SrcA2
pkg-a> copy/register
pkg-a> Installing library in D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\lib\x86_64-windows-ghc-9.0.2\pkg-a-0.1.0.0-7TTcRpdD52qDE7TR5sXSqB
pkg-a> Installing internal library pkg-a2 in D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\lib\x86_64-windows-ghc-9.0.2\pkg-a-0.1.0.0-7vCElFzsG48AFwRrOIRTIG-pkg-a2
pkg-a> Registering library for pkg-a-0.1.0.0..
pkg-a> Registering library 'pkg-a2' for pkg-a-0.1.0.0..
pkg-b> configure (lib)
Warning: pkg-b.cabal:31:13: colon specifier is experimental feature (issue
#5660)
Configuring pkg-b-0.1.0.0...
Cabal-simple_Z6RU0evB_3.4.1.0_ghc-9.0.2.exe: Encountered missing or private
dependencies:
pkg-a:{pkg-a, pkg-a2} ==0.1.0.0

Completed 2 action(s).

--  While building package pkg-b-0.1.0.0 (scroll up to its section to see the error) using:
      C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_3.4.1.0_ghc-9.0.2.exe --builddir=.stack-work\dist\d53b6a14 configure --with-ghc=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.0.2\bin\ghc-9.0.2.exe --with-ghc-pkg=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.0.2\bin\ghc-pkg-9.0.2.exe --user --package-db=clear --package-db=global --package-db=C:\sr\snapshots\d9ec050a\pkgdb --package-db=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\pkgdb --libdir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\lib --bindir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\bin --datadir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\share --libexecdir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\libexec --sysconfdir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\etc --docdir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\doc\pkg-b-0.1.0.0 --htmldir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\doc\pkg-b-0.1.0.0 --haddockdir=D:\Users\mike\Code\GitHub\demo233\.stack-work\install\8b0d0ec5\doc\pkg-b-0.1.0.0 --dependency=base=base-4.15.1.0 --dependency=pkg-a=pkg-a-0.1.0.0-7TTcRpdD52qDE7TR5sXSqB --ghc-options -haddock --extra-include-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20220503\mingw64\include --extra-lib-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20220503\mingw64\lib --extra-lib-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20220503\mingw64\bin --exact-configuration --ghc-option=-fhide-source-paths
    Process exited with code: ExitFailure 1

[philderbeast]

My understanding of the issue is that the Cabal library is experimenting with supporting multiple public libraries in a single package (haskell/cabal#5526 and haskell/cabal#5660, the latter still being an 'open' issue).

@mpilgrem I've used this feature in plugins-for-blobs. I've had no problem with it and really like being able to package related libraries together (as we can already do for executables and test suites).

library thoralf-plugin-defs
  visibility: public
  exposed-modules:
      Plugins.Thoralf.UnitDefs
  hs-source-dirs:
      thoralf-uom/defs/src
  build-depends:
      base >=4.9.1.0 && <5
    , plugins-for-blobs:thoralf-encode
    , plugins-for-blobs:thoralf-plugin
    , plugins-for-blobs:thoralf-plugin-uom
    , plugins-for-blobs:thoralf-theory
    , plugins-for-blobs:uom-quantity
    , plugins-for-blobs:uom-th
    , template-haskell >=2.9

I've tried building this with stack too and it works, see #5839 with the rebased the pull request for sub-library dependencies from @kk-hainq.

I tested this on a much larger project and found a problem. I built stack depending on Cabal-3.8.1.0 but stack still harks back to Cabal-3.2.1.0 when building the setup. Can you give me some pointers on how to configure stack so that it can forget about Cabal-3.2.1.0 and go all-in with Cabal-3.8.1.0?

stack/src/Stack/Types/Config.hs

Lines 2039 to 2045 in 55291b3

	, cpCabalVersion :: !Version
	-- ^ This is the version of Cabal that stack will use to compile Setup.hs files
	-- in the build process.
	--
	-- Note that this is not necessarily the same version as the one that stack
	-- depends on as a library and which is displayed when running
	-- @stack ls dependencies | grep Cabal@ in the stack project.

[mpilgrem]

@philderbeast, on the compiler's Cabal version, see my comment on your pull request.

[mpilgrem]

@philderbeast, the master branch version of Stack now builds with a dependency on Cabal-3.8.1.0.

[mpilgrem]

I am going to close this pull request because: (a) its main ideas (extending the Package type and Stack.Package.packageFromPackageDescription; and updating the cabal-sublibrary-dependency integration test are now incorporated in Stack's code base in the master branch); and (b) it has been overtaken by #5839, which is dealing with the residual problem of the cabal-sublibrary-dependency integration test passing.