Automate setup of ghcjs

[mgsloan]

Rough outline of what needs to be done here:

• Create and upload binary distributions for GHCJS
• Modify Stack.Setup to know how to install these binary distributions. If they work the same as the normal GHC ones, then this change should be pretty minimal:
  • Add soptsUseGHCJS :: Bool to SetupOpts
  • Most spots that use the string "ghc" will likely need to instead use "ghcjs" when this bool is true.
  • When downloading, get the url from a new ghcjs section of stack-setup.yaml instead of the ghc section.
  • Probably more I can't anticipate
• Add URLs for the binary distributions to https://raw.githubusercontent.com/fpco/stackage-content/master/stack/stack-setup.yaml
  • NOTE: make sure that extending this won't cause older stack versions to fail

[wuzzeb]

Just to point out, ghcjs doesn't install anything like normal GHC so I think instead what should happen is that once ghcjs is released to hackage (which should hopefully happen soon!), it should just be added into stackage. At that point, stack itself can compile ghcjs. Just to see that this actually works, I just built ghcjs today using stack and the following stack.yaml

packages:
- location:
    git: git@github.com:ghcjs/ghcjs
    commit: f4d437c21e7ec7c929748222151cf95e8efae4e7
resolver: lts-3.2

This gives the ghcjs binary in $PWD/.stack-work/install/x86_64-linux/lts-3.2/7.10.2/bin which isn't too useful, but if ghcjs was in the stackage snapshot then the ghcjs binary would be put into $HOME/.stack/snapshots/x86_64-linux/lts-3.2/7.10.2/bin directory. Then when stack was asked to compile using ghcjs, it can just call the ghcjs binary from the .stack/snapshots directory.

That gets the ghcjs binary. Next, https://github.com/ghcjs/ghcjs-boot/tree/improved-base should be merged into stackage somehow. Indeed, ghcjs-boot is a mini-stackage, with fixed versions of dependencies and a few patches on top of those packages. That way, the given stackage snapshot versions of things like bytestring, aeson, etc. are used.

Finally, the ghcjs-boot exe (source at https://github.com/ghcjs/ghcjs/blob/improved-base/src-bin/Boot.hs) must have parts of it re-implemented in stack. What it does now is clone the ghcjs-boot tree, apply patches, and build using ghcjs all these packages. But this is exactly what stack can do once ghcjs-boot is merged into stackage somehow.

[mgsloan]

Hey, thanks for looking into this!

ghcjs doesn't install anything like normal GHC

I've discussed it with Luite a little bit at ICFP, and it's certainly possible to have a binary build of GHCJS which installs the same way as GHC. This is better than having a source distribution because it takes lots of time to build everything, and things can go wrong along the way. The new GHCJS instructions seem quite promising (just cabal install http://ghcjs.luite.com/improved-base.tar.gz; ghcjs-boot). For me, haskell-src-exts failed to install for some reason, but I haven't looked into it..

It may indeed make sense to have stack install ghcjs from source in the meantime, I'll discuss this further with Luite.

Indeed, ghcjs-boot is a mini-stackage, with fixed versions of dependencies and a few patches on top of those packages. That way, the given stackage snapshot versions of things like bytestring, aeson, etc. are used.

Right, down the road we will need a separate set of stackage snapshots for GHCJS. Until then all dependencies will need to be explicit via the extra-deps field.

[mgsloan]

@wuzzeb , good news! We're going with pretty much the approach you describe. At some later date, it'll hopefully be switched over to binary distributions. @snoyberg Convinced me that we shouldn't "let perfect be the enemy of good" here, which makes sense to me. Here's how it's going to work:

• Download a .tar.gz much like the one at http://ghcjs.luite.com/improved-base.tar.gz
• If a recent enough version of cabal isn't on PATH, then install one locally, from source (needed for ghcjs-boot)
• Build it, hopefully with stack, possibly with cabal-install
• Run ghcjs-boot with the appropriate flags

I've discussed this with Luite a bit, and we've concluded that these pre-release versions of GHCJS should have version numbers which include the build date in them. For example, ghcjs-0.1.0.20150917. Once we have a tarball for a release like this we can do the following:

• Potentially host the tarball? Downloading it from ghcjs.luite.com would also work - it's up to Luite. I noticed that it doesn't support https, but this isn't much of an issue since download's hash is checked.
• Add a GHCJS section to https://github.com/fpco/stackage-content/blob/master/stack/stack-setup-2.yaml
• Merge the ghcjs-setup branch to master (not yet created)

Until this bit happens, I'll probably just have the implementation on the ghcjs-setup branch use the improved-base tarball

[snoyberg]

I'd recommend using Github releases for hosting the tarballs, if possible. That should also save Luite some bandwidth ;).

[mgsloan]

Here's how things currently work:

• It downloads the tarball specified in stack-setup-2.yaml:

ghcjs:
    source:
        ghcjs-0.1.0_ghc-7.10.2:
            url: ...

• It installs GHCJS from source using the stack.yaml in the tarball
• It boots GHCJS in the environment of the stack.yaml, locally installing cabal-install if necessary

TODO:

• Before this can work out of the box, a source release of GHCJS needs to be made, and stack-setup-2.yaml needs to be updated with the URL of the tarball. The GHCJS release needs to include these two PRs I made recently: Make ghcjs-boot --clean work for fresh install ghcjs/ghcjs#406 Add a stack.yaml ghcjs/ghcjs#407 . I don't think this should block merging the code in, though, as it should just make "stack setup" try to do something, where before it would just error out.
• Test if it works on windows

Questions:

• Install Node? I'm leaning towards not doing this unless there's demand. The ghcjs boot process will complain if it isn't around, and the user can install it with their package management of choice.
• Is it fine if we end up installing a new version of cabal to one of the user's snapshots? Seems fine to me, but it does mean that when using exec with that snapshot, it will shadow the system install.
• Should the GHCJS package DB go inside ~/.stack/programs/...? Currently, ghcjs-boot places it in ~/.ghcjs/x86_64-linux-0.1.0-7.10.2/, and doesn't seem to have flags to change it. I can only think of one reason to not leave it the way it is: if it's stored in ~/.stack/programs/.., then fully removing the version of ghcjs and its global DB is just a matter of removing a few things from that directory.

[ocharles]

Isn't node required to build anything that uses Template Haskell?

On Wed, 23 Sep 2015 4:04 am Michael Sloan notifications@github.com wrote:

Here's how things currently work:

• It downloads the tarball specified in stack-setup-2.yaml:

ghcjs:
source:
ghcjs-0.1.0_ghc-7.10.2:
url: ...

It installs GHCJS from source using the stack.yaml in the tarball

It boots GHCJS in the environment of the stack.yaml, locally
installing cabal-install if necessary

TODO:

Before this can work out of the box, a source release of GHCJS needs
to be made, and [
https://github.com/fpco/stackage-content/blob/1a661da435bf0de8da2e331014a18b17ad293dd8/stack/stack-setup-2.yaml]
needs to be updated with the URL of the tarball. The GHCJS release needs to
include these two PRs I made recently: ghcjs/ghcjs#406
ghcjs/ghcjs#406 ghcjs/ghcjs#407
ghcjs/ghcjs#407 . I don't think this should
block merging the code in, though, as it should just make "stack setup" try
to do something, where before it would just error out.

Test if it works on windows

Questions:

Install Node? I'm leaning towards not doing this unless there's
demand. The ghcjs boot process will complain if it isn't around, and the
user can install it with their package management of choice.

Is it fine if we end up installing a new version of cabal to one of
the user's snapshots?

Should the GHCJS package DB go inside ~/.stack/programs/.../?
Currently it ends up in ~/.ghcjs/x86_64-linux-0.1.0-7.10.2/

—
Reply to this email directly or view it on GitHub
#749 (comment)
.

[jwhitley]

I'll argue for not installing node. As you say, users will have a variety of management options and opinions around it. Tools like ndenv make it straightforward to install and manage multiple versions of node, and to automatically select between them on a per-project basis.

OTOH, guarding against and failing vociferously if the provided version of node is known to be incompatible would be very useful.

[mgsloan]

Isn't node required to build anything that uses Template Haskell?

Yup, but it's even more essential than that - it's also used while building GHCJS projects, to run Setup.hs.

OTOH, guarding against and failing vociferously if the provided version of node is known to be incompatible would be very useful.

Yup, I think this is the responsibility of ghcjs-boot. One of the reasons I'm inclined towards not automatically installing node is that we'd need a good way to pick a version. One possibility would be to have each ghcjs version in stack-setup-2 also specify a node version, but this seems ugly.