Proposal: split e2e tests into their respective repositories

[arschles]

For example, all the tests that use the deis CLI to run stuff against the controller, should be back in deis/workflow and the git push tests should be in deis/builder. This repository should pull all those tests together and run them in a harness.

[arschles]

Requesting comments/ideas @jchauncey @slack @sgoings

[arschles]

@mboersma I know you have some PRs in flight right now, so let me know when you're done and I'll begin the split-out process

[sgoings]

How would one "run all the e2e tests" after this split up?

[jchauncey]

So my first thought is to do the following:

1. individual components have a directory that contains all the necessary go code to produce a binary that can be run to fully test that component. So for example logger would have a logger_test binary (producible out of the e2e testing directory). This would allow developers to run this test against an existing kubernetes cluster by filling in a few pieces of information like the k8s api. (In a container of course)
2. This part is still up for some debate:
   2a. We pull in all the component e2e testing directories as deps in a larger project and build all the binaries there. Package those up in a container and have a master script call each binary to run those tests.
   2b. Pull in all the deps and have a master testing binary for the full e2e suite.

[arschles]

I agree mostly with @jchauncey. Specifically, each component can have a package called tests or similar, and ginkgo build can make a test binary from that code. We can also build a docker image around that binary for testing inside k8s as he said.

However, we can construct that code in such a way that this repository can import into a "master" binary, which we can also ginkgo build into a binary and build a docker image around that binary as well.

I hope I've explained that sufficiently It'll be fairly self explanatory after showing a prototype for a test or 2.

[Cryptophobia]

This issue was moved to teamhephy/workflow-e2e#17