Reduce test matrix

[hugovk]

Closes #365.

Similar to the reasoning in #182, I'd like to suggest reducing the build matrix.

There are now 76 jobs (only looking at Travis CI), testing 8 Python versions against 4 pytest versions (spanning 3 major pytest releases), 4 xdist versions and 2 Coverage.py versions.

• Is it really necessary to test against all those old versions?

• Does pytest-cov need to make promises to support old releases?

• Do those projects promise to support so many old releases, or just the most recent?

Further, some two dozen of the build jobs are failing, giving less confidence in the CI/tests as a whole.

https://docs.travis-ci.com/user/customizing-the-build/#Build-Matrix says:

All build matrixes are currently limited to a maximum of 200 JOBS for both private and public repositories. If you are on an open-source plan, please remember that Travis CI provides this service free of charge to the community. So please only specify the matrix you actually need.

Python

Now: 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, py, py3

I suggest it's not necessary to test EOL Python versions (3.4): pytest and Coverage.py do not support it. --> Removed in #336.

(Consider also dropping Python 2.7 soon, compare the pytest plan and create a pytest-cov plan.)

pytest

Now: 3.10, 4.6, 5.1, 5.2

I suggest it's not needed to test 3.10 (released Nov 2018) at all.

pytest 4.6 is the last to support Python 2.7 and 3.4, so that is needed whilst they are still needed here.

Only test the latest pytest 5.x, remove 5.1.

xdist

Now: 1.27, 1.29, 1.30

I'd suggest only testing the latest 1.30, but as this is failing the CI I've not included any xdist changes in this PR.

Coverage.py

Now 4.5, 5a

Keep these, it is necessary to test both latest Coverage.py 4.5 and the upcoming Coverage.py 5 due to large changes.

Time improvement

	Before	After
Travis jobs	76	49
Travis total resource time	7h15m	4h40m
Travis end-to-end time	1h15m	40m
AppVeyor time	2h50m	1h25m

Thank you!

[blueyed]

@hugovk
Thanks for looking into it - the matrix here is really worth being improved/fixed.

(pytest-dev/pytest#6164 causes 7.5h of build time with 78 jobs.)

It seems like builds are broken currently anyway already for some reason - have you looked into that?

[blueyed]

I've not looked at in detail, but I think it is a good step in general.

[hugovk]

It seems like builds are broken currently anyway already for some reason - have you looked into that?

I had a cursory look, they're test_dist_not_collocated*, test_dist_subprocess_not_collocated and test_dist_missing_data failures, but it was nothing super obvious to me and they also fail in master so I didn't look into it further. It's also why I didn't reduce the xdist matrix here.

[hugovk]

Updated .travis.yml by running python ci/bootstrap.py.

The jobs are the same, they're just in a different order.

[ionelmc]

I updated some versions to latest. Will do a rebase soon, looks like something is conflicting.

[ionelmc]

@nicoddemus I can't edit the project on appveyor, can you go in the settings and enable rolling builds? It's taking too long to not have rolling builds on.