Test coverage with Docker builds

[hugovk]

Hoisting from python-pillow/Pillow#2761 (comment)

We're not getting coverage from Docker builds.

There's some divergence already with lcov (eg. CentOS 6):

Setting up lcov (1.10-1build1) ...
Setting up libgd-gd2-perl (1:2.46-3.1build1) ...
Capturing coverage data from .
Found gcov version: 4.8.4
Scanning . for .gcda files ...
geninfo: ERROR: no .gcda files found in .!

Versus (eg. native Python 2.7):

Setting up lcov (1.10-1build1) ...
Setting up libgd-gd2-perl (1:2.46-3.1build1) ...
Capturing coverage data from .
Found gcov version: 4.8.4
Scanning . for .gcda files ...
Found 1 data files in .
Processing _webp.gcda
Finished .info-file creation

One thing: Docker and native both run the after_success.sh script (which collates and sends coverage data off) but not install.sh or script.sh (in which Pillow is built with the coverage switch).

Are the Dockers built with lcov/CFLAGS="-coverage"?

Where is the Pillow build defined for Docker?

[wiredfool]

If you look at the Dockerfiles, they're all running /test.sh, which activates the virtualenv, then runs make clean && make install && test-installed.py .

[hugovk]

Okay, so native is doing in script.sh:

coverage erase
python setup.py clean
CFLAGS="-coverage" python setup.py build_ext --inplace

coverage run --append --include=PIL/* selftest.py
coverage run --append --include=PIL/* -m nose -vx Tests/test_*.py

And Docker is doing from the Makefile:

clean:
	python setup.py clean
	rm PIL/*.so || true
	rm -r build || true
	find . -name __pycache__ | xargs rm -r || true
...
install:
	python setup.py install
	python selftest.py --installed

then test-installed.py calls nose.main() with no coverage.

---

So perhaps add this to the Makefile:

install-coverage:
	CFLAGS="-coverage" python setup.py build_ext install
	python selftest.py --installed

And have the Dockerfiles run:

make clean && make install-coverage && test-installed.py . --with-coverage

[wiredfool]

We might be able to pass the CFLAGS in via travis, but the --with-coverage flag will have to be a change in the Dockerfiles.

Do we have all the OS packages we need for coverage in the various builds?