Make local testing easier with Vagrantfile

[obestwalter]

While tox solves the problem of testing with many versions of Python it does not solve the problem of installing many versions of Python.

This could be solved by adding a Vagrantfile and prepared boxes on atlas. I started this already and would like to know if there are any objections to add this. I have one Arch Linux box already and the other one would be a Windows 10 with evaluation license.

[fschulze]

If you do this, then please provide docs or scripts to be able to rebuild those boxes from scratch. Otherwise +1

[obestwalter]

My (lazy) approach atm is:

1. take an existing box
2. Modify it to our needs
3. repackage and publish to Atlas, so it is publically accessible

The ideal approach would be to do this with packer but as I am not even shure if this will be used by many people I keep the effort as minimal as possible for now.

Linux is done already - although everything is preinstalled on the box to keep it fast, it will also be updated on provisioning, which also serves as executable documentation how the interpreters are installed and configured).

From the view of a contributor who wants to run tests for their local changes on all interpreters, all they have to do is:

$ cd </path/to/tox/clone>
$ vagrant up linux
$ vagrant ssh
$ tox

Windows is not as simple and I have not done that yet, I haven't played much with Windows 10 yet, but thanks to ssh out of the box it might be doable to run it headless and run tests from the commandline.

It would be even cooler to just fire up the boxes and run the tests with pytest xdist, but this is something I don't know much about yet from own experience and some guidance would be appreciated.

I will add that documentation for contributors - we don't have any docs like that for tox yet, so I might add a page and make a little guide for contributors.

[obestwalter]

Thinking about this a bit more - I think it is better, if I do not add that to the tox project explicitly but create a little repo that provides multi python machines for whoever needs it with a default of including pytest/tox/detox/devpi.

As Vagrantfiles are written in Ruby it is simple to work off a little json config that does the right thing by default and is malleable for other needs as well. I have done that already for an internal project at work and will filch that to make tt more broadly usable right away.

[obestwalter]

I can't find anything that does this already (only something pretty specific for Django) so I will go ahead and create a semi generalized project to provide that rather than having something only for tox.

[nicoddemus]

Nice. I'm curious, how does Windows 10 license works for Vagrant boxes? Is that free for use, free for personal use only, ...?

[obestwalter]

I did a bit of research when I was doing something similar for SaltStack and the result was that it is o.k. to use evalution licenses for testing boxes. IANAL though :)

But this is really the only way to set up any kind of publically available Windows testing boxes. It's being done already: e.g. https://atlas.hashicorp.com/inclusivedesign/boxes/windows10-eval is publically available and could be used as a base.

[nicoddemus]

Thanks for the info!

[obestwalter]

Introducing bindlestiff. Linux version works already. Windows and some friendlier docs coming up soon.

[obestwalter]

Retracting bindlestiff :) - spent today having some fun with it, but in the end this might just be another brainfart of mine and might never really see the day of light in a usable way.

I will keep things simple for now and do instead what the issue says. I started looking into packer to have boxes that can be updated with not too much effort and will send a PR soon to add the Vagrantfile.

If this proves useful, I can still put my rocket science lab coat back on and generalize it for all possible Python projects - past and future.