general overview of tox - documentation

[jhagege]

Hi,
I would like to understand what is the basic flow of tox (which is an amazing project by the way !):

• I understand there is a run of setup.py in both environments (2.7, 3.6).
• But then, regarding the tests ran with pytest (or other test suite): are they running against package in source code, or against package installed in site-packages (by setup.py install) ?

It's important in my case because my setup.py downloads a binary that I need and puts it in the site-packages directory, which is not originally in my source code package (because I need it to be cross-platform).

Thanks for help !

[obestwalter]

Hi @cyberjoac, thanks for the 💐 :)

About your question: In part this depends on your configuration and in part it depends on the way the project is structured.

To get your desired outcome you need to make sure the following:

0. do not set skipsdist=True (docs)in the global tox config
1. Do not set usedevelope=True (docs) in your env where you want to test the package. tox will then install the package it built into the env and run commands of that env in that context.
2. Make sure that the way you run your tests are actually against the installed package (see below for more details)

From the way how I phrased 2. you can see that you need to be aware that tox has no actual control over what the tests are run against. What tox does is create and activate the v[irtual]env, install what you asked for, cd into the dir that you asked for, pass through the env vars you asked for, etc. ... and then run an arbitrary set of commands that you have chosen. There is no way anyone can tell you what your tests are run against without having all the details of your project and your tox.ini.

I could think of a few ways to make sure that your tests run against the installed package, the way you have set up your project at the moment. The simplest would be to add a test that can only pass in a proper installation (e.g. stupidly checking the path of an imported package filefor containing "site-packages"). If that test fails in a tox run, then you know something is wrong.

There is one specific problem, when you use pytest together with tox, where I best refer you to these comprehensive articles about the problem (and the solution): Packaging a python library and Testing & Packaging.

The short answer how to make always absolutely sure to avoid that pitfall is introducing a project layout that makes it impossible to import modules from a project package when the package is not installed. The layout proposed in that article and that we also use in the tox source code is having your packages in a src folder.

[jhagege]

Thanks much for the incredibly detailed answer, it's very helpful !!

[gaborbernat]

We'll add a flow diagram to the documentation soon, and a general introduction to help new users with getting started. Will keep this issue open to track that.