Installing development tools

[LourensVeen]

The current set-up has the development tools listed in setup.py, and has you install them via python setup.py .[dev]. This command does two things: it installs your package and its dependencies into your local environment, and it installs the development tools into your local environment.

Next, if you run python setup.py test, setuptools will create a separate environment, install your package in it and any dependencies that aren't already in your local environment, and then run your tests. They pass, you push, and your CI fails, because it runs python setup.py test in a clean environment (not having done python setup.py .[dev] some time ago) and so ends up with a newer version of a library that breaks your code. Locally, it works because it's using the stale dependency that was installed when you installed the development tools.

So I don't like having my package and especially its dependencies installed in my working environment, because it makes my environment less similar to the one on the CI. But the way things are now set up, I can't have that unless I either manually install the development tools, or install them using setup.py and then manually uninstall my own package.

So I'd propose using a requirements-dev.txt instead with a list of development tools, so that you can pip install -r requirements-dev.txt them. But this stuff is complicated, and I'm probably missing some considerations. Comments welcome :-).

[bouweandela]

I would recommend just running your tests in a different environment than where you do your development if you want to be as close to CI as possible, possibly starting from a docker container.

The solution you propose does not guarantee a clean environment either, because your development dependencies may share dependencies with your program. And of course you may have other stuff installed from somewhere else.

Having your package installed in your development environment is nice for various reasons:

• you can actually run it if it is a program
• your editor understands your 3rd party imports, so you can conveniently browse the code of some module that you are importing
• tests are fast to run, because no dependencies need to be (built and) installed

[LourensVeen]

Of course I can work around this, that is not the issue. And of course improving this does not solve all the world's problems. And I'm not proposing to make it impossible to install the package locally. I'm proposing to separate installation of the package and installation of the development tools, so that those who want to do one but not the other can do so conveniently.

[bouweandela]

Maybe we could consider recommending poetry? It looks rather neat, solves this and many other issues, and is recommended on python.org for "use cases where the repository being managed is structured as a Python project", i.e. the kind of software project we recommend. I have no experience with it myself though.

[LourensVeen]

Poetry does look neat, but from a quick look it doesn't look like it installs development tools, just dependencies. And if I understand correctly, all it really does is combine pip install with pip freeze and have you write yet another configuration file. I'm not sure the overhead of using yet another tool is worth the gain here to be honest.

So that doesn't solve my problem, which is that I'd like to be able to install the development tools without doing anything else.

[bouweandela]

I think it does a bit more than what you say, but you're right that it does not allow only installing development tools, so it does not solve your problem.

[fdiblen]

@LourensVeen and @bouweandela
We recently removed requirements.txt (see #100)
and included the dev dependencies in setup.py.

python-template/{{cookiecutter.project_slug}}/setup.py

Line 59 in 4dd4d50

extras_require={

Can we close this issue if you don't have anything else to add?

[egpbos]

I think this can be closed. This is a matter of environment management. There are a few ways to work around this issue that don't involve an extra file (like requirements.dev.txt) and accompanying extra maintenance burden. In addition to what @bouweandela suggested, I would recommend to use tox, which always uses its own environment. In fact, I think tox should be part of the template, but this is another issue :)