From 07458f091418b05daef4449f893992d5f0efcd61 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Sat, 25 Mar 2017 20:55:50 +0100 Subject: [PATCH 01/12] add post about isolated environments --- _posts/2017-11-11-environments.md | 132 ++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100644 _posts/2017-11-11-environments.md diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md new file mode 100644 index 00000000..c5aed2ed --- /dev/null +++ b/_posts/2017-11-11-environments.md @@ -0,0 +1,132 @@ +--- +layout: post +title: Working with different package versions +--- + +In science projects often take years until they are complete. During that time +your the libraries you are using will be updated and most have new features you +like to use. But sometimes the upgrade of a library means that old programs and +scripts will stop working. It can also be that you need a new library version +for another project you are working on. The solution to this problem is to +install the same programs multiple times. + +On this post I will explain several ways to keep maintain different versions of +the same program on a computer. For this I'm introducing several ways to create +isolated environments. The guide will mostly focus on python packages but large +parts can independently be applied to other languages and programs as well. + +# Conda Environments + +[conda]() is a general package manager for scientific applications. It is mostly +used for python packages but the system can be used with any programs. The +[conda-forge] community also provides a large collection of scientific software +for python, R and perl. Conda should be your first choice to manage different +software versions. + +In this guide we will only concentrate on creating and managing environments +with conda. For more information on general installation of package please refer +to the [official documentation](). + +To create a new environment for your next project that uses numpy in version +0.10.0 run: + +{% highlight bash %} +conda create -n awesome_project numpy=0.10.0 -y +{% endhighlight %} + +This only create a new environment + +{% highlight bash %} +source activate awesome_project +{% endhighlight %} + +To list your environments + +{% highlight bash %} +conda env list +{% endhighlight %} + +To remove an environment + +{% highlight bash %} +conda remove -all -n awesome_project +{% endhighlight %} + + +More information about conda environments can be found in +the [official documentation](). + +# Linux Environment Modules + +You likely already use environment modules if you ever used a super computer. +They have been a part of most unix systems for decades now. To see if you +currently have any modules installed run. + +{% highlight bash %} +module avail +{% endhighlight %} + +Since the command is called `module` we will refer to environments as modules in +this section. You can show information about a module with + +{% highlight bash %} +module info +{% endhighlight %} + +Modules work by changing environment variables like `PATH` and `LDD_PATH` to +point to software packages you want to use. This means that you should be +familiar with common unix environment variables and installing software in +custom locations. The advantage of this approach though is that you don't have +to install anything new and it can be easier to use software that hasn't been +packaged with conda. + + +## Create a Environment Module + +To start we will create some example environments and later we will show how to +install different versions of vmd. Modules are defined using module-files. So we +first need to create a folder and change into it. + +{% highlight bash %} +mkdir -p ~/modules/modulefiles +cd ~/modules/modulefiles +{% endhighlight %} + +We also need to tell the module system that it should look for module-files in +that folder. Assuming you are using bash run + +{% highlight bash %} +echo "export MODULEPATH=$MODULEPATH:~/modules/modulefiles" >> .bashrc +source .bashrc +{% endhighlight %} + +Our first environment will just set a new environment variable called `TEST` + +{% highlight bash %} +{% endhighlight %} + +More information on linux modules can be found with `man module` and `man modulefile`. + + +## Installing software into an environment + +## Handling conflicts + +The module command allows you to have several modules active at the same time. +To avoid potential version conflicts you have to tell specify in the module file +which other modules are incompatible. + + +# Python Virtual Environments + +Virtual environments will only work for python packages. + +# Automatically Change Environment Based On Folder + +After you created an isolated environment for a project it would be nice if it +would be activated automatically when ever you work on the project. If you use +bash/zsh this can be done on a per folder basis. + + + + From 0624db485394f9b6a44953b089a17f13c8e74b07 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Mon, 27 Mar 2017 19:16:25 +0200 Subject: [PATCH 02/12] remove linux modules To complicated for our purpose --- _posts/2017-11-11-environments.md | 93 +++++++++---------------------- 1 file changed, 26 insertions(+), 67 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index c5aed2ed..7610a0be 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -10,10 +10,8 @@ scripts will stop working. It can also be that you need a new library version for another project you are working on. The solution to this problem is to install the same programs multiple times. -On this post I will explain several ways to keep maintain different versions of -the same program on a computer. For this I'm introducing several ways to create -isolated environments. The guide will mostly focus on python packages but large -parts can independently be applied to other languages and programs as well. +In this post I will explain how conda and python virtual envs can be used to +manage different package versions. # Conda Environments @@ -27,95 +25,56 @@ In this guide we will only concentrate on creating and managing environments with conda. For more information on general installation of package please refer to the [official documentation](). -To create a new environment for your next project that uses numpy in version -0.10.0 run: +As a preparation add our conda-channel to your configuration {% highlight bash %} -conda create -n awesome_project numpy=0.10.0 -y +conda config --add channels MDAnalysis {% endhighlight %} -This only create a new environment +To create a new environment for your next project that uses MDAnalysis in version +0.15.0 run: {% highlight bash %} -source activate awesome_project +conda create -n mda-15 MDAnalysis=0.15.0 -y {% endhighlight %} -To list your environments - -{% highlight bash %} -conda env list -{% endhighlight %} - -To remove an environment - -{% highlight bash %} -conda remove -all -n awesome_project -{% endhighlight %} - - -More information about conda environments can be found in -the [official documentation](). - -# Linux Environment Modules - -You likely already use environment modules if you ever used a super computer. -They have been a part of most unix systems for decades now. To see if you -currently have any modules installed run. +This only create a new environment. You still have to activate it to be able to +use it. {% highlight bash %} -module avail +source activate mda-15 {% endhighlight %} -Since the command is called `module` we will refer to environments as modules in -this section. You can show information about a module with - -{% highlight bash %} -module info -{% endhighlight %} - -Modules work by changing environment variables like `PATH` and `LDD_PATH` to -point to software packages you want to use. This means that you should be -familiar with common unix environment variables and installing software in -custom locations. The advantage of this approach though is that you don't have -to install anything new and it can be easier to use software that hasn't been -packaged with conda. - - -## Create a Environment Module - -To start we will create some example environments and later we will show how to -install different versions of vmd. Modules are defined using module-files. So we -first need to create a folder and change into it. +To list your environments {% highlight bash %} -mkdir -p ~/modules/modulefiles -cd ~/modules/modulefiles +conda env list {% endhighlight %} -We also need to tell the module system that it should look for module-files in -that folder. Assuming you are using bash run +A nice feature of using conda-environments is that they are easy to share with +colleagues or transferred to other computers. To store the state of the +environment we created in a file called `mda-15-environment` {% highlight bash %} -echo "export MODULEPATH=$MODULEPATH:~/modules/modulefiles" >> .bashrc -source .bashrc +conda list --explicit --md5 -n mda-15 > mda-15-environment {% endhighlight %} -Our first environment will just set a new environment variable called `TEST` +You can now copy this file to a colleague or onto another computer. The first 3 +lines also contain instructions how this file can be used with conda. {% highlight bash %} +# This file may be used to create an environment using: +# $ conda create --name --file +# platform: linux-64 {% endhighlight %} -More information on linux modules can be found with `man module` and `man modulefile`. - - -## Installing software into an environment +Instead of just creating environments for specific versions of software packages +you can of course also create a new environment for each project you are working +on. -## Handling conflicts - -The module command allows you to have several modules active at the same time. -To avoid potential version conflicts you have to tell specify in the module file -which other modules are incompatible. +More information about conda environments can be found in +the [official documentation](). # Python Virtual Environments From 8144f289377614b08e0c6e4f0caa13ca05df4de9 Mon Sep 17 00:00:00 2001 From: Richard Gowers Date: Sat, 1 Apr 2017 08:19:08 +0100 Subject: [PATCH 03/12] Update 2017-11-11-environments.md (#48) --- _posts/2017-11-11-environments.md | 64 +++++++++++++++++-------------- 1 file changed, 36 insertions(+), 28 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index 7610a0be..c3bf5307 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -1,62 +1,75 @@ --- layout: post -title: Working with different package versions +title: Managing software versioning using Conda environments --- -In science projects often take years until they are complete. During that time -your the libraries you are using will be updated and most have new features you -like to use. But sometimes the upgrade of a library means that old programs and -scripts will stop working. It can also be that you need a new library version -for another project you are working on. The solution to this problem is to -install the same programs multiple times. +Research projects can often take months to years to complete, however the +precise version of software they use will often have a shorter lifetime than +this. These new versions of software will often include new features which +might be of great use, but they might also introduce changes which break +your existing work and introduce compatibility issues with other pieces +of software. So whilst for existing projects we might wish to freeze +all pieces of software installed, for newer projects we instead want to +use the most up-to-date versions, leaving us needing to install multiple +versions of multiple different pieces of software. + +In this post we will try to explain how conda and Python virtual environments +can be used to precisely manage the software used across many independent +research projects. -In this post I will explain how conda and python virtual envs can be used to -manage different package versions. # Conda Environments -[conda]() is a general package manager for scientific applications. It is mostly -used for python packages but the system can be used with any programs. The -[conda-forge] community also provides a large collection of scientific software -for python, R and perl. Conda should be your first choice to manage different +[Conda](https://conda.io/docs/index.html) is a general package manager for scientific +applications. It is mostly used for Python packages but the system can be used with +any programs. The [conda-forge] community also provides a large collection of scientific software +for Python, R and perl. Conda should be your first choice to manage different software versions. -In this guide we will only concentrate on creating and managing environments +In this guide we will concentrate only on creating and managing environments with conda. For more information on general installation of package please refer to the [official documentation](). -As a preparation add our conda-channel to your configuration +Software is made available through different conda channels, which each act as a +source for different software. When attempting to install packages into a conda +environment, these channels are searched. In this post we will be using the +MDAnalysis channel which you can add to your configuration like so: {% highlight bash %} conda config --add channels MDAnalysis {% endhighlight %} +For each research project, it is advised that you create a new environment so that +the software used in each project does interfere across different projects. To create a new environment for your next project that uses MDAnalysis in version 0.15.0 run: {% highlight bash %} -conda create -n mda-15 MDAnalysis=0.15.0 -y +conda create -n myproject MDAnalysis=0.15.0 -y {% endhighlight %} -This only create a new environment. You still have to activate it to be able to -use it. +This has created a new software environment called `myproject` but has not affected +anything currently! To have access to all the software installed within it we +must first activate it {% highlight bash %} -source activate mda-15 +source activate myproject {% endhighlight %} -To list your environments +To list your available environments {% highlight bash %} conda env list {% endhighlight %} A nice feature of using conda-environments is that they are easy to share with -colleagues or transferred to other computers. To store the state of the -environment we created in a file called `mda-15-environment` +colleagues or transferred to other computers. This allows all team members on +a project to use an identical set of software and makes your research projects +to be reproducible. To store the state of the environment we created in a file +called `myproject-environment` {% highlight bash %} -conda list --explicit --md5 -n mda-15 > mda-15-environment +conda list --explicit --md5 -n myproject > myproject-environment {% endhighlight %} You can now copy this file to a colleague or onto another computer. The first 3 @@ -68,11 +81,6 @@ lines also contain instructions how this file can be used with conda. # platform: linux-64 {% endhighlight %} -Instead of just creating environments for specific versions of software packages -you can of course also create a new environment for each project you are working -on. - - More information about conda environments can be found in the [official documentation](). From 5780eebd1394f6d338b91a6f239ca7d60dce49a6 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Sat, 1 Apr 2017 10:28:43 +0200 Subject: [PATCH 04/12] add virtual env description --- _posts/2017-11-11-environments.md | 50 ++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index c3bf5307..b6e924c0 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -20,15 +20,16 @@ research projects. # Conda Environments -[Conda](https://conda.io/docs/index.html) is a general package manager for scientific -applications. It is mostly used for Python packages but the system can be used with -any programs. The [conda-forge] community also provides a large collection of scientific software -for Python, R and perl. Conda should be your first choice to manage different -software versions. +[Conda](https://conda.io/docs/index.html) is a general package manager for +scientific applications. It is mostly used for Python packages but the system +can be used with any programs. The [conda-forge](https://conda-forge.github.io/) +community also provides a large collection of scientific software for Python, R +and perl. Conda should be your first choice to manage different software +versions. In this guide we will concentrate only on creating and managing environments with conda. For more information on general installation of package please refer -to the [official documentation](). +to the [official documentation](https://conda.io/docs/using/pkgs.html). Software is made available through different conda channels, which each act as a source for different software. When attempting to install packages into a conda @@ -45,7 +46,7 @@ To create a new environment for your next project that uses MDAnalysis in versio 0.15.0 run: {% highlight bash %} -conda create -n myproject MDAnalysis=0.15.0 -y +conda create -n myproject mdanalysis=0.15.0 -y {% endhighlight %} This has created a new software environment called `myproject` but has not affected @@ -63,10 +64,10 @@ conda env list {% endhighlight %} A nice feature of using conda-environments is that they are easy to share with -colleagues or transferred to other computers. This allows all team members on -a project to use an identical set of software and makes your research projects -to be reproducible. To store the state of the environment we created in a file -called `myproject-environment` +colleagues or transferred to other computers. This allows all collaborators on a +project to use an identical set of software and makes your research projects +reproducible. To store the state of the environment we created in a file called +`myproject-environment` {% highlight bash %} conda list --explicit --md5 -n myproject > myproject-environment @@ -82,18 +83,33 @@ lines also contain instructions how this file can be used with conda. {% endhighlight %} More information about conda environments can be found in -the [official documentation](). +the [official documentation](https://conda.io/docs/using/envs.html). # Python Virtual Environments -Virtual environments will only work for python packages. +Virtual environments are a tool to manage different versions of python packages. +To use virtual environments you have to install the virtualenv package first with. -# Automatically Change Environment Based On Folder +{% highlight bash %} +pip install virtualenv +{% endhighlight %} -After you created an isolated environment for a project it would be nice if it -would be activated automatically when ever you work on the project. If you use -bash/zsh this can be done on a per folder basis. +Virtual environments can be created per project directory. +{% highlight bash %} +cd myproject +virtualenv myproject-env +{% endhighlight %} +This will create a new folder `myproject-env`. This folder contains the virtual +environment and all packages you have installed in it. To activate it run. + +{% highlight bash %} +source myproject-env/bin/activate +{% endhighlight %} +Now you can install packages via `pip` without affecting your global environment. +The Hitchhikers Guide to Python has a +good [tutorial](http://docs.python-guide.org/en/latest/dev/virtualenvs/) that +gives a more in depth explanation of virtual environments. From 98fb74bfa07d057acb2d7ec29cc0def38eeea0e1 Mon Sep 17 00:00:00 2001 From: Richard Gowers Date: Sun, 2 Apr 2017 09:15:34 +0100 Subject: [PATCH 05/12] Update 2017-11-11-environments.md --- _posts/2017-11-11-environments.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index b6e924c0..f7a770b3 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -3,7 +3,7 @@ layout: post title: Managing software versioning using Conda environments --- -Research projects can often take months to years to complete, however the +Research projects can often take months to years to complete, but the precise version of software they use will often have a shorter lifetime than this. These new versions of software will often include new features which might be of great use, but they might also introduce changes which break @@ -13,7 +13,7 @@ all pieces of software installed, for newer projects we instead want to use the most up-to-date versions, leaving us needing to install multiple versions of multiple different pieces of software. -In this post we will try to explain how conda and Python virtual environments +In this post we will try to explain how Python virtual environments can be used to precisely manage the software used across many independent research projects. @@ -24,11 +24,10 @@ research projects. scientific applications. It is mostly used for Python packages but the system can be used with any programs. The [conda-forge](https://conda-forge.github.io/) community also provides a large collection of scientific software for Python, R -and perl. Conda should be your first choice to manage different software -versions. +and perl. In this guide we will concentrate only on creating and managing environments -with conda. For more information on general installation of package please refer +with conda. For more general information on installing conda please refer to the [official documentation](https://conda.io/docs/using/pkgs.html). Software is made available through different conda channels, which each act as a @@ -41,7 +40,7 @@ conda config --add channels MDAnalysis {% endhighlight %} For each research project, it is advised that you create a new environment so that -the software used in each project does interfere across different projects. +the software used in each project does not interfere across different projects. To create a new environment for your next project that uses MDAnalysis in version 0.15.0 run: From 2cf14121363af079d319fde95f1d5be031c5d56e Mon Sep 17 00:00:00 2001 From: Richard Gowers Date: Sun, 2 Apr 2017 09:31:23 +0100 Subject: [PATCH 06/12] Update 2017-11-11-environments.md --- _posts/2017-11-11-environments.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index f7a770b3..ef023772 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -8,14 +8,18 @@ precise version of software they use will often have a shorter lifetime than this. These new versions of software will often include new features which might be of great use, but they might also introduce changes which break your existing work and introduce compatibility issues with other pieces -of software. So whilst for existing projects we might wish to freeze -all pieces of software installed, for newer projects we instead want to -use the most up-to-date versions, leaving us needing to install multiple -versions of multiple different pieces of software. +of software. So whilst for newer projects we might want to use the most +up-to-date versions of software, for existing projects we want to be able +to freeze the versions of software that are in use. This leads to us needing +to install and manage multiple versions of software across our various +research projects. +With the upcoming release of 0.16 of MDAnalysis, alongside various improvements, +we are also introducing some changes which could break existing code. In this post we will try to explain how Python virtual environments -can be used to precisely manage the software used across many independent -research projects. +can be used to manage this transition, allowing you to finish existing +projects with version 0.15, while also enjoying the benefits provided in +version 0.16. # Conda Environments From 04968d3d1566fc100cc2cbbca739a22f6f3a5a89 Mon Sep 17 00:00:00 2001 From: Jonathan Barnoud Date: Sun, 2 Apr 2017 10:39:30 +0200 Subject: [PATCH 07/12] expand on virtualenv (#51) --- _posts/2017-11-11-environments.md | 92 ++++++++++++++++++++++++++++--- 1 file changed, 85 insertions(+), 7 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index ef023772..b71936d7 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -90,29 +90,107 @@ the [official documentation](https://conda.io/docs/using/envs.html). # Python Virtual Environments -Virtual environments are a tool to manage different versions of python packages. -To use virtual environments you have to install the virtualenv package first with. +Like conda, virtual environments managed with virtualenv allow you to use +different versions of python and python packages for your different project. On +the contrary to conda, however, virtualenv is not a general-purpose package +manager; it leverages what is available on your system, and let you install +python packages using pip. + +To use virtual environments you have to install the virtualenv package first. +This can be done with either pip or the package manager of your system: {% highlight bash %} pip install virtualenv +# or on ubuntu +sudo apt install virtualenv +# or on fedora +sudo dnf install python-virtualenv {% endhighlight %} Virtual environments can be created per project directory. {% highlight bash %} cd myproject +# if you want to use the default python interpretor virtualenv myproject-env +# or if you want to explicitly use python 2.7 +virtualenv -p /usr/bin/python2.7 myproject-env +# or if you want to use python 3.5 +virtualenv -p /usr/bin/python3.5 myproject-env +# or maybe you want to use pypy +virtualenv -p /usr/bin/pypy myproject-env {% endhighlight %} This will create a new folder `myproject-env`. This folder contains the virtual -environment and all packages you have installed in it. To activate it run. +environment and all packages you have installed in it. To activate it in the +current terminal run: {% highlight bash %} source myproject-env/bin/activate {% endhighlight %} -Now you can install packages via `pip` without affecting your global environment. +Now you can install packages via `pip` without affecting your global +environment. The packages you install when the environment is activated will be +available in terminal sessions that have the environment activated. You can +deactivate the virtual environment by running + +{% highlight bash %} +deactivate +{% endhighlight %} + +The [`virtualenvwrapper` package](https://virtualenvwrapper.readthedocs.io) +makes virtual environment easier to use. It provides some very useful features: + +* it organizes the virtual environment in a single directory, avoiding to have + them scattered throughout the file system; +* it defines command to easy the creation, deletion, and copy of virtual + environments; +* it defines a command to activate a virtual environment using its name; +* all commands defined by `virtualenvwrapper` have tab-completion for virtual + environment names. + +To use `virtualenvwrapper` you first need to install it *outside* of a virtual +environment: + +{% highlight bash %} +pip install virtualenvwrapper +# or on ubuntu +sudo apt install virtualenvwrapper +# or on fedora +sudo dnf install python-virtualenvwrapper +{% endhighlight %} + +Then, you need to have it loaded in your terminal session. Add the following +lines in `~/.bashrc`, they will be executed every time you open a new terminal +session: + +{% highlight bash %} +# Decide where to store the virtual environments +export WORKON_HOME=~/Envs +# Make sure the directory exists +mkdir -p ${WORKON_HOME} +# Load virtualenvwrapper +source /usr/local/bin/virtualenvwrapper.sh +{% endhighlight %} + +Open a new terminal or run `source ~/.bashrc` to update your session. You can +now create a virtual environment with + +{% highlight bash %} +mkvirtualenv my-project +{% endhighlight %} + +Like the `virtualenv` command we saw earlier, `mkvirtualenv` lets you choose +your python interpretor with the `-p` option. Regardless of your current +working directory, we created the virtual environment in `~/Envs/` and it is +now loaded in our terminal session. + +You can load your virtual environments by running `workon my-project`, and +exit them by running `deactivate`. -The Hitchhikers Guide to Python has a -good [tutorial](http://docs.python-guide.org/en/latest/dev/virtualenvs/) that -gives a more in depth explanation of virtual environments. +Virtual environments, especially with `virtualenvwrapper`, can do much more. +The Hitchhikers Guide to Python has a good +[tutorial](http://docs.python-guide.org/en/latest/dev/virtualenvs/) that +gives a more in depth explanation of virtual environments. The +[`virtualenvwrapper` documentation](https://virtualenvwrapper.readthedocs.io) +is also a good resource to read. From 0a55f8f770ceff21c09286e712f3d557ed84aec4 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Sun, 2 Apr 2017 12:36:29 +0200 Subject: [PATCH 08/12] fix up links and minor formatting --- _posts/2017-11-11-environments.md | 39 ++++++++++++++++--------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index b71936d7..996ba592 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -3,32 +3,33 @@ layout: post title: Managing software versioning using Conda environments --- -Research projects can often take months to years to complete, but the -precise version of software they use will often have a shorter lifetime than -this. These new versions of software will often include new features which -might be of great use, but they might also introduce changes which break -your existing work and introduce compatibility issues with other pieces -of software. So whilst for newer projects we might want to use the most -up-to-date versions of software, for existing projects we want to be able -to freeze the versions of software that are in use. This leads to us needing -to install and manage multiple versions of software across our various -research projects. - -With the upcoming release of 0.16 of MDAnalysis, alongside various improvements, -we are also introducing some changes which could break existing code. -In this post we will try to explain how Python virtual environments -can be used to manage this transition, allowing you to finish existing -projects with version 0.15, while also enjoying the benefits provided in -version 0.16. +Research projects can often take months to years to complete, but the precise +version of software they use will often have a shorter lifetime than this. These +new versions of software will often include new features which might be of great +use, but they might also introduce changes which break your existing work and +introduce compatibility issues with other pieces of software. So whilst for +newer projects we might want to use the most up-to-date versions of software, +for existing projects we want to be able to freeze the versions of software that +are in use. This leads to us needing to install and manage multiple versions of +software across our various research projects. + +With the upcoming release of 0.16.0 of MDAnalysis, alongside various +improvements, we are also introducing some changes which could break existing +code. In this post we will try to explain +how [conda environments](#conda-environments) +and [Python virtual environments](#python-virtual-environments) can be used to +manage this transition, allowing you to finish existing projects with version +0.15.0, while also enjoying the benefits provided in version 0.16.0. + # Conda Environments [Conda](https://conda.io/docs/index.html) is a general package manager for scientific applications. It is mostly used for Python packages but the system -can be used with any programs. The [conda-forge](https://conda-forge.github.io/) +can be used with any program. The [conda-forge](https://conda-forge.github.io/) community also provides a large collection of scientific software for Python, R -and perl. +, Perl and others. In this guide we will concentrate only on creating and managing environments with conda. For more general information on installing conda please refer From a316602dfb6ee8373831fa264e7627a66abbb4e0 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Sun, 2 Apr 2017 12:40:32 +0200 Subject: [PATCH 09/12] switch language to conda OR venvs --- _posts/2017-11-11-environments.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index 996ba592..a61d434d 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -15,14 +15,11 @@ software across our various research projects. With the upcoming release of 0.16.0 of MDAnalysis, alongside various improvements, we are also introducing some changes which could break existing -code. In this post we will try to explain -how [conda environments](#conda-environments) -and [Python virtual environments](#python-virtual-environments) can be used to +code. In this post we will explain how [conda environments](#conda-environments) +or [Python virtual environments](#python-virtual-environments) can be used to manage this transition, allowing you to finish existing projects with version 0.15.0, while also enjoying the benefits provided in version 0.16.0. - - # Conda Environments [Conda](https://conda.io/docs/index.html) is a general package manager for From 4d35aaf89387c49ee1031c9c0d608218e9bdb2d9 Mon Sep 17 00:00:00 2001 From: Max Linke Date: Tue, 4 Apr 2017 14:32:00 +0200 Subject: [PATCH 10/12] use conda-forge channel --- _posts/2017-11-11-environments.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-11-11-environments.md index a61d434d..d0e46425 100644 --- a/_posts/2017-11-11-environments.md +++ b/_posts/2017-11-11-environments.md @@ -35,10 +35,10 @@ to the [official documentation](https://conda.io/docs/using/pkgs.html). Software is made available through different conda channels, which each act as a source for different software. When attempting to install packages into a conda environment, these channels are searched. In this post we will be using the -MDAnalysis channel which you can add to your configuration like so: +conda-forge channel which you can add to your configuration like so: {% highlight bash %} -conda config --add channels MDAnalysis +conda config --add channels conda-forge {% endhighlight %} For each research project, it is advised that you create a new environment so that From 5040540bceb5e746ce71225ce8e0ec2cb5eaf40d Mon Sep 17 00:00:00 2001 From: Max Linke Date: Fri, 7 Apr 2017 18:19:48 +0200 Subject: [PATCH 11/12] use correct blog post date --- _posts/{2017-11-11-environments.md => 2017-04-07-environments} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename _posts/{2017-11-11-environments.md => 2017-04-07-environments} (100%) diff --git a/_posts/2017-11-11-environments.md b/_posts/2017-04-07-environments similarity index 100% rename from _posts/2017-11-11-environments.md rename to _posts/2017-04-07-environments From 173087ccb7edf49a4e452c48e88673e45bc550ba Mon Sep 17 00:00:00 2001 From: Max Linke Date: Fri, 7 Apr 2017 18:21:31 +0200 Subject: [PATCH 12/12] add authors --- _posts/2017-04-07-environments | 2 ++ 1 file changed, 2 insertions(+) diff --git a/_posts/2017-04-07-environments b/_posts/2017-04-07-environments index d0e46425..ef29ed88 100644 --- a/_posts/2017-04-07-environments +++ b/_posts/2017-04-07-environments @@ -192,3 +192,5 @@ The Hitchhikers Guide to Python has a good gives a more in depth explanation of virtual environments. The [`virtualenvwrapper` documentation](https://virtualenvwrapper.readthedocs.io) is also a good resource to read. + +- @kain88-de , @jbarnoud and @richardjgowers