Merge pull request #252 from MAAP-Project/add-ref-to-example-yaml-files

Author: emileten

docs/source/system_reference_guide/custom-environments.ipynb

@@ -38,7 +38,7 @@
     "\n",
     "### Package manager\n",
     "\n",
-    "We use `mamba` (a fast `conda` drop-in replacement) as a package manager to install, update or remove packages (libraries). `mamba` works with 'environments' that are directories in your local file system containing a set of packages. When you work 'in a given environment', it means you that your programs will look for dependencies in that environment's `mamba` directory. All workspaces launch with a environment called `base`, which is a `mamba` environment that has all the pre-installed libraries. If you open a terminal launcher after creating a `Basic Stable` workspace : \n",
+    "We use `mamba` (a fast `conda` drop-in replacement) as a package manager to install, update or remove packages (libraries). `mamba` works with 'environments' that are directories in your local file system containing a set of packages. When you work 'in a given environment', it means that your programs will look for dependencies in that environment's `mamba` directory. All workspaces launch with an environment called `base`, which is a `mamba` environment that has all the pre-installed libraries. If you open a terminal launcher after creating a `Basic Stable` workspace : \n",
     "\n",
     "\n",
     "![Base conda environment location](../_static/base_conda_environment_location.png)\n",
@@ -49,14 +49,19 @@
     "\n",
     "**Note : any modification to the `base` environment does not survive a workspace restart. In other words, modifications to `/opt/conda` disappear after a workspace restart**. \n",
     "\n",
-    "Extending an existing `mamba` environment means adding packages on top of what it contains, which works provided there are no dependency conflicts. You can use a configuration file specifying all the new libraries to add. It is recommended for reproducibility and shareability. See the below sections for configuration file usage. \n",
-    "\n",
-    "Alternatively, you can install libraries using the `mamba install` command to install additional packages in your current environment (run `mamba --help` to learn more about how to use `mamba` commands). For example :\n",
+    "Extending an existing `mamba` environment means adding packages on top of what it contains, which works provided there are no dependency conflicts. You can install libraries using the `mamba install` command to install additional packages in your current environment (run `mamba --help` to learn more about how to use `mamba` commands). For example :\n",
     "\n",
     "```\n",
     "mamba install xarray\n",
     "```\n",
-    "\n"
+    "\n",
+    "However, it is recommended to use configuration files for reproducibility and shareability. With this approach, assuming your configuration file is named `config.yml`, the command to use is : \n",
+    "\n",
+    "```\n",
+    "mamba env update -f config.yml\n",
+    "```\n",
+    "\n",
+    "For more details on configuration files, see the [Custom environments section](#Custom-environments) and for an example of this command, refer to the [subsection about updating an environment with a configuration file](#Updating-an-existing-environment-with-a-configuration-file).\n"
    ]
   },
   {
@@ -66,19 +71,21 @@
    "source": [
     "## Custom environments\n",
     "\n",
-    "Note that for the rest of this README, you can find the example configuration files in the `examples-environment-configuration-files` page, in the same section as this page.\n",
+    "*For the rest of this README, in each section we provide a link to download an example YAML configuration file.*\n",
     "\n",
     "You can use the `mamba` CLI to create a new, custom environment. The parameters (the list of libraries, the location where to search for them, etc...) can be passed either from a configuration YAML file or directly on the console. We recommend using the first option (a YAML file is easier to share and modify). \n",
     "\n",
     "### Basic custom environment\n",
     "\n",
-    "See the `env-example` in the examples notebook. This configuration installs specific versions `python`, `pandas` and `geopandas` from `conda-forge`. If versions aren't specified, the latest is installed. We recommend to always specify the version for reproducibility. The basic command to create this environment would be :\n",
+    "*Example config file for this section [here](./example_conda_configuration_files/env-example.yml).*\n",
+    "\n",
+    "This configuration installs specific versions `python`, `pandas` and `geopandas` from `conda-forge`. If versions aren't specified, the latest is installed. We recommend to always specify the version for reproducibility. The basic command to create this environment would be :\n",
     "\n",
     "```\n",
     "mamba env create -f env-example.yml\n",
     "```\n",
     "\n",
-    "However, this stores this environment files in `/opt/conda`, which a directory that is reset when the workspace restarts, and so custom environments are lost. Therefore, you want to specify a storage location in your user directory with the `--prefix` parameter\n",
+    "However, this stores this environment files in `/opt/conda`, which is a directory that is recreated when the workspace restarts, and so custom environments are lost. Therefore, you want to specify a storage location in your user directory with the `--prefix` parameter\n",
     "\n",
     "```\n",
     "mamba env create -f env-example.yml --prefix /projects/env\n",
@@ -92,19 +99,31 @@
     "\n",
     "### Updating an existing environment with a configuration file\n",
     "\n",
-    "You can update an existing environment with a configuration file as well. For example, let's assume you have a `mamba` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. See the `env-extend` example in configuration examples notebook. Running `mamba env update -f env-extend.yml` will update `base` by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n",
+    "*Example config file for this section [here](./example_conda_configuration_files/env-extend.yml).*\n",
+    "\n",
+    "You can *update* an existing environment with a configuration file as well. For example, let's assume you have a `mamba` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. Using the linked example config : \n",
+    "\n",
+    "```\n",
+    "mamba env update -f env-extend.yml\n",
+    "```\n",
+    "\n",
+    "This command will update `base` by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n",
     "\n",
     "\n",
     "### Using `pip` for python packages\n",
     "\n",
-    "Some python packages might not be availabe in the channel you are using, or in any `mamba` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `mamba` environment to download packages. The recommended way is to specify this in the configuration file. An example of this configuration in the examples notebook is `env-with-pip`. In that example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n",
+    "*Example config file for this section [here](./example_conda_configuration_files/env-with-pip.yml).*\n",
+    "\n",
+    "Some python packages might not be availabe in the channel you are using, or in any `mamba` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `mamba` environment to download packages. The recommended way is to specify this in the configuration file. In the linked example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n",
     "\n",
     "### Using custom environments in jupyter notebooks\n",
     "\n",
+    "*Example config file for this section [here](./example_conda_configuration_files/env-with-ipykernel.yml).*\n",
+    "\n",
     "The following instruction steps are for python kernels.\n",
     "\n",
-    "- Make sure ipykernel is listed as a dependency in your configuration file. See the `env-with-ipykernel` example in the examples notebook.\n",
-    "- Create your environment using that configuration file.\n",
+    "- Make sure ipykernel is listed as a dependency in your configuration file.\n",
+    "- Create your environment using the linked configuration file.\n",
     "- Install the environment as a kernel by running the following command (parameter values follow the example mentioned):\n",
     "  ```\n",
     "  python -m ipykernel install --user --name env-with-ipykernel --display-name \"Python env-with-ipykernel\".\n",
@@ -115,7 +134,9 @@
     "\n",
     "### Suggested packages for custom environment\n",
     "\n",
-    "MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` mamba environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. See the `env-with-maap-py` example in the examples notebook. You can note that inside of it `maap-py` is 'versioned' using a commit hash (at the end of the github URL). "
+    "*Example config file for this section [here](./example_conda_configuration_files/env-with-maap-py.yml)*\n",
+    "\n",
+    "MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` mamba environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. See the configuration example linked. You can note that in the example, `maap-py` is 'versioned' using a commit hash (at the end of the github URL). "
    ]
   }
  ],