Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 14 additions & 34 deletions docs/source/system_reference_guide/custom-environments.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -66,77 +66,57 @@
"source": [
"## Custom environments\n",
"\n",
"You can use the `mamba` CLI to create a new, custom environment. \n",
"Note that for the rest of this README, you can find the example configuration files in the `examples-environment-configuration-files` page, in the same section as this page.\n",
"\n",
"The parameters (the list of libraries, the location where to search for them, etc...) can be passed either from a configuration YAML file or directly on the console. We recommend using the first option (a YAML file is easier to share and modify). \n",
"You can use the `mamba` CLI to create a new, custom environment. The parameters (the list of libraries, the location where to search for them, etc...) can be passed either from a configuration YAML file or directly on the console. We recommend using the first option (a YAML file is easier to share and modify). \n",
"\n",
"### Basic custom environment\n",
"\n",
"Here is an example configuration file that we name `env` :\n",
"See the `env-example` in the examples notebook. This configuration installs specific versions `python`, `pandas` and `geopandas` from `conda-forge`. If versions aren't specified, the latest is installed. We recommend to always specify the version for reproducibility. The basic command to create this environment would be :\n",
"\n",
"```\n",
"# env.yml\n",
"name: env\n",
"channels:\n",
"\t- conda-forge\n",
"dependencies:\n",
"\t- python=3.8\n",
"\t- pandas=1.5.3\n",
" - geopandas=0.12.2\n",
"```\n",
"\n",
"It installs specific versions `python`, `pandas` and `geopandas` from either `conda-forge` or `defaults`. If versions aren't specified, the latest is installed. We recommend to always specify the version for reproducibility. The basic command to create this environment would be :\n",
"\n",
"```\n",
"mamba env create -f env.yml\n",
"mamba env create -f env-example.yml\n",
"```\n",
"\n",
"However, this stores this environment files in `/opt/conda`, which a directory that is reset when the workspace restarts, and so custom environments are lost. Therefore, you want to specify a storage location in your user directory with the `--prefix` parameter\n",
"\n",
"```\n",
"mamba env create -f env.yml --prefix /projects/env\n",
"mamba env create -f env-example.yml --prefix /projects/env\n",
"```\n",
"\n",
"and to activate it : \n",
"\n",
"```\n",
"mamba activate env\n",
"mamba activate env-example\n",
"```\n",
"\n",
"### Updating an existing environment with a configuration file\n",
"\n",
"For this section and the next, you can find example configuration files in `example_conda_configuration_files` in the same folder as this notebook. \n",
"\n",
"You can update an existing environment with a configuration file as well. For example, let's assume you have a `mamba` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. You can create a file like `base.yml`, that specifies updates to the `base` environment. Then, running `mamba env update -f base.yml` will update `base` by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n",
"You can update an existing environment with a configuration file as well. For example, let's assume you have a `mamba` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. See the `env-extend` example in configuration examples notebook. Running `mamba env update -f env-extend.yml` will update `base` by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n",
"\n",
"\n",
"### Using `pip` for python packages\n",
"\n",
"Some python packages might not be availabe in the channel you are using, or in any `mamba` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `mamba` environment to download packages. The recommended way is to specify this in the configuration file. You can find an example in `env2.yml`, that creates an environment named `env2`. In that example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n",
"Some python packages might not be availabe in the channel you are using, or in any `mamba` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `mamba` environment to download packages. The recommended way is to specify this in the configuration file. An example of this configuration in the examples notebook is `env-with-pip`. In that example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n",
"\n",
"### Using custom environments in jupyter notebooks\n",
"\n",
"The following instruction steps are for python kernels.\n",
"\n",
"- Make sure ipykernel is listed as a dependency in your configuration file. An example that has such a dependency is env3.yml\n",
"- Create your environment using that configuration file\n",
"- Make sure ipykernel is listed as a dependency in your configuration file. See the `env-with-ipykernel` example in the examples notebook.\n",
"- Create your environment using that configuration file.\n",
"- Install the environment as a kernel by running the following command (parameter values follow the example mentioned):\n",
" ```\n",
" python -m ipykernel install --user --name custom_env_name --display-name \"Python env3\".\n",
" python -m ipykernel install --user --name env-with-ipykernel --display-name \"Python env-with-ipykernel\".\n",
" ```\n",
" The above command installs the environment as a kernel in Jupyter, making it accessible in the notebook with a display name of \"Python env3\".\n",
"- Wait around 30 seconds and launch a new notebook. Among the kernel options, you should see \"Python env3\" listed. Below you can see a screenshot that shows what this step looks like:\n",
" The above command installs the environment as a kernel in Jupyter, making it accessible in the notebook with a display name of \"Python env-with-ipykernel\".\n",
"- Wait around 30 seconds and launch a new notebook. Among the kernel options, you should see \"Python env-with-ipykernel\" listed. Below you can see a screenshot that shows what this step looks like:\n",
"![Register a kernel with a conda environment and launch a notebook with it](../_static/launch_registered_kernel_custom_conda.png)\n",
"\n",
"### Suggested packages for custom environment\n",
"\n",
"MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` mamba environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. An example configuration file can be found at `env4.yml`. You can note that inside of it `maap-py` is 'versioned' using a commit hash (at the end of the github URL). "
"MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` mamba environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. See the `env-with-maap-py` example in the examples notebook. You can note that inside of it `maap-py` is 'versioned' using a commit hash (at the end of the github URL). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: env-example
channels:
- conda-forge
dependencies:
- python=3.8
- pandas=1.5.3
- geopandas=0.12.2
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
name: base
channels:
- conda-forge
dependencies:
- xarray=2023.3.0
- geopandas=0.12.2
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
name: env-with-ipykernel
channels:
- conda-forge
dependencies:
- python=3.9
- ipykernel=6.22.0
- xarray=2023.3.0
- pip=23.0.1
- pip:
- stackstac==0.4.3
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
name: env-with-maap-py
channels:
- conda-forge
dependencies:
- python=3.9
- pip=23.0.1
- pip:
- git+https://git@github.com/MAAP-Project/maap-py@d0f34e3440bf1121882892a5fe0b20484b413d43
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: env-with-pip
channels:
- conda-forge
dependencies:
- python=3.9
- xarray=2023.3.0
- pip=23.0.1
- pip:
- stackstac==0.4.3

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading