|
38 | 38 | "\n", |
39 | 39 | "### Package manager\n", |
40 | 40 | "\n", |
41 | | - "We use `mamba` (a fast `conda` drop-in replacement) as a package manager to install, update or remove packages (libraries). `mamba` works with 'environments' that are directories in your local file system containing a set of packages. When you work 'in a given environment', it means that your programs will look for dependencies in that environment's `mamba` directory. All workspaces launch with an environment called `base`, which is a `mamba` environment that has all the pre-installed libraries. If you open a terminal launcher after creating a `Basic Stable` workspace : \n", |
| 41 | + "We use `conda` with the libmamba solver as a package manager to install, update or remove packages (libraries). `conda` works with 'environments' that are directories in your local file system containing a set of packages. When you work 'in a given environment', it means that your programs will look for dependencies in that environment's `conda` directory. All workspaces launch with an environment called `base`, which is a `conda` environment that has all the pre-installed libraries. If you open a terminal launcher after creating a `Basic Stable` workspace : \n", |
42 | 42 | "\n", |
43 | 43 | "\n", |
44 | 44 | "\n", |
45 | 45 | "\n", |
46 | | - "You can notice that a `base` `mamba` environment is activated, and its libraries are located in `/opt/conda`. \n", |
| 46 | + "You can notice that a `base` `conda` environment is activated, and its libraries are located in `/opt/conda`. \n", |
47 | 47 | "\n", |
48 | 48 | "### Extending the `base` environment in a given workspace session.\n", |
49 | 49 | "\n", |
50 | 50 | "**Note : any modification to the `base` environment does not survive a workspace restart. In other words, modifications to `/opt/conda` disappear after a workspace restart**. \n", |
51 | 51 | "\n", |
52 | | - "Extending an existing `mamba` environment means adding packages on top of what it contains, which works provided there are no dependency conflicts. You can install libraries using the `mamba install` command to install additional packages in your current environment (run `mamba --help` to learn more about how to use `mamba` commands). For example :\n", |
| 52 | + "Extending an existing `conda` environment means adding packages on top of what it contains, which works provided there are no dependency conflicts. You can install libraries using the `conda install` command to install additional packages in your current environment (run `conda --help` to learn more about how to use `conda` commands). All `conda` install commands should use `-c conda-forge` otherwise it's unlikely to work, since many/most of the packages installed already are from conda-forge. For example :\n", |
53 | 53 | "\n", |
54 | 54 | "```\n", |
55 | | - "mamba install xarray\n", |
| 55 | + "conda install -c conda-forge xarray\n", |
| 56 | + "```\n", |
| 57 | + "\n", |
| 58 | + "libmamba is the default solver, but users are welcome to set the solver to \"classic\" with: \n", |
| 59 | + "\n", |
| 60 | + "```\n", |
| 61 | + "conda install --solver=classic -c conda-forge xarray\n", |
56 | 62 | "```\n", |
57 | 63 | "\n", |
58 | 64 | "However, it is recommended to use configuration files for reproducibility and shareability. With this approach, assuming your configuration file is named `config.yml`, the command to use is : \n", |
59 | 65 | "\n", |
60 | 66 | "```\n", |
61 | | - "mamba env update -f config.yml\n", |
| 67 | + "conda env update -f config.yml\n", |
62 | 68 | "```\n", |
63 | 69 | "\n", |
64 | 70 | "For more details on configuration files, see the [Custom environments section](#Custom-environments) and for an example of this command, refer to the [subsection about updating an environment with a configuration file](#Updating-an-existing-environment-with-a-configuration-file).\n" |
|
73 | 79 | "\n", |
74 | 80 | "*For the rest of this README, in each section we provide a link to download an example YAML configuration file.*\n", |
75 | 81 | "\n", |
76 | | - "You can use the `mamba` CLI to create a new, custom environment. The parameters (the list of libraries, the location where to search for them, etc...) can be passed either from a configuration YAML file or directly on the console. We recommend using the first option (a YAML file is easier to share and modify). \n", |
| 82 | + "You can use the `conda` CLI to create a new, custom environment. The parameters (the list of libraries, the location where to search for them, etc...) can be passed either from a configuration YAML file or directly on the console. We recommend using the first option (a YAML file is easier to share and modify). \n", |
77 | 83 | "\n", |
78 | 84 | "### Basic custom environment\n", |
79 | 85 | "\n", |
|
82 | 88 | "This configuration installs specific versions `python`, `pandas` and `geopandas` from `conda-forge`. If versions aren't specified, the latest is installed. We recommend to always specify the version for reproducibility. The basic command to create this environment would be :\n", |
83 | 89 | "\n", |
84 | 90 | "```\n", |
85 | | - "mamba env create -f env-example.yml\n", |
| 91 | + "conda env create -f env-example.yml\n", |
86 | 92 | "```\n", |
87 | 93 | "\n", |
88 | 94 | "However, this stores this environment files in `/opt/conda`, which is a directory that is recreated when the workspace restarts, and so custom environments are lost. Therefore, you want to specify a storage location in your user directory with the `--prefix` parameter\n", |
89 | 95 | "\n", |
90 | 96 | "```\n", |
91 | | - "mamba env create -f env-example.yml --prefix /projects/env\n", |
| 97 | + "conda env create -f env-example.yml --prefix /projects/env/env-example\n", |
92 | 98 | "```\n", |
93 | 99 | "\n", |
94 | 100 | "and to activate it : \n", |
95 | 101 | "\n", |
96 | 102 | "```\n", |
97 | | - "mamba activate env-example\n", |
| 103 | + "conda activate /projects/env/env-example\n", |
98 | 104 | "```\n", |
99 | 105 | "\n", |
100 | 106 | "### Updating an existing environment with a configuration file\n", |
101 | 107 | "\n", |
102 | 108 | "*Example config file for this section [here](./example_conda_configuration_files/env-extend.yml).*\n", |
103 | 109 | "\n", |
104 | | - "You can *update* an existing environment with a configuration file as well. For example, let's assume you have a `mamba` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. Using the linked example config : \n", |
| 110 | + "You can *update* an existing environment with a configuration file as well. For example, let's assume you have a `conda` environment with a set of packages already installed in it (for example the `base` environment), but it doesn't have `xarray` and `geopandas`. Using the linked example config : \n", |
105 | 111 | "\n", |
106 | 112 | "```\n", |
107 | | - "mamba env update -f env-extend.yml\n", |
| 113 | + "conda env update -f env-extend.yml\n", |
108 | 114 | "```\n", |
109 | 115 | "\n", |
110 | | - "This command will update `base` by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n", |
| 116 | + "This command will update the active environment by adding `xarray` and `geopandas`, provided it does not cause conflicts with the existing libraries. \n", |
111 | 117 | "\n", |
112 | 118 | "\n", |
113 | 119 | "### Using `pip` for python packages\n", |
114 | 120 | "\n", |
115 | 121 | "*Example config file for this section [here](./example_conda_configuration_files/env-with-pip.yml).*\n", |
116 | 122 | "\n", |
117 | | - "Some python packages might not be availabe in the channel you are using, or in any `mamba` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `mamba` environment to download packages. The recommended way is to specify this in the configuration file. In the linked example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n", |
| 123 | + "Some python packages might not be availabe in the channel you are using, or in any `conda` channel. If that package however is in `PyPI` (the official python package repository), one can use `pip` within a `conda` environment to download packages. The recommended way is to specify this in the configuration file. In the linked example, we add `stackstac` as a dependency to install from `PyPI` because it is not available in the `conda-forge` channel. \n", |
118 | 124 | "\n", |
119 | 125 | "### Using custom environments in jupyter notebooks\n", |
120 | 126 | "\n", |
|
126 | 132 | "- Create your environment using the linked configuration file.\n", |
127 | 133 | "- Install the environment as a kernel by running the following command (parameter values follow the example mentioned):\n", |
128 | 134 | " ```\n", |
129 | | - " python -m ipykernel install --user --name env-with-ipykernel --display-name \"Python env-with-ipykernel\".\n", |
| 135 | + " python -m ipykernel install --user --name env-with-ipykernel --display-name \"Python env-with-ipykernel\"\n", |
130 | 136 | " ```\n", |
131 | 137 | " The above command installs the environment as a kernel in Jupyter, making it accessible in the notebook with a display name of \"Python env-with-ipykernel\".\n", |
132 | 138 | "- Wait around 30 seconds and launch a new notebook. Among the kernel options, you should see \"Python env-with-ipykernel\" listed. Below you can see a screenshot that shows what this step looks like:\n", |
133 | | - "\n", |
| 139 | + "\n", |
| 140 | + "- Remove by listing kernelspecs `jupyter kernelspec list` to find name, then `jupyter kernelspec remove <env>`\n", |
134 | 141 | "\n", |
135 | 142 | "### Suggested packages for custom environment\n", |
136 | 143 | "\n", |
137 | 144 | "*Example config file for this section [here](./example_conda_configuration_files/env-with-maap-py.yml)*\n", |
138 | 145 | "\n", |
139 | | - "MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` mamba environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. See the configuration example linked. You can note that in the example, `maap-py` is 'versioned' using a commit hash (at the end of the github URL). " |
| 146 | + "MAAP users typically use the python `maap-py`. It's pre-installed in all workspaces, in the `base` conda environment, but any custom environment should specify it, otherwise it is not going to be accessible from that environment. However, `maap-py` is not packaged in a public package repository, like `PyPI` or `conda-forge`. It is possible to install it directly from its github repository with `pip` though. See the configuration example linked. You can note that in the example, `maap-py` is 'versioned' using a commit hash (at the end of the github URL). " |
140 | 147 | ] |
141 | 148 | } |
142 | 149 | ], |
143 | 150 | "metadata": { |
144 | 151 | "kernelspec": { |
145 | | - "display_name": "Python 3", |
| 152 | + "display_name": "Python 3.11.1 64-bit", |
146 | 153 | "language": "python", |
147 | 154 | "name": "python3" |
148 | 155 | }, |
|
156 | 163 | "name": "python", |
157 | 164 | "nbconvert_exporter": "python", |
158 | 165 | "pygments_lexer": "ipython3", |
159 | | - "version": "3.10.10" |
| 166 | + "version": "3.11.1" |
160 | 167 | }, |
161 | 168 | "metadata": { |
162 | 169 | "interpreter": { |
|
165 | 172 | }, |
166 | 173 | "vscode": { |
167 | 174 | "interpreter": { |
168 | | - "hash": "8a5edab282632443219e051e4ade2d1d5bbc671c781051bf1437897cbdfea0f1" |
| 175 | + "hash": "5c7b89af1651d0b8571dde13640ecdccf7d5a6204171d6ab33e7c296e100e08a" |
169 | 176 | } |
170 | 177 | } |
171 | 178 | }, |
|
0 commit comments