Bifurcate Dependency Lists (rapidsai#1073)

## Summary

This PR uses [rapids-dependency-file-generator](https://github.com/rapidsai/dependency-file-generator) along with a new `dependencies.yaml` file to bifurcate the existing dependency lists in the `conda/environments` directory. `rapids-dependency-file-generator` will generate `conda` environment files based on the contents of the new `dependencies.yaml` file (see [rapidsai/dependency-file-generator](https://github.com/rapidsai/dependency-file-generator) for more details). There is also a new GitHub Action that runs a shared workflow (which lives [here](https://github.com/rapidsai/shared-action-workflows/blob/main/.github/workflows/dependency-files.yaml)) to ensure that the generated dependency files are up-to-date in each PR. This shared workflow can easily be implemented across other RAPIDS repositories.

## Why is this PR Needed?

This PR is necessary to support some upcoming changes in CI. We've recently created some new, slimmer CI images (~450MB vs. the current 6.5GB) in anticipation of the move to GitHub Actions. These new images omit the large dependency packages (i.e. `rapids-{build,notebook,doc}-env`) from the [rapidsai/integration](https://github.com/rapidsai/integration) repository, which will eventually be deprecated and deleted entirely. One of the consequences of removing the `integration` packages from our CI images is that commonly used test dependencies (i.e. `pytest`) are no longer included in the CI images by default. Therefore, we need to ensure that each repository has a list of the dependencies that are required to test its respective libraries. A similar list will also be needed in other repositories for JupyterLab Notebook dependencies (which will be utilized in our `runtime` end-user images), but that list does not apply to `rmm`.

## Impacts of these Changes

The changes in this PR impact developers and CI as follows:

### Developers

Developers should now update the dependencies in the `dependencies.yaml` file instead of the generated dependency files. If they accidentally update the generated dependency files, the new GitHub Action will fail. At this point, the developer will need to install `rapids-dependency-file-generator` and run it from the project's root directory. For example:

```sh
pip install rapids-dependency-file-generator
rapids-dependency-file-generator
```

This step can also be implemented as a pre-commit hook.

### CI

The new GitHub Action CI process will use the dependencies in the `test` list in `dependencies.yaml` to create a new test environment for `rmm`/`librmm` in CI. For example:

```sh
ENV_NAME="rmm_test"

rapids-dependency-file-generator \
  --file_key "test" \
  --generate "conda" \
  --matrix "cuda=11.5;arch=$(arch)" > env.yaml
mamba env create --file env.yaml
mamba activate "$ENV_NAME"

# ... run `rmm`/`librmm` tests
```

## Additional Information

The new GitHub Action will enforce that an `all` environment (under the `files.all` key) is included in the `dependencies.yaml` file and that it includes all of the dependency lists in the `dependencies.conda` and `dependencies.conda_and_requirements` sections. This is enforced in the shared workflow [here](https://github.com/rapidsai/shared-action-workflows/blob/d5cf5f802a0da73aed5ed01ee18daf5dd3a0f095/.github/workflows/dependency-files.yaml#L42-L80). It exists to provide some standardization across the RAPIDS libraries and will eventually replace the `rapids-{build,notebook,doc}-env` packages in our `devel` end-user images.

Authors:
   - AJ Schmidt (https://github.com/ajschmidt8)

Approvers:
   - Bradley Dice (https://github.com/bdice)
   - Ray Douglass (https://github.com/raydouglass)

Author: ajschmidt8

.github/workflows/dependency-files.yml

@@ -0,0 +1,8 @@
+name: dependency-files
+
+on:
+  pull_request:
+
+jobs:
+  check-generated-files:
+    uses: rapidsai/shared-action-workflows/.github/workflows/dependency-files.yaml@main

README.md

@@ -91,7 +91,7 @@ $ cd rmm
 - Create the conda development environment `rmm_dev`
 ```bash
 # create the conda environment (assuming in base `rmm` directory)
-$ conda env create --name rmm_dev --file conda/environments/rmm_dev_cuda11.0.yml
+$ conda env create --name rmm_dev --file conda/environments/all_cuda-115_arch-x86_64.yaml
 # activate the environment
 $ conda activate rmm_dev
 ```

conda/environments/rmm_dev_cuda11.5.yml …vironments/all_cuda-115_arch-x86_64.yamlconda/environments/rmm_dev_cuda11.5.yml renamed to conda/environments/all_cuda-115_arch-x86_64.yaml conda/environments/rmm_dev_cuda11.5.yml renamed to conda/environments/all_cuda-115_arch-x86_64.yaml

@@ -1,24 +1,25 @@
-name: rmm_dev
+# This file is generated by `rapids-dependency-file-generator`.
+# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
 channels:
 - rapidsai
 - conda-forge
 dependencies:
-- clang=11.1.0
+- black=22.3.0
 - clang-tools=11.1.0
+- clang=11.1.0
 - cmake>=3.20.1,!=3.23.0
 - cmakelang=0.6.13
+- cuda-python>=11.5,<11.7.1
+- cudatoolkit=11.5
+- cython>=0.29,<0.30
 - flake8=3.8.3
-- black=22.3.0
+- gcovr>=5.0
 - isort=5.10.1
-- python>=3.8,<3.10
 - numba>=0.49
 - numpy>=1.19
-- cffi>=1.10.0
 - pytest
-- cudatoolkit=11.5
+- pytest-cov
+- python>=3.8,<3.10
 - scikit-build>=0.13.1
 - spdlog>=1.8.5,<1.9
-- cython>=0.29,<0.30
-- gcovr>=5.0
-- cuda-python>=11.5,<11.7.1
-- sphinx_rtd_theme
+name: all_cuda-115_arch-x86_64

conda/environments/rmm_dev_cuda11.6.yml …vironments/all_cuda-116_arch-x86_64.yamlconda/environments/rmm_dev_cuda11.6.yml renamed to conda/environments/all_cuda-116_arch-x86_64.yaml conda/environments/rmm_dev_cuda11.6.yml renamed to conda/environments/all_cuda-116_arch-x86_64.yaml

@@ -1,24 +1,25 @@
-name: rmm_dev
+# This file is generated by `rapids-dependency-file-generator`.
+# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
 channels:
 - rapidsai
 - conda-forge
 dependencies:
-- clang=11.1.0
+- black=22.3.0
 - clang-tools=11.1.0
+- clang=11.1.0
 - cmake>=3.20.1,!=3.23.0
 - cmakelang=0.6.13
+- cuda-python>=11.6,<11.7.1
+- cudatoolkit=11.6
+- cython>=0.29,<0.30
 - flake8=3.8.3
-- black=22.3.0
+- gcovr>=5.0
 - isort=5.10.1
-- python>=3.8,<3.10
 - numba>=0.49
 - numpy>=1.19
-- cffi>=1.10.0
 - pytest
-- cudatoolkit=11.6
+- pytest-cov
+- python>=3.8,<3.10
 - scikit-build>=0.13.1
 - spdlog>=1.8.5,<1.9
-- cython>=0.29,<0.30
-- gcovr>=5.0
-- cuda-python>=11.6,11.7.1
-- sphinx_rtd_theme
+name: all_cuda-116_arch-x86_64

dependencies.yaml

@@ -0,0 +1,64 @@
+# Dependency list for https://github.com/rapidsai/dependency-file-generator
+files:
+  all:
+    generate: conda
+    matrix:
+      cuda: ["11.5", "11.6"]
+      arch: [x86_64]
+    includes:
+      - build
+      - develop
+      - run
+      - test
+  test:
+    generate: none
+    includes:
+      - test
+channels:
+  - rapidsai
+  - conda-forge
+dependencies:
+  conda_and_requirements:
+    common:
+      build:
+        - cmake>=3.20.1,!=3.23.0
+        - cython>=0.29,<0.30
+        - python>=3.8,<3.10
+        - scikit-build>=0.13.1
+      run:
+        - numba>=0.49
+        - numpy>=1.19
+      develop:
+        - black=22.3.0
+        - clang=11.1.0
+        - cmakelang=0.6.13
+        - flake8=3.8.3
+        - gcovr>=5.0
+        - isort=5.10.1
+      test:
+        - pytest
+        - pytest-cov
+    specific:
+      - matrix:
+          cuda: "11.5"
+        build:
+          - cuda-python>=11.5,<11.7.1
+      - matrix:
+          cuda: "11.6"
+        build:
+          - cuda-python>=11.6,<11.7.1
+  conda:
+    common:
+      build:
+        - spdlog>=1.8.5,<1.9
+      develop:
+        - clang-tools=11.1.0
+    specific:
+      - matrix:
+          cuda: "11.5"
+        build:
+          - cudatoolkit=11.5
+      - matrix:
+          cuda: "11.6"
+        build:
+          - cudatoolkit=11.6