diff --git a/.vscode/cspell.json b/.vscode/cspell.json index 4a494888e05a..4abb2c97d208 100644 --- a/.vscode/cspell.json +++ b/.vscode/cspell.json @@ -197,6 +197,8 @@ "avroencoder", "Avs", "azcmagent", + "azpysdk", + "pyenv", "azsdk", "azurecr", "azuremgmtcore", diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cbd2959e88bd..e17a4a5472ee 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,6 +15,8 @@ We utilize a variety of tools to ensure smooth development, testing, and code qu - Virtualenv: [Virtualenv](https://virtualenv.pypa.io/en/latest/) is leveraged by Tox to create isolated environments for each test suite, ensuring consistent dependencies and reducing conflicts. +- UV: [UV](https://docs.astral.sh/uv/) is a fast package manager that can manage Python versions, run and install Python packages, and be used instead of pip, virtualenv, and more. + - Pytest: [Pytest](https://docs.pytest.org/en/stable/) is the test framework we use for writing and running our unit tests. It supports fixtures, parameterized tests, and other features that make testing more powerful and flexible. - Pylint: [Pylint](https://pylint.readthedocs.io/en/stable/) is used for code linting to enforce coding standards and catch potential issues early. Maintaining a consistent code style is important, and Pylint helps achieve that goal. diff --git a/doc/tool_usage_guide.md b/doc/tool_usage_guide.md new file mode 100644 index 000000000000..5ac7ad8a05bb --- /dev/null +++ b/doc/tool_usage_guide.md @@ -0,0 +1,116 @@ +# Tool Usage Guide + +As part of shipping SDKs that can be trusted by customers, the azure-sdk-for-python dev team maintains a suite of `checks` that can be utilized by developers to ensure predictable quality measures. These checks help determine whether a package meets the required quality standards for release or needs further work before it can be shipped. These `checks` are implemented in a single entrypoint within the local development package `eng/tools/azure-sdk-tools`. This package provides: + + - Templated package generation for mgmt packages + - The test-framework that all packages within this monorepo use (including record/playback integration) + - A ton of CI related tasks and boilerplate + - Static analysis code + +A `tool` in this context is merely a single entrypoint provided by the `azpysdk` entrypoint in the `eng/tools/azure-sdk-tools` package. + +## Available Tools + +This repo is currently migrating all checks from a slower `tox`-based framework, to a lightweight implementation that uses `asyncio` to simultaneously run checks. This tools list is the current set that has been migrated from `tox` to the `azpysdk` entrypoint. + +|tool|description|invocation| +|---|---|---| +|`pylint`| Runs `pylint` checks or `next-pylint` checks. (based on presence of `--next` argument) | `azpysdk pylint .` | +|`mypy`| Runs `mypy` checks or `next-mypy` checks. (based on presence of `--next` argument) | `azpysdk mypy --next=True .` | +|`sphinx`| Generates a documentation website for the targeted packages. Runs `sphinx` or `next-sphinx` (based on presence of `--next` argument). | `azpysdk sphinx .` | +|`black`| Runs `black` checks. | `azpysdk black .` | +|`import_all`| Installs the package w/ default dependencies, then attempts to `import *` from the base namespace. Ensures that all imports will resolve after a base install and import. | `azpysdk import_all .` | + +## Common arguments + +### Globbing +The azpysdk is intended to be used from within the azure-sdk-for-python repository. A user can invoke + +``` +/azure-sdk-for-python/> azpysdk import_all azure-storage* # will run import_all for all packages starting with `azure-storage` + +/azure-sdk-for-python/> azpysdk import_all azure-storage-blob,azure-core # invoke import_all for two packages + +/azure-sdk-for-python/sdk/core/azure-core/> azpysdk import_all . # invoke import_all for the local package only +``` + +### Automatically isolating the environment + +The targeted tool should be able to run in an isolated environment for a couple reasons: +- When attempting to diagnose an issue, sometimes a user is best served by completely wiping their `venv` and starting over from scratch. Issues may stem from having additional deps downloaded that normally wouldn't be installed with the package. +- In `CI`, we _have_ to run in isolated virtual environments for some checks to allow processes like `multiple pytest` runs to invoke without accidentally stomping on each other's progress or hitting file contention errors. CI passes this flag by default from `dispatch_checks.py`. + +To utilize this feature, add `--isolate` to any `azpysdk` invocation: + +```bash +/> azpysdk import_all azure-storage-blob --isolate +# will install and run within a venv in `sdk/storage/azure-storage-blob/.venv_import_all/ +``` + +## Prerequisite + +- You need to have Python installed +- The monorepo requires a minimum of `python 3.9`, but `>=3.11` is required for the `sphinx` check due to compatibility constraints with external processes. +- You may optionally use the ["uv"](https://docs.astral.sh/uv/) tool, which is fast and handles Python version and venv creation automatically. + +## Initial setup + +- Go to the folder of the package you're working on, for instance `sdk/contoso/azure-contoso` + +### Setup with uv + +`uv venv` + +```bash +# for WSL +source .venv/bin/activate + +# for Windows +.venv\Scripts\activate +``` + +`uv pip install -r dev_requirements.txt` + +### Setup with pip/python + +`python -m venv .venv` + +```bash +# for WSL +source .venv/bin/activate + +# for Windows +.venv\Scripts\activate +``` + +`pip install -r dev_requirements.txt` + +## Using the CLI + +```bash +# You can call the command directly if you activated the venv +azpysdk +``` + +## Advanced scenario + +### Using different Python version + +You can locally test different Python versions to check compatibility with all the required Python versions. + +Note that this is optional, and you can rely on CI to test python versions. + +#### With uv + +You can specify a Python version upon venv creation: +`uv venv --python 3.11` + +#### With Python + +You need to install [pyenv](https://github.com/pyenv/pyenv?tab=readme-ov-file#installation), which lets you easily switch between multiple versions of Python. + +To switch Python versions: +``` +pyenv install 3.10 +pyenv global 3.10 +``` \ No newline at end of file