Merge pull request #851 from valory-xyz/docs/upgrade-guide-v2-1-0

docs: add comprehensive 2.1.0 upgrade and history notes

Author: DavidMinarsch

.spelling

@@ -298,6 +298,24 @@ v1
 v0
 ipfsdaemon
 py3.6
+toolchain
+runtime
+failover
+awaitable
+asyncio
+hard-coded
+hardcoded
+flag_value
+sync_type
+pytest
+utility
+behaviour
+2.1.x
+8.3.x
+flashbots
+open-aea-ledger-ethereum-flashbots
+resolver
+v8
 async
 gnosis
 abci
@@ -397,3 +415,4 @@ docs
 EIP-1559
 RPCs
 pyinstaller
+awaitability

HISTORY.md

@@ -1,5 +1,31 @@
 # Release History - open AEA
 
+## 2.1.0 (upcoming)
+
+AEA:
+- Extends Python support from `3.10-3.11` to `3.10-3.14`. #831
+- Adds support for pytest v8 and updates related test and utility modules. #835
+- Makes CLI `flag_value` defaults robust across Click processing-order changes (including `packages sync` defaults and registry flag defaults). #839 #848
+- Adds `AEA_PASSWORD` environment variable support for CLI password handling. #825
+- Fixes multiple high-impact audit findings across behaviours, multiplexer, manager, dependency installation, and registry handling. #846
+
+Plugins:
+- Adds multiple RPC rotation with automatic failover in the Ethereum ledger plugin stack. #829
+- Fixes `open-aea-ledger-ethereum-flashbots` dependency constraints to be compatible with `open-aea-ledger-ethereum` in the `2.1.0rc6` line. #850
+
+Tooling and dependencies:
+- Bumps `tomte` from `0.4.0` to `0.6.1`. #831
+- Updates Click range to `<8.4.0` and applies compatibility fixes for 8.3.x behaviour rules. #838 #839 #848
+- Updates several pinned dependencies for Python 3.12-3.14 compatibility, including `packaging==26`, `pytest>=8.2,<10`, `protobuf>=5,<6`, `requests>=2.32.5,<3`, `openapi-core==0.22.0`, `openapi-spec-validator>=0.7.0,<0.8.0`, `docker==7.1.0`, `hypothesis==6.151.9`, and `cosmpy>=0.11.0,<0.12`. #831
+- Removes obsolete/deprecated compatibility dependencies and paths (including `gym` and `distutils` usage). #831 #836
+
+Compatibility and regression fixes:
+- Fixes Python 3.13/3.14 regressions in async/multiprocessing paths (including `multiprocessing.Manager` context handling and asyncio event-loop compatibility changes). #843
+- Reworks ready-awaitable internals to avoid Python 3.14 warnings/regressions and follow-up flaky behaviour. #831 #847
+- Fixes local connection cross-thread queue loop handling to avoid race conditions caused by private asyncio queue internals. #847
+- Adds follow-up fixes for CLI and framework regressions found after the initial interpreter/dependency bump, including critical audit issues and command behaviour corrections. #846 #847 #848
+
+
 ## 2.0.8 (2026-01-20)
 
 Packages:

docs/upgrading.md

@@ -9,6 +9,94 @@ Below we describe the additional manual steps required to upgrade between differ
 
 ### Upgrade guide
 
+## `v2.0.8` to `v2.1.0`
+
+- Python support is now `3.10-3.14` (previously `3.10-3.11`).
+- Regenerate your environment and lock files when upgrading, as several toolchain and runtime dependencies were bumped to support newer Python versions.
+
+Main dependency updates to account for:
+
+- `tomte: 0.4.0 -> 0.6.1`
+- `click: >=8.1.0,<8.3.0 -> >=8.1.0,<8.4.0`
+- `pytest: >=7.0.0,<8.0.0 -> >=8.2,<10`
+- `packaging: >=23.1,<24.0 -> ==26`
+- `protobuf: >=4.21.6,<4.25.0 -> >=5,<6`
+- `requests: ==2.28.1 -> ==2.32.5`
+- `openapi-core: 0.15.0 -> 0.22.0`
+- `openapi-spec-validator: >=0.4.0,<0.5.0 -> >=0.7.0,<0.8.0`
+- `docker: 4.2.0 -> 7.1.0`
+- `hypothesis: 6.21.6 -> 6.151.9`
+- `cosmpy: ==0.9.2 -> >=0.11.0,<0.12`
+
+Exact APIs/functions to check:
+
+### 1) Click `flag_value` default handling
+
+If your downstream CLI uses Click options with `flag_value`, audit these exact patterns against Open AEA fixes:
+
+- `aea/cli/utils/click_utils.py::registry_flag`
+- `aea/cli/utils/click_utils.py::remote_registry_flag`
+- `aea/cli/packages.py::sync`
+- `aea/cli/upgrade.py::upgrade`
+- `scripts/update_package_versions.py` options `--update-minor` / `--update-patch`
+
+Required behaviour:
+
+- Do **not** rely on decorator order to select defaults when multiple flags write to the same parameter.
+- Use explicit normalization in code for unset values (e.g. `if sync_type is None: sync_type = SyncTypes.THIRD_PARTY`).
+- Ensure upgrade command handlers consume a single `registry: str` mode rather than split boolean flags (`local` / `remote`).
+
+### 2) Python 3.13/3.14 asyncio/multiprocessing compatibility
+
+Audit downstream code for these exact anti-patterns and replacements:
+
+- `multiprocessing.Manager()` without explicit context on Python 3.14+.
+  - Required fix pattern: use `multiprocessing.get_context("spawn").Manager()` where process-manager context must be controlled.
+- Accessing private queue loop internals (e.g. `queue._loop`).
+  - Required fix pattern: store/use explicit event loop references instead of private attributes.
+- Removed/unsupported asyncio APIs in modern Python:
+  - `asyncio.StreamReader(loop=...)`
+  - `asyncio.ensure_future(..., loop=...)`
+  - unconditional `asyncio.get_child_watcher()` / child-watcher usage on Python 3.14+
+- Ready-awaitable internals relying on module-level coroutine/future objects.
+  - Required fix pattern: use a lightweight awaitable object/factory that is loop-safe across Python 3.10-3.14.
+
+### 3) Plugin dependency compatibility (Ethereum + Flashbots)
+
+If you install both plugins, verify dependency constraints in your lock/constraints files:
+
+- The Flashbots plugin package must be compatible with the same 2.1.x line of the Ethereum plugin package.
+- Reject locks where Flashbots still constrains Ethereum to `<2.1.0`.
+
+Concrete verification commands to run after upgrade:
+
+1. Upgrade and reinstall:
+    - `pip install --upgrade "open-aea==2.1.0"`
+    - or for pre-release validation: `pip install --upgrade "open-aea==2.1.0rc6"`
+2. Re-lock dependencies:
+    - `pipenv lock` (or your equivalent lock workflow)
+3. Verify CLI default routing behaviour:
+    - `aea packages sync` (must default to third-party sync mode)
+    - `aea packages sync --third-party`
+    - `aea packages sync --dev`
+    - `aea packages sync --all`
+4. Verify interpreter matrix:
+    - run your tests on Python `3.10` and `3.14`
+5. Verify plugin resolution when Flashbots is used:
+    - install both `open-aea-ledger-ethereum` and `open-aea-ledger-ethereum-flashbots` in a clean environment and ensure dependency resolution succeeds.
+
+Known caveat:
+
+- Historical installer scripts may still contain hard-coded checks/messages for Python `3.10/3.11`. Treat package metadata and release notes as the source of truth for supported runtime versions.
+
+### API compatibility notes
+
+- CLI/API-surface changes:
+  - `aea/cli/utils/click_utils.py::password_option` now always prompts on `-p` and supports `AEA_PASSWORD` for `--password`.
+
+- Side-effect risk to audit in downstream code:
+  - `aea/helpers/async_utils.py::Runnable.wait_completed` internals changed to use a lightweight ready-awaitable for loop safety on Python 3.14; callers that relied on strict `Future` type checks should switch to awaitability checks instead.
+
 ## `v2.0.7` to `v2.0.8`
 
 - No backwards incompatible changes