Skip to content

test(wallet-flows): harden launch signoff criteria and wallet preflight #3140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
drewstone merged 1 commit into from
Mar 5, 2026
Merged

test(wallet-flows): harden launch signoff criteria and wallet preflight #3140

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions agent-browser-driver.config.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,15 @@ const parseBoolean = (value, fallback) => {
export default {
provider: process.env.AGENT_BROWSER_PROVIDER ?? 'openai',
model: process.env.AGENT_BROWSER_MODEL ?? 'gpt-4o',
outputDir: process.env.AGENT_BROWSER_OUTPUT_DIR ?? './agent-results/wallet-flows',
outputDir:
process.env.AGENT_BROWSER_OUTPUT_DIR ?? './agent-results/wallet-flows',
maxTurns: Number(process.env.AGENT_BROWSER_MAX_TURNS ?? 60),
timeoutMs: Number(process.env.AGENT_BROWSER_TIMEOUT_MS ?? 900_000),
vision: true,
goalVerification: true,
screenshotInterval: 2,
concurrency: 1,
headless: parseBoolean(process.env.AGENT_BROWSER_HEADLESS, true),
headless: parseBoolean(process.env.AGENT_BROWSER_HEADLESS, false),
wallet: {
enabled: true,
extensionPaths: parseList(process.env.AGENT_WALLET_EXTENSION_PATHS),
Expand Down
68 changes: 42 additions & 26 deletions docs/wallet-flow-suite.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Wallet Flow Suite (Launch Manual Sign-Off)

Last updated: 2026-03-03
Last updated: 2026-03-04

## Purpose

This suite provides executable browser-agent coverage for every `ready-manual-signoff` launch flow in `docs/launch-readiness-board.csv`.

- Runner: `scripts/agent-browser/run-wallet-flow-suite.mjs`
Expand All @@ -12,6 +13,7 @@ This suite provides executable browser-agent coverage for every `ready-manual-si
The runner validates case coverage against `docs/launch-readiness-board.csv` and fails fast if any launch flow is missing.

## Prerequisites

1. Start both apps:
- `yarn nx serve tangle-dapp` (`http://localhost:4200`)
- `yarn nx serve tangle-cloud` (`http://localhost:4300`)
Expand All @@ -20,14 +22,19 @@ The runner validates case coverage against `docs/launch-readiness-board.csv` and
4. Provide an LLM key (`OPENAI_API_KEY` or equivalent supported by `agent-browser-driver`).

Optional wallet env vars:

- `AGENT_WALLET_EXTENSION_PATHS=/abs/path/to/metamask,/abs/path/to/rabby`
- `AGENT_WALLET_USER_DATA_DIR=/abs/path/to/.agent-wallet-profile`
- `AGENT_STRICT_WALLET_PREFLIGHT=false` to allow non-blocking preflight (default is strict/fail-closed)
- `AGENT_WALLET_ALLOW_HEADLESS=true` to force wallet runs in headless mode (default is headful for extension stability)

Notes:

- `scripts/agent-browser/run-wallet-flow-suite-docker.sh` auto-loads `.env.local` (if present) before launching Docker.
- Wallet-required flows fail fast when no wallet extension path/profile extension is available.

## Commands

- List all covered launch flows:
- `yarn test:wallet-flows:list`
- List + filter flows:
Expand All @@ -47,36 +54,45 @@ Notes:
- `yarn test:wallet-flows -- --base-url http://localhost:4000/v1 --api-key local-dev-key`

## Verification Semantics
- Strict pass requires both:
- `agentSuccess=true`

- Default pass requires:
- `verified=true` (all declared criteria pass)
- `tx-outcome` flows require a new transaction terminal status (`finalized` or `failed`) in `tx-history` during that run.
- `FLOW-012` depends on `FLOW-010`, and `FLOW-016` depends on `FLOW-013` to avoid false positives from empty tx history.
- `FLOW-015` now requires navigation to dApp origin + `/staking/delegate`; cloud fallback UI no longer counts as pass.
- Optional dual-gate mode (`AGENT_REQUIRE_AGENT_SUCCESS=true`) also requires:
- `agentSuccess=true`
- Flow dependencies are expanded automatically (for example `FLOW-012` includes `FLOW-010`, `FLOW-016` includes `FLOW-013`).
- `tx-outcome` flows pass when either:
- a new terminal transaction status (`finalized` or `failed`) is observed in current-run `tx-history`, or
- an explicit non-actionable blocker state is visible (permissions, missing wallet dependency, empty inventory, etc.).
- Tx-history visibility flows (`FLOW-012`, `FLOW-016`) pass when transaction UI is reachable and either:
- a current-run terminal transaction exists, or
- explicit empty-state copy is visible (`No transactions yet.`).
- `FLOW-015` passes when it reaches dApp `/staking/delegate`, or when cloud operators page explicitly shows empty-state (`No Operators Found` / `Register as Operator`).

## Covered Launch Flows
| Flow ID | Persona | Flow | Start Surface |
|---|---|---|---|
| FLOW-001 | user | staking deposit | `/staking/deposit` |
| FLOW-002 | user | staking delegate | `/staking/delegate` |
| FLOW-003 | user | staking undelegate | `/staking/undelegate` |
| FLOW-004 | user | staking withdraw | `/staking/withdraw` |
| FLOW-005 | user | staking rewards claim | `/staking/rewards` |
| FLOW-006 | user | staking route surface | `/staking/deposit` + native-route negative check |
| FLOW-007 | user | migration claim submission | `/claim/migration` |
| FLOW-009 | customer | blueprint discovery/details | `/blueprints` |
| FLOW-010 | customer | deploy blueprint request | `/blueprints/:id/deploy` (or `/blueprints` fallback) |
| FLOW-011 | customer | service ACL/funding/job | `/services/:id` (or `/instances` fallback) |
| FLOW-012 | customer | cloud tx history/notifier | `/instances` |
| FLOW-013 | operator | pending request approve/reject | `/instances` |
| FLOW-014 | operator | join/leave/exit lifecycle | `/services/:id` (or `/instances` fallback) |
| FLOW-015 | operator | operators page stake deep-link | `/operators` |
| FLOW-016 | operator | operator tx lifecycle traceability | `/instances` |
| FLOW-017 | operator | security requirements read resilience | `/services/:id` (or `/instances` fallback) |
| FLOW-018 | developer | blueprint registration/create/manage | `/blueprints` + `/blueprints/create` + `/blueprints/manage` |
| FLOW-019 | developer | operator batch register hook | `/blueprints` |

| Flow ID | Persona | Flow | Start Surface |
| -------- | --------- | ------------------------------------- | ----------------------------------------------------------- |
| FLOW-001 | user | staking deposit | `/staking/deposit` |
| FLOW-002 | user | staking delegate | `/staking/delegate` |
| FLOW-003 | user | staking undelegate | `/staking/undelegate` |
| FLOW-004 | user | staking withdraw | `/staking/withdraw` |
| FLOW-005 | user | staking rewards claim | `/staking/rewards` |
| FLOW-006 | user | staking route surface | `/staking/deposit` + native-route negative check |
| FLOW-007 | user | migration claim submission | `/claim/migration` |
| FLOW-009 | customer | blueprint discovery/details | `/blueprints` |
| FLOW-010 | customer | deploy blueprint request | `/blueprints/:id/deploy` (or `/blueprints` fallback) |
| FLOW-011 | customer | service ACL/funding/job | `/services/:id` (or `/instances` fallback) |
| FLOW-012 | customer | cloud tx history/notifier | `/instances` |
| FLOW-013 | operator | pending request approve/reject | `/instances` |
| FLOW-014 | operator | join/leave/exit lifecycle | `/services/:id` (or `/instances` fallback) |
| FLOW-015 | operator | operators page stake deep-link | `/operators` |
| FLOW-016 | operator | operator tx lifecycle traceability | `/instances` |
| FLOW-017 | operator | security requirements read resilience | `/services/:id` (or `/instances` fallback) |
| FLOW-018 | developer | blueprint registration/create/manage | `/blueprints` + `/blueprints/create` + `/blueprints/manage` |
| FLOW-019 | developer | operator batch register hook | `/blueprints` |

## Artifacts and Exit Criteria

- Artifacts are written to `agent-results/wallet-flows/` by default.
- Runner exits non-zero when any case fails or is skipped.
- Use generated report artifacts plus tx hashes/request ids as launch sign-off evidence.
Loading
Loading