Skip to content

Commit ef2c195

Browse files
shanselmanCopilot
andcommitted
Add formal MXC E2E validation path
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 1c377cb commit ef2c195

4 files changed

Lines changed: 247 additions & 3 deletions

File tree

AGENTS.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,20 @@ Notes:
2929
- Prefer isolated worktrees for PR validation. Use `git-wt` for worktree workflows; `wt.exe` may resolve to WorkTrunk instead of Windows Terminal, so use the full Windows Terminal path when explicitly launching Terminal.
3030
- Do not claim completion without reporting validation results.
3131

32+
## Targeted Validation Paths
33+
34+
Run the required validation above for every code change, then add the targeted path that matches the touched subsystem.
35+
36+
### MXC / `system.run` / Windows node command execution
37+
38+
When changing MXC sandboxing, `system.run`, exec approvals, Windows node command execution, gateway setup/connect E2E behavior, or files under `src\OpenClaw.Shared\Mxc`, run:
39+
40+
```powershell
41+
.\scripts\validate-mxc-e2e.ps1
42+
```
43+
44+
The script sets `OPENCLAW_RUN_E2E` and `OPENCLAW_RUN_MXC_E2E` itself, then runs the real WSL Gateway -> Windows node -> `system.run` MXC E2E proofs. It fails if the MXC proof skips. Use `-AllowSkip` only to document that the current host is not MXC-capable; do not report an `-AllowSkip` run as merge validation for MXC-related work.
45+
3246
## Architecture Context for New Agents
3347

3448
Start with these docs before changing connection, pairing, node, MCP, or tray UX behavior:

docs/TEST_COVERAGE.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,19 @@ The method inventory is a source scan of `[Fact]` and `[Theory]` attributes. Use
5959
- **OpenClaw.WinNode.Cli.Tests** covers the standalone Windows node CLI contract.
6060
- **OpenClawTray.OnboardingV2.Tests** and **OpenClawTray.FunctionalUI.Tests** cover newer UI surfaces outside the main tray test project.
6161

62+
## Formal validation paths
63+
64+
Use the smallest lane that proves the changed subsystem, but always include the
65+
required closeout lane for code changes.
66+
67+
| Lane | Entry point | Required when |
68+
|---|---|---|
69+
| Required closeout | `.\build.ps1`, Shared tests, Tray tests | Every code change and every agent closeout |
70+
| GitHub-hosted PR/main CI | `.github\workflows\ci.yml` | Every pull request and push to `main`; runs normal E2E shards but skips MXC proofs on hosted runners |
71+
| Local E2E | `OPENCLAW_RUN_E2E=1` with `OpenClaw.E2ETests` | Gateway setup/connect, recovery, or pairing changes that need real WSL Gateway coverage |
72+
| Local MXC E2E | `.\scripts\validate-mxc-e2e.ps1` | MXC sandboxing, `system.run`, exec approvals, Windows node command execution, gateway setup/connect changes that affect MXC |
73+
| Product WSL setup validation | `.\scripts\validate-wsl-gateway.ps1` | Tray onboarding/setup-engine changes that must prove the product WSL install path |
74+
6275
## Running tests
6376

6477
```powershell
@@ -76,6 +89,10 @@ dotnet test
7689
$env:OPENCLAW_RUN_E2E = "1"
7790
dotnet test .\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj -r win-x64
7891
92+
# Formal MXC validation path. This sets the required integration/E2E env vars
93+
# itself and fails when MXC proofs skip unless -AllowSkip is explicitly supplied.
94+
.\scripts\validate-mxc-e2e.ps1
95+
7996
# Single project
8097
dotnet test .\tests\OpenClaw.Connection.Tests\OpenClaw.Connection.Tests.csproj
8198

docs/WINDOWS_NODE_TESTING.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -122,17 +122,17 @@ When the node connects, it advertises these capabilities:
122122
### Local sandbox validation
123123
- Sandbox integration tests are intended for local Windows development machines and may skip when the required local sandbox prerequisites are unavailable.
124124
- Build the tray app before running local sandbox validation so the required sandbox helper binaries are present in the app output.
125+
- For MXC-related merge validation, prefer the formal script below because it sets the required gates and fails if MXC is skipped.
125126

126127
```powershell
127-
.\build.ps1
128-
$env:OPENCLAW_RUN_INTEGRATION='1'
129-
dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --filter "FullyQualifiedName~Mxc"
128+
.\scripts\validate-mxc-e2e.ps1
130129
```
131130

132131
### Full Gateway `system.run` MXC runtime proof
133132
- The focused E2E below provisions a fresh WSL Gateway, starts an isolated tray instance, sets a local exec approval rule through MCP, invokes `system.run` through the real Gateway `node.invoke` path, and verifies tray MXC diagnostics show contained `mxc-direct-appc` execution for both allowed execution and denied writes to the tray data directory.
134133
- Run it when validating the Gateway/Windows node runtime path, not just direct MCP or shared library behavior.
135134
- GitHub-hosted Actions runners do not provide a working MXC/AppContainer runtime. The regular cloud E2E matrix should report these MXC proofs as skipped while still running the rest of setup-connect. Run the proof on a local MXC-enabled Windows machine. Only set `OPENCLAW_RUN_MXC_E2E=1` in GitHub Actions when using an MXC-enabled self-hosted runner.
135+
- Use `.\scripts\validate-mxc-e2e.ps1` for normal local validation. It sets `OPENCLAW_RUN_E2E` and `OPENCLAW_RUN_MXC_E2E`, runs the real Gateway MXC proofs, and fails if the MXC proof skips. `-AllowSkip` is only for documenting a non-MXC host, not for merge validation of MXC-related work.
136136
- When reproducing this manually against an existing Gateway, make sure `gateway.nodes.allowCommands` includes `system.run`, `system.run.prepare`, and `system.which`, then approve any `pending-reapproval` request with `openclaw nodes approve <pendingRequestId>`. The node can advertise `system.run` while the Gateway still blocks it until both gates are updated.
137137

138138
```powershell

scripts/validate-mxc-e2e.ps1

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
<#
2+
.SYNOPSIS
3+
Runs the formal local MXC validation path.
4+
5+
.DESCRIPTION
6+
Builds the required Windows projects, enables the E2E/MXC gates, and runs
7+
the real WSL Gateway -> Windows node -> system.run MXC E2E proofs.
8+
9+
This script is intentionally stricter than the regular GitHub-hosted E2E
10+
shard: MXC skips fail by default so MXC-related work cannot accidentally
11+
claim end-to-end validation on a host that did not exercise MXC.
12+
13+
.PARAMETER NoBuild
14+
Skip build steps and run against existing outputs.
15+
16+
.PARAMETER AllowSkip
17+
Return success when MXC tests are reported but skipped. Use only for
18+
discovery/documentation of a non-MXC host; do not use as merge validation
19+
for MXC, system.run, exec approval, Windows node command execution, or
20+
gateway setup/connect changes.
21+
#>
22+
[CmdletBinding()]
23+
param(
24+
[switch]$NoBuild,
25+
26+
[switch]$AllowSkip,
27+
28+
[ValidateSet("Debug", "Release")]
29+
[string]$Configuration = "Debug",
30+
31+
[ValidateSet("win-x64", "win-arm64")]
32+
[string]$RuntimeIdentifier,
33+
34+
[string]$ResultsDirectory
35+
)
36+
37+
Set-StrictMode -Version Latest
38+
$ErrorActionPreference = "Stop"
39+
40+
$repoRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
41+
Set-Location $repoRoot
42+
43+
if ([string]::IsNullOrWhiteSpace($RuntimeIdentifier)) {
44+
$RuntimeIdentifier = switch ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) {
45+
([System.Runtime.InteropServices.Architecture]::Arm64) { "win-arm64"; break }
46+
default { "win-x64" }
47+
}
48+
}
49+
50+
if ([string]::IsNullOrWhiteSpace($ResultsDirectory)) {
51+
$ResultsDirectory = Join-Path $repoRoot "TestResults\MxcE2E"
52+
}
53+
54+
New-Item -ItemType Directory -Force -Path $ResultsDirectory | Out-Null
55+
56+
function Invoke-Checked {
57+
param(
58+
[Parameter(Mandatory = $true)][string]$Name,
59+
[Parameter(Mandatory = $true)][scriptblock]$Command
60+
)
61+
62+
Write-Host ""
63+
Write-Host "=== $Name ===" -ForegroundColor Cyan
64+
& $Command
65+
if ($LASTEXITCODE -ne 0) {
66+
throw "$Name failed with exit code $LASTEXITCODE."
67+
}
68+
}
69+
70+
function Read-Trx {
71+
param([Parameter(Mandatory = $true)][string]$Path)
72+
73+
if (-not (Test-Path -LiteralPath $Path)) {
74+
throw "Expected TRX file was not created: $Path"
75+
}
76+
77+
[xml](Get-Content -LiteralPath $Path -Raw)
78+
}
79+
80+
function Get-TrxUnitTestResults {
81+
param([Parameter(Mandatory = $true)][xml]$Trx)
82+
83+
@($Trx.SelectNodes("//*[local-name()='UnitTestResult']"))
84+
}
85+
86+
function Get-TrxResultText {
87+
param([Parameter(Mandatory = $true)][System.Xml.XmlElement]$Result)
88+
89+
$parts = New-Object System.Collections.Generic.List[string]
90+
foreach ($node in @($Result.SelectNodes(".//*[local-name()='Message' or local-name()='StdOut' or local-name()='StdErr']"))) {
91+
if ($node -and -not [string]::IsNullOrWhiteSpace($node.InnerText)) {
92+
$parts.Add($node.InnerText)
93+
}
94+
}
95+
96+
[string]::Join("`n", $parts)
97+
}
98+
99+
function Assert-GatewayMxcProofsPassed {
100+
param([Parameter(Mandatory = $true)][xml]$Trx)
101+
102+
$expectedProofs = @(
103+
"RealGateway_SystemRun_ExecutesThroughWindowsNodeMxcSandbox",
104+
"RealGateway_SystemRun_BlocksWritesToTrayDataDirectoryInMxcSandbox"
105+
)
106+
$results = Get-TrxUnitTestResults -Trx $Trx
107+
$errors = New-Object System.Collections.Generic.List[string]
108+
109+
foreach ($proof in $expectedProofs) {
110+
$result = @($results | Where-Object { $_.GetAttribute("testName") -like "*$proof*" }) | Select-Object -First 1
111+
if ($null -eq $result) {
112+
$errors.Add("Gateway MXC proof was not reported in TRX: $proof")
113+
continue
114+
}
115+
116+
$outcome = $result.GetAttribute("outcome")
117+
if ($outcome -eq "Passed") {
118+
Write-Host "Gateway MXC proof passed: $proof" -ForegroundColor Green
119+
continue
120+
}
121+
122+
if ($outcome -eq "NotExecuted" -or $outcome -eq "Skipped") {
123+
$text = Get-TrxResultText -Result $result
124+
$skipMessage = if ([string]::IsNullOrWhiteSpace($text)) { "no skip reason in TRX" } else { $text.Trim() }
125+
$message = "Gateway MXC proof skipped: $proof ($skipMessage)"
126+
if ($AllowSkip) {
127+
Write-Warning $message
128+
} else {
129+
$errors.Add("$message. Run on an MXC-enabled Windows machine or pass -AllowSkip only when documenting a blocked host.")
130+
}
131+
continue
132+
}
133+
134+
$errors.Add("Gateway MXC proof '$proof' had unexpected outcome '$outcome'.")
135+
}
136+
137+
if ($errors.Count -gt 0) {
138+
throw [string]::Join("`n", $errors)
139+
}
140+
}
141+
142+
function Set-ProcessEnv {
143+
param(
144+
[Parameter(Mandatory = $true)][string]$Name,
145+
[string]$Value
146+
)
147+
148+
[Environment]::SetEnvironmentVariable($Name, $Value, "Process")
149+
}
150+
151+
$trackedEnvVars = @(
152+
"OPENCLAW_REPO_ROOT",
153+
"OPENCLAW_RUN_E2E",
154+
"OPENCLAW_RUN_MXC_E2E"
155+
)
156+
$previousEnv = @{}
157+
foreach ($name in $trackedEnvVars) {
158+
$previousEnv[$name] = [Environment]::GetEnvironmentVariable($name, "Process")
159+
}
160+
161+
try {
162+
Set-ProcessEnv -Name "OPENCLAW_REPO_ROOT" -Value $repoRoot
163+
Set-ProcessEnv -Name "OPENCLAW_RUN_E2E" -Value "1"
164+
Set-ProcessEnv -Name "OPENCLAW_RUN_MXC_E2E" -Value "1"
165+
166+
Write-Host "OpenClaw MXC validation"
167+
Write-Host " Repo: $repoRoot"
168+
Write-Host " Configuration: $Configuration"
169+
Write-Host " RuntimeIdentifier: $RuntimeIdentifier"
170+
Write-Host " Results: $ResultsDirectory"
171+
if ($AllowSkip) {
172+
Write-Warning "-AllowSkip is enabled. This run may document a non-MXC host, but it is not sufficient merge validation for MXC-related work."
173+
}
174+
175+
if (-not $NoBuild) {
176+
Invoke-Checked -Name "Build repository" -Command {
177+
$powerShellExe = (Get-Process -Id $PID).Path
178+
& $powerShellExe -NoProfile -File (Join-Path $repoRoot "build.ps1") -Configuration $Configuration
179+
}
180+
181+
Invoke-Checked -Name "Build tray app for $RuntimeIdentifier" -Command {
182+
& dotnet build ".\src\OpenClaw.Tray.WinUI\OpenClaw.Tray.WinUI.csproj" -c $Configuration -r $RuntimeIdentifier
183+
}
184+
185+
Invoke-Checked -Name "Build E2E tests for $RuntimeIdentifier" -Command {
186+
& dotnet build ".\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj" -c $Configuration -r $RuntimeIdentifier
187+
}
188+
}
189+
190+
$e2eTrx = Join-Path $ResultsDirectory "OpenClaw.E2ETests.Mxc.trx"
191+
$e2eConsoleLog = Join-Path $ResultsDirectory "OpenClaw.E2ETests.Mxc.console.log"
192+
Invoke-Checked -Name "Run Gateway MXC E2E proofs" -Command {
193+
& dotnet test ".\tests\OpenClaw.E2ETests\OpenClaw.E2ETests.csproj" `
194+
--no-build `
195+
--no-restore `
196+
-c $Configuration `
197+
-r $RuntimeIdentifier `
198+
--verbosity normal `
199+
--results-directory $ResultsDirectory `
200+
--logger "trx;LogFileName=OpenClaw.E2ETests.Mxc.trx" `
201+
--logger "console;verbosity=detailed" `
202+
--filter "FullyQualifiedName~OpenClaw.E2ETests.Setup.MxcSetupAndConnectTests" `
203+
2>&1 | Tee-Object -FilePath $e2eConsoleLog
204+
}
205+
Assert-GatewayMxcProofsPassed -Trx (Read-Trx -Path $e2eTrx)
206+
207+
Write-Host ""
208+
Write-Host "MXC validation completed successfully." -ForegroundColor Green
209+
} finally {
210+
foreach ($name in $trackedEnvVars) {
211+
[Environment]::SetEnvironmentVariable($name, $previousEnv[$name], "Process")
212+
}
213+
}

0 commit comments

Comments
 (0)