[Bug]: E2E tests leak namespaces and VMs on cluster

[mrhillsman]

Bug Description

E2E test runs leave behind virtwork-test-* namespaces with running VMs, Services, and Secrets on the cluster. The AfterEach cleanup in test files either silently fails or is skipped entirely. Repeated test runs accumulate resources indefinitely with no suite-level safety net. Observed 90 leaked namespaces with 81 running VMs on the test cluster.

Steps to Reproduce

1. Run ginkgo -r -tags e2e ./tests/e2e/
2. Interrupt the run partway through, or let it complete with any test failures
3. Run oc get namespaces | grep virtwork-test — orphaned namespaces remain
4. Run oc get vm -A | grep virtwork-test — VMs are still running

Expected Behavior

After a test run completes (whether passing, failing, or interrupted), all virtwork-test-* namespaces and their resources should be cleaned up. At minimum, a suite-level safety net should sweep any leftovers.

Actual Behavior

Three issues cause resource leaks:

1. Missing --yes flag in run_test.go AfterEach (line 24): The cleanup command is called without --yes, so the safety guardrail prompt causes it to no-op in non-interactive mode. All other test files (cleanup_test.go, fullcycle_test.go, tps_test.go) correctly pass --yes.

2. Silent error swallowing in all AfterEach blocks: Every test file discards all return values from the cleanup call (_, _, _, _ = testutil.RunVirtwork(...)). If cleanup fails (timeout, API error, binary crash), no error is logged and the namespace is permanently leaked.

3. No AfterSuite safety net: If a test panics, times out, or is interrupted before its AfterEach runs, there is no suite-level sweep of stale virtwork-test-* namespaces.

Environment

Field	Value
Command	virtwork cleanup
Workload Types	cpu, memory, network, tps (all types affected)
Go Version	go1.26.1 linux/amd64
OpenShift Version	4.21.6
Deployment Mode	CLI from local machine (kubeconfig)

Additional Context

Cluster snapshot at time of discovery:

• 90 virtwork-test-* namespaces (32 e2e-run, 18 cleanup-test, 18 e2e-clean, 11 e2e-cycle, 11 e2e-tps)
• 81 running VMs, 17 Services, ~81 cloud-init Secrets
• 35 namespaces had VMs already cleaned but namespace itself not deleted (missing --delete-namespace or cleanup partially succeeded)

Proposed fix:

• Add --yes to run_test.go AfterEach
• Log cleanup failures in AfterEach via GinkgoWriter instead of discarding
• Add AfterSuite to e2e_suite_test.go that sweeps remaining virtwork-test-* namespaces using direct K8s client calls (not the binary)