From cf1ef6e10af4b85d49d05828551d6829a04d468b Mon Sep 17 00:00:00 2001 From: Robby Cochran Date: Thu, 4 Jun 2026 21:30:38 -0700 Subject: [PATCH 1/3] X-Smart-Branch-Parent: main From 53369052ae7fdcc2fe70722693dd98482ac152e4 Mon Sep 17 00:00:00 2001 From: Robby Cochran Date: Thu, 4 Jun 2026 21:40:38 -0700 Subject: [PATCH 2/3] =?UTF-8?q?docs:=20SPEC.md=20accuracy=20pass=20?= =?UTF-8?q?=E2=80=94=20retry=20delays,=20policy=20hosts,=20test=20matrix?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Sandbox lifecycle: document 5s retry delay (local) vs 10s (OCP launcher) - Network policy: add platform.claude.com and sentry.io to claude_telemetry - Testing: document --reuse-gateway flag, make validate, test matrix structure Closes #18. Validated: unit 38+7, bats 29+29, podman 19+19, OCP 17/17. Co-Authored-By: Claude Opus 4.6 (1M context) --- SPEC.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/SPEC.md b/SPEC.md index 824333e..e0737ba 100644 --- a/SPEC.md +++ b/SPEC.md @@ -163,7 +163,7 @@ JIRA_USERNAME = "user@example.com" - `--upload /tmp/openshell:/sandbox/.config` — files land at `/sandbox/.config/openshell/` - `-- bash -c '. /sandbox/startup.sh && exec '` (tty mode) - `-- bash /sandbox/startup.sh` (no-tty mode) -4. On failure (supervisor race), delete sandbox and retry (up to 5 times, 5s between) +4. On failure (supervisor race), delete sandbox and retry (up to 5 times, 5s between for local, 10s for OCP launcher) ### Startup (inside sandbox) @@ -213,7 +213,7 @@ The image is multi-arch (`linux/amd64` + `linux/arm64`), built with `docker buil | Policy | Hosts | Binaries | |--------|-------|----------| -| `claude_telemetry` | `*.anthropic.com`, `downloads.claude.ai` | claude, node | +| `claude_telemetry` | `*.anthropic.com`, `downloads.claude.ai`, `platform.claude.com`, `sentry.io` | claude, node | | `github_git` | `github.com` (GET info/refs, POST git-upload-pack only) | git | | `github_downloads` | `*.githubusercontent.com`, `codeload.github.com` | curl, gh, git, uv | | `google_workspace` | `*.googleapis.com`, `oauth2.googleapis.com` | gws | @@ -279,9 +279,20 @@ The launcher connects to the gateway at `https://openshell.openshell.svc.cluster ### Integration Tests (`test/test-flow.sh`) -End-to-end validation requiring a live gateway. Supports `--go` flag to test the Go binary instead of bash scripts: +End-to-end validation requiring a live gateway: - Quick mode: deploy → providers → gateway check → teardown - Full mode: + sandbox create → verify env/GWS/MCP/Claude → delete → teardown - Error scenarios: bad profile, teardown idempotency, missing providers - Targets: `podman`, `ocp`, `all` -- Test matrix: `{bash, go}` × `{podman, ocp}` via `make validate` + +Flags: +- `--go` — run the Go binary instead of bash scripts +- `--full` — include sandbox lifecycle tests +- `--reuse-gateway` — skip helm deploy/teardown-k8s, reuse existing gateway (49s vs 137s for OCP) + +### Test Matrix (`make validate`) + +Full validation across all paths. Run before every commit: +1. Go unit tests (harness + launcher) +2. Bats preflight (Python + Go paths) +3. Integration: `{bash, go}` × `{podman, ocp}` From cda5509dca4fda4d679a1ff8b63f86d6a4e05c7e Mon Sep 17 00:00:00 2001 From: Robby Cochran Date: Thu, 4 Jun 2026 21:43:24 -0700 Subject: [PATCH 3/3] docs: add TODO.md tracking migration status and roadmap Migration scorecard (8/12 native Go), remaining bash wrappers with effort estimates, architecture improvements (gRPC, Python removal, launcher consolidation), low-priority cleanup from audit, and testing gaps to fill. Co-Authored-By: Claude Opus 4.6 (1M context) --- TODO.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 TODO.md diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..2073e09 --- /dev/null +++ b/TODO.md @@ -0,0 +1,88 @@ +# TODO — Go Migration & Roadmap + +## Migration Status + +| Command | Go Status | Bash stays? | Notes | +|---------|-----------|-------------|-------| +| `new --local` | Native | Reference only | Profile parsing, provider validation, sandbox create with retry | +| `new --remote` | Bash wrapper | Yes | K8s Job YAML generation, kubectl apply | +| `connect` | Native | Reference only | exec into `openshell sandbox connect` | +| `deploy --local` | Native | Reference only | Podman check, gateway find/select/verify | +| `deploy --remote` | Bash wrapper | Yes | Helm install, route, mTLS, RBAC, SCCs | +| `teardown --sandboxes` | Native | Reference only | SandboxList + SandboxDelete via Gateway | +| `teardown --providers` | Native | Reference only | ProviderList + ProviderDelete + InferenceRemove | +| `teardown --k8s` | Bash wrapper | Yes | Helm uninstall, CRDs, SCCs, secrets, namespace | +| `preflight` | Native | Reference only | All 29 bats tests pass against Go | +| `providers` | Native | Reference only | Eliminates jq dependency | +| `test` | Bash wrapper | Yes | test-flow.sh orchestration | +| **Launcher** | Native | Reference only | In-cluster Go binary, UBI9 + openssh | + +**Score: 8/12 paths native Go.** Remaining 4 are kubectl/helm-heavy operations. + +## Remaining Bash Wrappers + +### `new --remote` — Medium lift +- Generates K8s Job YAML inline (ConfigMap, volumes, env vars) +- Applies via `kubectl apply`, watches Job, tails logs +- Could use Go `text/template` for YAML + `k8s.io/client-go` or `os/exec kubectl` +- Prerequisite: decide whether to depend on client-go or keep shelling out to kubectl + +### `deploy --remote` — Largest lift +- ~160 lines: namespace, CRDs, SCCs (oc adm), Helm install, route, mTLS cert copy +- Uses: kubectl, helm, oc (OpenShift-specific) +- Consider: keep as bash or accept client-go + helm SDK dependency +- Low priority — rarely changes, works reliably + +### `teardown --k8s` — Medium lift +- Helm uninstall, delete CRDs/SCCs/secrets/namespace, gateway config cleanup +- Mirror of deploy --remote in reverse +- Port together with deploy --remote or not at all + +### `test` — Low priority +- test-flow.sh is a bash test harness, not a harness subcommand +- Porting to Go would mean rewriting the test framework +- No value — bash is the right tool for test orchestration + +## Architecture Improvements + +### Direct gRPC (future) +- OpenShell gateway exposes 54 gRPC RPCs (proto files in NVIDIA/OpenShell repo) +- Generate Go stubs from proto files → `gateway.GRPC` implementation +- Swap `gateway.NewCLI(cli)` → `gateway.NewGRPC(conn)` — one line change +- Eliminates: openshell CLI binary dependency, output parsing fragility +- Prerequisite: proto files stabilize (OpenShell is alpha) + +### Remove Python dependency +- `providers.py` and `parse-profile.py` still in repo, called by bash path (`bin/harness`) +- Go implementations exist: `internal/preflight/` and `internal/profile/` +- Remove when: bash path is no longer needed for dual-testing +- Blocked by: decision to stop maintaining bash path + +### Launcher consolidation +- `sandbox/launcher/` is a separate Go module (runs in-cluster) +- Has its own `parseConfig` duplicating `internal/profile/` +- Can't share code (different execution context, separate binary) +- Consider: extract shared types to a third module, or accept duplication + +## Low-Priority Cleanup (from audit) + +- [ ] `copyFile` in launcher: check Close error (`defer out.Close()` discards it) +- [ ] Launcher `configureGateway`: route stderr to `os.Stderr` not `os.Stdout` (line 61) +- [ ] Launcher `run()` helper: log errors instead of silently discarding +- [ ] Unexport internal-only functions in `internal/preflight/` and `internal/profile/` +- [ ] Stale comment in `providers.sh` line 18: says "default: us-east5", actual default is "global" +- [ ] `SandboxExec`/`SandboxUpload` on Gateway interface — no callers yet (premature abstraction, but harmless) + +## Testing + +### Current coverage +- 38 Go unit tests (gateway, profile, cmd) +- 7 launcher tests +- 29 bats preflight tests (Python + Go paths) +- Integration: `{bash, go}` × `{podman, ocp}` via `make validate` +- `--reuse-gateway` for fast OCP cycles (49s vs 137s) + +### Gaps to fill +- [ ] Integration test for `providers --force` (currently no test exercises force mode) +- [ ] Go + OCP integration (`test-flow.sh ocp --full --go`) — not yet validated +- [ ] Preflight Go unit tests (internal/preflight/ has no _test.go)