diff --git a/README.md b/README.md index 0cdb7d5..51a88c7 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,56 @@ # OpenShell Harness -A thin orchestration wrapper around [OpenShell](https://github.com/NVIDIA/OpenShell) for deploying and managing AI agent sandboxes. The harness handles the multi-step setup that OpenShell leaves to the user — gateway deployment, provider registration, credential validation, and sandbox configuration — so that one command gets you from zero to a running sandbox. +An orchestration layer for [OpenShell](https://github.com/NVIDIA/OpenShell) that manages gateway deployment, provider registration, credential validation, and sandbox configuration. One command gets you from zero to a running AI agent sandbox on local Podman or remote OpenShift. ## Relationship to OpenShell -The harness wraps `openshell`, it doesn't replace it. Every operation delegates to the OpenShell CLI. Users can drop to raw `openshell` commands at any time. +The harness wraps `openshell` — it doesn't replace it. Every operation delegates to the OpenShell CLI via `exec.Command`. Users can drop to raw `openshell` commands at any time. The harness exists to bridge gaps in OpenShell's current workflow: -- **Gateway deployment** — OpenShell provides the gateway binary and Helm chart but not the orchestration to deploy it (namespace setup, CRDs, SCCs on OpenShift, mTLS cert extraction, Helm install with correct values). The harness automates this for both local (Podman) and remote (OpenShift) targets. +- **Gateway deployment** — OpenShell provides the gateway binary and Helm chart but leaves orchestration to the user (namespace setup, CRDs, SCCs on OpenShift, mTLS cert extraction, Helm values). The harness automates this via config-driven gateway definitions (`gateways/local/`, `gateways/ocp/`, `gateways/kind/`). -- **Provider lifecycle** — OpenShell manages credentials once registered, but doesn't validate prerequisites, discover credentials from local tooling, or compose provider profiles with sandbox configuration. The harness adds preflight checks, credential discovery, and profile-driven provider selection. +- **Provider lifecycle** — OpenShell manages credentials once registered, but doesn't validate prerequisites or discover credentials from local tooling. The harness adds preflight checks (env vars, files, connectivity probes) and profile-driven provider selection. -- **Parity across targets** — a sandbox created locally via Podman should behave identically to one on OpenShift. The harness provides this parity by using the same profiles, provider catalog, and validation on both targets. +- **Credential validation** — preflight checks verify credentials are present and valid on the host before registration. In-sandbox verification (confirming providers work end-to-end from inside a running sandbox) is planned but not yet implemented. + +- **Parity across targets** — a sandbox created locally via Podman should behave identically to one on OpenShift. The harness enforces this by using the same profiles, provider catalog, and validation on both. As OpenShell matures, the harness should shrink. Every workaround tracks the upstream issue that would eliminate it (see [AGENTS.md](AGENTS.md)). +## How It Compares + +| Concern | OpenShell Harness | [Kaiden](https://github.com/openkaiden/kaiden) | [Plandex](https://github.com/plandex-ai/plandex) | +|---------|-------------------|--------|---------| +| **Primary focus** | Gateway deploy + provider orchestration | GUI workspace management, resource selection | Plan-driven coding agent | +| **Sandbox runtime** | OpenShell (delegates entirely) | OpenShell (migrating to it) | None (runs locally) | +| **Entry point** | Container image (image-first) | Local folder or git URL (source-first) | Local directory | +| **Provider management** | Preflight validation + registration | References by name (delegates to OpenShell) | N/A | +| **Target environments** | Local Podman + remote K8s/OCP | Local only (desktop app) | Local only | +| **Credential isolation** | Proxy-resolved placeholders, sandbox never sees tokens | Delegates to OpenShell | None | +| **Configuration** | TOML profiles + provider catalog | JSON projects (GUI-driven) | YAML plans | + +The harness operates at the infrastructure layer — deploying gateways, registering providers, validating credentials. Kaiden operates at the workspace layer — selecting which skills, MCP servers, and knowledge bases a workspace gets. They are complementary, not competing. See [profile.md](profile.md) for a detailed analysis. + ## Goals -1. **One command to working sandbox.** `harness new` takes a developer from zero to a running sandbox — gateway deployed, providers registered, credentials validated. +1. **One command to working sandbox.** `harness new` chains gateway deployment, provider registration, and sandbox creation into a single invocation. 2. **Reproducible environments.** Profiles define exactly what a sandbox needs. The same profile produces the same environment regardless of who runs it or where. -3. **Credential visibility.** Preflight checks validate credentials locally before registration. Provider health checks verify they work end-to-end inside the sandbox. +3. **Credential visibility.** Preflight checks validate credentials locally before registration — env vars set, files present, connectivity confirmed. You know what's broken before you try to create a sandbox. 4. **Clean separation of concerns.** Infrastructure config, provider management, and sandbox profiles are independent. Changing your Jira token doesn't require editing sandbox profiles. Switching clusters doesn't require re-registering providers. 5. **Thin wrapper, not a platform.** Orchestration and validation on top of OpenShell. No reimplementation of sandbox runtime, network policy, or credential injection. -6. **Image-first, no host mounts.** Sandboxes boot from container images with tools baked in. Files are uploaded, not bind-mounted. This is deliberate — host-mounted workflows break parity between local and remote targets, bypass credential isolation, and create implicit dependencies on the host filesystem. If it doesn't work on OpenShift, it shouldn't work locally either. The sandbox interacts with the outside world through providers — git commits, PR reviews, Jira comments, email — not by writing files that get pulled back to the host. +6. **Image-first, no host mounts.** Sandboxes boot from container images with tools baked in. Files are uploaded, not bind-mounted. Host-mounted workflows break parity between local and remote targets, bypass credential isolation, and create implicit dependencies on the host filesystem. If it doesn't work on OpenShift, it shouldn't work locally either. The sandbox interacts with the outside world through providers — git commits, PR reviews, Jira comments, email — not by writing files that get pulled back to the host. ## Three Domains | Domain | Question | Config | Commands | |--------|----------|--------|----------| -| **Infrastructure** | How is the gateway deployed? | `openshell.toml` | `deploy`, `teardown --k8s` | +| **Infrastructure** | How is the gateway deployed? | `gateways//gateway.toml` | `deploy`, `teardown --k8s` | | **Providers** | What credentials are available and valid? | `providers.toml` | `providers`, `preflight` | | **Sandbox** | What sandbox do I want? | `profiles/*.toml` | `new`, `connect` | @@ -43,30 +59,32 @@ Each domain has its own config, its own code boundary, and its own concerns. A s ## Quick Start ```bash +# Install OpenShell CLI +curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh + # Authenticate with Google Cloud (Vertex AI inference) -gcloud auth application-default login +gcloud auth application-default login --project your-gcp-project-id # Set credentials export GITHUB_TOKEN="ghp_..." export JIRA_API_TOKEN="..." +export JIRA_URL="https://mysite.atlassian.net" +export JIRA_USERNAME="you@example.com" -# Local (Podman gateway) +# Build the harness +make cli + +# Local — deploy gateway, register providers, create sandbox harness new --local -# Remote (OpenShift gateway) +# Remote — same flow on OpenShift harness new --remote -# Reconnect +# Reconnect to a running sandbox harness connect ``` -## Prerequisites - -- [OpenShell CLI](https://github.com/NVIDIA/OpenShell) (`openshell`) -- `gcloud auth application-default login` (Vertex AI) -- Podman (local) or kubectl + helm (OpenShift) - -Optional: `gws` CLI (Google Workspace), `bats` (tests) +Podman is required for local sandboxes. kubectl + helm are required for remote. ## Commands @@ -77,8 +95,8 @@ harness new [--local|--remote] [--profile NAME] harness connect [NAME] Reconnect to a running sandbox. -harness deploy [--local|--remote] - Deploy or verify the gateway without creating a sandbox. +harness deploy [GATEWAY_NAME] + Deploy or verify a gateway by name (local, ocp, kind). harness providers [--force] Register providers with the gateway. @@ -97,7 +115,7 @@ Sandboxes are configured via TOML profiles. A profile defines the sandbox shape ```toml # profiles/default.toml name = "agent" -image = "quay.io/rcochran/openshell:sandbox" +from = "ghcr.io/robbycochran/harness-openshell:sandbox" command = "claude --bare" providers = ["github", "vertex-local", "atlassian"] @@ -120,32 +138,35 @@ type = "openshell" description = "Jira and Confluence (Basic auth resolved by proxy)" inputs = [ { key = "JIRA_API_TOKEN", kind = "env", secret = true }, - { key = "JIRA_URL", kind = "env" }, + { key = "JIRA_URL", kind = "env", sandbox = true }, + { key = "JIRA_USERNAME", kind = "env", sandbox = true }, ] ``` -Providers with `type = "openshell"` are registered with the gateway and managed by OpenShell's credential proxy. Providers with `type = "custom"` are workarounds for integrations OpenShell doesn't natively support yet — each tracks its upstream issue. +Providers with `type = "openshell"` are registered with the gateway and managed by OpenShell's credential proxy. Providers with `type = "custom"` are workarounds for integrations OpenShell doesn't natively support yet — each tracks its upstream issue. See [PROVIDERS-SPEC.md](PROVIDERS-SPEC.md) for the full schema. ## Architecture ``` -Your machine OpenShift cluster +Local (Podman) Remote (OpenShift) ┌──────────┐ ┌──────────────────────────────┐ -│ harness │ Route (mTLS) │ Gateway (StatefulSet) │ -│ CLI ├───────────────────►│ ├─ gRPC API │ -│ │ │ ├─ inference.local proxy │ +│ harness │ openshell CLI │ Gateway (StatefulSet) │ +│ CLI ├───────────────────►│ ├─ OpenShell API │ +│ │ localhost:17670 │ ├─ inference.local proxy │ └──────────┘ │ ├─ Provider credential store │ │ └─ OAuth token refresh │ - │ │ + or │ │ │ Sandbox Pods │ - │ ├─ Claude Code → Vertex AI │ - │ ├─ mcp-atlassian │ - │ ├─ gws CLI │ - │ ├─ gh CLI │ +┌──────────┐ Route (mTLS) │ ├─ Claude Code → Vertex AI │ +│ harness ├───────────────────►│ ├─ mcp-atlassian │ +│ CLI │ OCP :443 │ ├─ gws CLI │ +└──────────┘ │ ├─ gh CLI │ │ └─ L7 network proxy │ └──────────────────────────────┘ ``` +The harness talks to the gateway via the `openshell` CLI (exec). Direct gRPC is a planned future improvement — the Gateway interface already abstracts the transport. + Each sandbox gets credential isolation (proxy-resolved placeholders, the sandbox never sees real tokens), per-binary network policy enforcement, and a reproducible toolchain pinned in the container image. ## Project Layout @@ -159,23 +180,23 @@ Each sandbox gets credential isolation (proxy-resolved placeholders, the sandbox | `internal/preflight/` | Provider prerequisite validation | | `profiles/` | Sandbox profiles (TOML) | | `providers.toml` | Provider catalog (inputs, prerequisites, upstream tracking) | +| `gateways/` | Per-target gateway configs (`local/`, `ocp/`, `kind/`) | | `sandbox/` | Sandbox container image (Dockerfile, startup, policy, agent instructions) | | `sandbox/launcher/` | In-cluster launcher for OCP sandboxes | | `sandbox/profiles/` | OpenShell provider type profiles (YAML, imported into gateway) | -| `deploy/` | K8s manifests (RBAC, Route) | | `test/` | Tests (bats preflight, test-flow.sh integration) | ## Future Direction -- **Proto-based profiles** — align profile schema with OpenShell's proto types (`SandboxSpec`, `ProviderProfile`) for compile-time upstream compatibility. See [proto_migration.md](proto_migration.md). -- **Gateway configs** — per-target deployment configs (`gateways/ocp/`, `gateways/kind/`) replacing hardcoded deploy logic. See [docs/design.md](docs/design.md). +- **In-sandbox provider verification** — validate that providers actually work from inside a running sandbox, not just that credentials are present on the host. Catches expired tokens, proxy misconfig, and endpoint reachability issues. +- **Proto-based profiles** — align profile schema with OpenShell's proto types (`SandboxSpec`, `ProviderProfile`) for compile-time upstream compatibility. - **Direct gRPC** — replace CLI exec with gRPC calls to the gateway, eliminating output parsing. The Gateway interface already abstracts this swap. - **Shrink** — as OpenShell adds native support for GWS credentials, provider config injection, and in-cluster sandbox creation, remove the corresponding harness workarounds. ## Testing ```bash -make validate # full matrix: {bash,go} x {podman,ocp} +make validate-dev # build dev images + run integration tests go test ./... # Go unit tests bats test/preflight.bats # 29 preflight unit tests ``` diff --git a/cmd/new.go b/cmd/new.go index 97a0ab8..ed9f048 100644 --- a/cmd/new.go +++ b/cmd/new.go @@ -204,10 +204,7 @@ func newRemote(harnessDir string, gwCfg *gateway.GatewayConfig, gw gateway.Gatew "name": "launcher", "image": launcherImage, "imagePullPolicy": "Always", - "env": []map[string]any{ - {"name": "GATEWAY_ENDPOINT", "value": launcherEndpoint}, - {"name": "HOME", "value": "/tmp"}, - }, + "env": launcherEnv(launcherEndpoint, cfg.From), "volumeMounts": []map[string]any{ {"name": "config", "mountPath": "/etc/openshell/sandbox", "readOnly": true}, {"name": "gws", "mountPath": "/secrets/gws", "readOnly": true}, @@ -402,3 +399,14 @@ func newLocal(opts newLocalOpts) error { } return nil // unreachable but required by compiler } + +func launcherEnv(gatewayEndpoint, sandboxImage string) []map[string]any { + env := []map[string]any{ + {"name": "GATEWAY_ENDPOINT", "value": gatewayEndpoint}, + {"name": "HOME", "value": "/tmp"}, + } + if sandboxImage != "" { + env = append(env, map[string]any{"name": "SANDBOX_IMAGE", "value": sandboxImage}) + } + return env +} diff --git a/sandbox/launcher/main.go b/sandbox/launcher/main.go index 3defe2a..47c3708 100644 --- a/sandbox/launcher/main.go +++ b/sandbox/launcher/main.go @@ -267,6 +267,10 @@ func main() { os.Exit(1) } + if envImage := os.Getenv("SANDBOX_IMAGE"); envImage != "" { + cfg.From = envImage + } + fmt.Println("=== Sandbox Launcher ===") fmt.Printf(" Name: %s\n", cfg.Name) fmt.Printf(" From: %s\n", cfg.From)