diff --git a/.gitignore b/.gitignore index 88bacfe..620b4fd 100644 --- a/.gitignore +++ b/.gitignore @@ -15,10 +15,12 @@ __pycache__/ # Build artifacts harness -sandbox/launcher/openshell -sandbox/launcher/launcher +build/runner/harness infracluster/ -dashboard.html .claude/ -build/runner/harness -build/runner/openshell + +# Planning artifacts (local only) +plans/ +openshell-arch/ +strategy-dashboard.html +dashboard.html diff --git a/.goreleaser.yaml b/.goreleaser.yaml index 527177f..069e13a 100644 --- a/.goreleaser.yaml +++ b/.goreleaser.yaml @@ -1,7 +1,8 @@ version: 2 builds: - - env: + - binary: harness + env: - CGO_ENABLED=0 goos: - linux @@ -9,6 +10,8 @@ builds: goarch: - amd64 - arm64 + ldflags: + - -s -w -X main.version={{.Version}} archives: - format: tar.gz diff --git a/AGENTS.md b/AGENTS.md index d15412d..f63d77e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,7 +73,7 @@ Current workarounds and their upstream tracking: | Custom gateway image | `google-vertex-ai` provider not in released builds yet | Will ship in upstream release | | `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1` | Vertex AI rejects `context_management` beta header | Anthropic/Google to align APIs | | Atlassian `JIRA_URL`/`JIRA_USERNAME` as `--config` material | Provider v2 config keys not injected as env vars yet | OpenShell roadmap | -| In-cluster launcher Job | OpenShell doesn't have a native K8s-triggered sandbox creation | Potential future CRD | +| In-cluster runner Job | OpenShell doesn't have a native K8s-triggered sandbox creation | Potential future CRD | Previously worked around, now resolved: @@ -96,7 +96,7 @@ registration, credential injection, and the GWS OAuth token lifecycle. Runs in GitHub Actions on every PR. ``` ---ci flag = --no-providers --profile=ci --full +--ci flag = --no-providers --agent=ci --full ``` ### Make targets diff --git a/PROVIDERS-SPEC.md b/PROVIDERS-SPEC.md deleted file mode 100644 index 40423ab..0000000 --- a/PROVIDERS-SPEC.md +++ /dev/null @@ -1,102 +0,0 @@ -# Configuration Spec - -Three config files drive the harness: - -- **`providers.toml`** — provider definitions (inputs, types, checks) -- **`openshell.toml`** — which providers to enable, inference model -- **`profiles/*.toml`** — per-agent sandbox config (image, command, env vars) - -## providers.toml - -### Provider entry - -| Field | Required | Description | -|-------|----------|-------------| -| `name` | yes | Unique identifier | -| `type` | yes | `"openshell"` (registered with gateway) or `"custom"` (harness workaround) | -| `description` | yes | Shown in preflight output | -| `required` | no | If `true`, preflight `--strict` fails when inputs missing | -| `method` | no | Registration method (e.g., `"from-gcloud-adc"`) | -| `upstream` | no | Link to upstream issue (custom providers only) | - -### Input entry - -Inline tables in the `inputs` array: - -| Field | Required | Description | -|-------|----------|-------------| -| `key` | yes | Env var name, file path, or shell command | -| `kind` | yes | `"env"`, `"file"`, or `"check"` | -| `secret` | no | Mask value in preflight output. Default: `false` | - -### Input kinds - -- **`env`** — checks if env var is set. Shows `✓ local env: VAR=value` or masked if secret. -- **`file`** — checks file exists. Extracts metadata from known formats (ADC project, GWS client_id). -- **`check`** — runs shell command. Shows `✓ check: command` or `✗ check: command`. - -### Example - -```toml -[[providers]] -name = "github" -type = "openshell" -description = "GitHub API and git operations" -required = true -inputs = [ - { key = "GITHUB_TOKEN", kind = "env", secret = true }, -] - -[[providers]] -name = "atlassian" -type = "openshell" -description = "Jira and Confluence" -inputs = [ - { key = "JIRA_API_TOKEN", kind = "env", secret = true }, - { key = "JIRA_URL", kind = "env" }, - { key = "JIRA_USERNAME", kind = "env" }, - { key = "curl -sf ${JIRA_URL}/rest/api/2/serverInfo -o /dev/null", kind = "check" }, -] -``` - -## openshell.toml - -```toml -providers = ["github", "vertex-local", "atlassian"] -providers-custom = ["gws"] - -[inference] -model = "claude-sonnet-4-6" -``` - -If absent, all providers are enabled. - -## profiles/*.toml - -Per-agent sandbox configuration. `sandbox-podman.sh` and `sandbox-ocp.sh` read these. - -```toml -name = "agent" -image = "quay.io/rcochran/openshell:sandbox" -command = "claude --bare" -keep = true -providers = ["github", "vertex-local", "atlassian"] - -[env] -ANTHROPIC_BASE_URL = "https://inference.local" -ANTHROPIC_API_KEY = "sk-ant-openshell-proxy-managed" -JIRA_URL = "https://mysite.atlassian.net" -JIRA_USERNAME = "user@example.com" -``` - -The `[env]` section is uploaded as `sandbox.env` and sourced inside the sandbox. - -## Preflight - -`openshell-harness-preflight.sh` reads `providers.toml` + `openshell.toml` and checks: - -1. OpenShell CLI installed -2. Gateway reachable (podman or k8s, based on active gateway) -3. Each enabled provider's inputs (env vars, files, commands) -4. Reports `✓`/`✗` per input with `local env:`, `local file:`, `check:` prefixes -5. With `--strict`, exits non-zero if any required provider has missing inputs diff --git a/README.md b/README.md index da80be4..4963bd5 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,15 @@ # OpenShell Harness -Orchestration CLI for [OpenShell](https://github.com/NVIDIA/OpenShell). Automates gateway deployment, provider registration, and sandbox creation across local Podman and remote Kubernetes/OpenShift targets. Wraps the `openshell` CLI -- does not replace it. +Orchestration library and CLI for [OpenShell](https://github.com/NVIDIA/OpenShell). +* Wraps the `openshell` CLI -- does not replace it. +* Automates gateway deployment, provider registration, and sandbox creation across local Podman and remote Kubernetes/OpenShift targets. ## Where This Fits [OpenShell](https://github.com/NVIDIA/OpenShell) provides the runtime: gateway, sandboxes, L7 network policy, and credential proxy. It handles sandbox lifecycle and credential injection once providers are registered, but leaves gateway deployment orchestration, credential validation, and multi-target configuration to the user. -[NemoClaw](https://github.com/NVIDIA/NemoClaw) is NVIDIA's reference stack for running agents inside OpenShell with managed Nemotron inference. It bundles a specific model backend and onboarding flow. - This harness fills a different gap: multi-provider credential management (preflight validation, registration, health checks) across deployment targets (local Podman, kind, OpenShift) with declarative agent configs. It is model-agnostic -- the agent config chooses the entrypoint and inference backend. The harness orchestrates the infrastructure around it. -The harness wraps `openshell` CLI commands. As OpenShell adds native support for provider validation, multi-target deployment, and agent config rendering, the corresponding harness code shrinks. Every workaround tracks the upstream issue that would eliminate it (see [AGENTS.md](AGENTS.md)). - ## Example An agent config declares the sandbox image, entrypoint, and which providers to attach: @@ -61,7 +59,7 @@ tty: false ### Prerequisites - [OpenShell CLI](https://github.com/NVIDIA/OpenShell) (`brew install openshell && brew services start openshell` on macOS) -- Podman +- Podman/Docker - Go 1.23+ ### Credentials @@ -150,4 +148,4 @@ harness teardown [--sandboxes] [--providers] [--k8s] - [AGENTS.md](AGENTS.md) -- coding guidelines, project principles, workaround tracking - [SPEC.md](SPEC.md) -- full CLI specification -- [PROVIDERS-SPEC.md](PROVIDERS-SPEC.md) -- provider configuration format +- [TODO.md](TODO.md) -- roadmap and known gaps diff --git a/SPEC.md b/SPEC.md index d7f76fd..49fdd35 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,304 +1,132 @@ # OpenShell Harness Specification -This document specifies the behavior of the OpenShell Harness independently of its implementation. Any conforming implementation (bash, Go, Rust, Python) should produce the same observable behavior. +Behavior specification for the OpenShell Harness CLI. ## Overview -The harness deploys and manages AI agent sandboxes on two platforms: -- **Local** — Podman containers via a local OpenShell gateway -- **Remote** — Kubernetes pods via an OpenShift-hosted OpenShell gateway +The harness deploys and manages AI agent sandboxes on three targets: +- **Local** -- Podman containers via a local OpenShell gateway +- **Kind** -- Kubernetes pods via a kind cluster +- **Remote** -- Kubernetes pods via an OpenShift-hosted OpenShell gateway -Each sandbox is an isolated container with a Claude Code agent, credential providers, MCP servers, network policies, and uploaded configuration. +Each sandbox is an isolated container running an agent entrypoint, with credential providers, network policies, and a rendered payload (env.sh, run.sh, task.md). ---- +## Agent Config -## CLI - -The harness exposes a single entry point (`harness`) with subcommands. - -### `harness up [--local|--remote] [--profile NAME] [--name SANDBOX_NAME] [--no-tty]` - -Full flow: deploy gateway, register providers, and create a sandbox. This is the primary command. It performs these steps in order: - -1. **Ensure gateway** — if `--local`, verify the local podman gateway is running. If `--remote`, deploy to OpenShift (Helm chart, CRDs, SCCs, route, mTLS certs). If neither, check for an active gateway. -2. **Ensure providers** — if no providers are registered on the gateway, run provider registration. -3. **Ensure credentials** (remote only) — if K8s secrets for GWS/Atlassian don't exist, create them. -4. **Parse profile** — read `profiles/.toml` (default: `default`). -5. **Stage files** — write `sandbox.env` from profile `[env]`, export GWS credentials. -6. **Create sandbox** — call `openshell sandbox create` with `--from` (image), `--provider` (each provider), `--upload` (staged files), and the startup command. Retry up to 5 times for supervisor race conditions. +Agent configs live in `agents/*.yaml`. Each declares the sandbox image, entrypoint, providers, and environment: -If `--no-tty` is passed, the sandbox runs `startup.sh` and exits (for testing). Otherwise, it runs `startup.sh` then execs into the configured command (e.g., `claude --bare`). +```yaml +name: agent +image: ghcr.io/robbycochran/harness-openshell:sandbox +entrypoint: claude --bare +tty: true -If `--name` is not provided, the sandbox name comes from the profile's `name` field. +providers: + - profile: github + - profile: vertex-local + - profile: atlassian + config: + JIRA_URL: ${JIRA_URL} + JIRA_USERNAME: ${JIRA_USERNAME} + - profile: gws -### `harness create [NAME] [--profile NAME] [--no-tty]` - -Create a sandbox without deploying the gateway or registering providers. Errors if the gateway is not already running. Use this when the gateway and providers are already set up (e.g., after a previous `harness up` or `harness deploy` + `harness providers`). - -Accepts a positional `NAME` argument or `--name` flag. Otherwise behaves like the sandbox creation step of `harness up`. +env: + ANTHROPIC_BASE_URL: https://inference.local +``` -### `harness connect [SANDBOX_NAME]` +Fields: +- `name` (required) -- sandbox name, used for `openshell sandbox connect` +- `image` -- container image for the sandbox +- `entrypoint` -- command to run (default: `claude --bare`) +- `tty` -- enable TTY (default: false) +- `task` -- path to a task.md file, passed as argument to entrypoint +- `providers` -- list of provider profile references +- `providers[].profile` -- OpenShell provider profile name +- `providers[].config` -- non-secret config vars (resolved via `os.ExpandEnv`) +- `env` -- additional environment variables +- `include` -- extra files to include in the payload +- `policy` -- path to a network policy YAML + +Provider profiles live in `agents/providers/profiles/`. These are imported to the gateway during provider registration. -Reconnect to a running sandbox via `openshell sandbox connect`. - -### `harness deploy --local|--remote [--kubeconfig PATH]` +## CLI -Deploy or verify the gateway without creating a sandbox. Requires `--local` or `--remote`. +### `harness up [--local|--remote] [--agent NAME] [-f FILE] [--name SANDBOX] [--no-tty]` -**Local:** Check podman installed, find a gateway with endpoint `127.0.0.1`, select it, verify it responds. +Full flow: deploy gateway, register providers, render agent config, create sandbox. -**Remote:** Create namespace, install CRDs, grant SCCs, deploy Helm chart from OCI registry (`oci://ghcr.io/nvidia/openshell/helm-chart`), create TLS passthrough route, register CLI gateway with mTLS certs from the cluster. `--kubeconfig` sets the kubeconfig path (or set `KUBECONFIG` env var). +1. **Ensure gateway** -- deploy if needed (local: Podman, remote: Helm to K8s/OCP). +2. **Parse agent config** -- read `agents/.yaml` (default: `default`). `-f` overrides with a direct file path. +3. **Ensure providers** -- validate providers declared in the agent config. Auto-register missing ones. +4. **Render payload** -- `env.sh` (resolved env vars), `run.sh` (entrypoint wrapper), `task.md` (if set). +5. **Create sandbox** -- `openshell sandbox create` with `--upload` (payload) and the startup command. Retry up to 5 times for supervisor race conditions. -### `harness teardown [--sandboxes] [--providers] [--k8s]` +Local: sandbox created directly via the openshell CLI on the user's machine. +Remote: a runner Job is deployed to the cluster (`harness launch`), which creates the sandbox from inside the cluster with mTLS gateway access. -Tear down resources. Default (no flags) tears down everything applicable. +### `harness create [--agent NAME] [-f FILE] [--name SANDBOX]` -- `--sandboxes` — delete all sandboxes on the active gateway -- `--providers` — delete all providers and inference config (requires no running sandboxes) -- `--k8s` — Helm uninstall, delete CRDs, SCCs, secrets, namespace, and gateway config +Create a sandbox without deploying the gateway. Errors if no gateway is active. -### `harness preflight [--strict]` +### `harness connect [NAME]` -Read-only environment check. Validates all inputs defined in `providers.toml` for each enabled provider in `openshell.toml`. Reports per-input status with `✓`/`✗` prefixes. +Reconnect to a running sandbox via `openshell sandbox connect`. -With `--strict`, exits non-zero if any `required` provider has missing inputs. +### `harness deploy [local|ocp|kind]` -Subcommands: -- `harness preflight available` — print space-separated names of openshell-type providers where all inputs pass -- `harness preflight names` — print space-separated names of all enabled openshell-type providers +Deploy or verify the gateway for a target. Reads `gateways//gateway.toml`. ### `harness providers [--force]` -Register credential providers with the gateway: - -1. Enables providers v2 via `openshell settings set` -2. Imports custom provider profiles from `sandbox/profiles/` -3. Registers each provider (github, vertex-local, atlassian) if the required env vars are set: - - `github` — requires `GITHUB_TOKEN` - - `vertex-local` — requires ADC file + project ID (`ANTHROPIC_VERTEX_PROJECT_ID` or fallback from ADC's `quota_project_id`). Sets inference model from `OPENSHELL_MODEL` (default: `claude-sonnet-4-6`). - - `atlassian` — requires `JIRA_API_TOKEN` -4. Skips providers that already exist - -With `--force`: deletes existing providers and custom profiles before recreating. Requires no running sandboxes. - -### `harness test [podman|ocp|all] [--full]` +Register providers with the gateway. Reads `providers.toml` for the catalog, imports profiles from `agents/providers/profiles/`. -End-to-end validation. Quick mode: deploy → providers → gateway check → teardown. Full mode adds: sandbox create → verify env vars, GWS creds, MCP config, Claude responds → sandbox delete → teardown. +### `harness preflight [--strict]` ---- +Validate local credentials and prerequisites against `providers.toml`. -## Configuration +### `harness teardown [--sandboxes] [--providers] [--k8s]` -### `providers.toml` +Tear down resources. At least one flag required. -Catalog of provider definitions. Each `[[providers]]` entry has: +### `harness launch` (hidden) -| Field | Required | Description | -|-------|----------|-------------| -| `name` | yes | Unique identifier | -| `type` | yes | `"openshell"` or `"custom"` | -| `description` | yes | Shown in preflight | -| `required` | no | If true, `--strict` preflight fails when inputs missing | -| `method` | no | Registration method (e.g., `"from-gcloud-adc"`) | -| `upstream` | no | Link to upstream issue (custom providers) | +In-cluster command for the runner Job. Reads agent config from `/etc/openshell/sandbox/agent.yaml` (mounted ConfigMap), configures mTLS gateway, renders payload, creates sandbox, sets up environment. Not meant for direct user invocation. -Each provider has an `inputs` array of inline tables: +## Config Files -| Field | Required | Description | -|-------|----------|-------------| -| `key` | yes | Env var name, file path, or shell command | -| `kind` | yes | `"env"`, `"file"`, or `"check"` | -| `secret` | no | Mask value in preflight output | +| File | Purpose | +|------|---------| +| `agents/*.yaml` | Agent config: image, entrypoint, providers, env, task | +| `agents/providers/profiles/` | OpenShell provider profile YAMLs | +| `providers.toml` | Provider catalog: required inputs, health checks | +| `gateways/*/gateway.toml` | Deployment target config with Helm, images, RBAC | +| `openshell.toml` | Deployment-level overrides (enabled providers, inference model) | +| `sandbox/Dockerfile` | Sandbox image: OpenShell base + MCP servers + CLI tools | +| `build/runner/Dockerfile` | Runner image: harness binary + openshell CLI | +| `sandbox/policy.yaml` | Network egress rules applied to sandboxes | -### `openshell.toml` +## Environment Variables -Deployment configuration: +| Variable | Purpose | +|----------|---------| +| `SANDBOX_IMAGE` | Override sandbox image (dev/CI builds) | +| `RUNNER_IMAGE` | Override runner image (dev/CI builds) | +| `HARNESS_DIR` | Override harness directory detection | +| `OPENSHELL_NAMESPACE` | Override K8s namespace (default: `openshell`) | +| `OPENSHELL_CLI` | Override openshell binary path | +| `KUBECONFIG` | K8s cluster config for remote targets | -```toml -providers = ["github", "vertex-local", "atlassian"] -providers-custom = ["gws"] +## Payload -[inference] -model = "claude-sonnet-4-6" +The harness renders agent config into a self-contained payload uploaded to `/sandbox/.config/openshell/`: -[upstream] -chart-version = "0.0.55" ``` - -### `profiles/.toml` - -Per-sandbox configuration: - -```toml -name = "agent" -image = "quay.io/rcochran/openshell:sandbox" -command = "claude --bare" -keep = true -providers = ["github", "vertex-local", "atlassian"] - -[env] -ANTHROPIC_BASE_URL = "https://inference.local" -ANTHROPIC_API_KEY = "sk-ant-openshell-proxy-managed" -JIRA_URL = "https://mysite.atlassian.net" -JIRA_USERNAME = "user@example.com" +openshell/ + env.sh -- export KEY="value" (resolved from agent config) + run.sh -- sources env.sh, validates entrypoint, execs it + sandbox.env -- same as env.sh (sourced by sandbox startup.sh) + task.md -- task file with envsubst applied + bin/ -- wrapper scripts ``` -| Field | Default | Description | -|-------|---------|-------------| -| `name` | `"agent"` | Sandbox name (overridden by `--name`) | -| `image` | none | Container image for the sandbox | -| `command` | `"claude --bare"` | Command to exec after startup | -| `keep` | `true` | Keep sandbox alive after command exits | -| `providers` | `[]` | Provider names to attach | -| `[env]` | `{}` | Environment variables injected into the sandbox | - ---- - -## Sandbox Lifecycle - -### Creation - -1. Profile parsed → `SANDBOX_IMAGE`, `SANDBOX_COMMAND`, `SANDBOX_PROVIDERS`, `SANDBOX_ENV` -2. Files staged to `/tmp/openshell/`: - - `sandbox.env` — export statements from `[env]` section - - `credentials.json` — GWS OAuth credentials (if available) - - `client_secret.json` — GWS OAuth client config (if available) -3. `openshell sandbox create` called with: - - `--from ` — sandbox container image - - `--provider ` — for each provider - - `--upload /tmp/openshell:/sandbox/.config` — files land at `/sandbox/.config/openshell/` - - `-- bash -c '. /sandbox/startup.sh && exec '` (tty mode) - - `-- bash /sandbox/startup.sh` (no-tty mode) -4. On failure (supervisor race), delete sandbox and retry (up to 5 times, 5s between for local, 10s for OCP launcher) - -### Startup (inside sandbox) - -`startup.sh` runs once at creation: -1. Source `/sandbox/.config/openshell/sandbox.env` → append to `.bashrc` -2. Run `gh auth setup-git` - -### Connection - -`openshell sandbox connect ` opens an interactive session. Environment variables from `.bashrc` are inherited by Claude Code and its MCP servers. - -### Deletion - -`openshell sandbox delete ` or `harness teardown --sandboxes`. - ---- - -## Credential Flow - -| Credential | Provider Type | How It Works | -|------------|--------------|--------------| -| GitHub | `github` | PAT stored in gateway, proxy-managed. Sandbox sees `GITHUB_TOKEN` placeholder. | -| Vertex AI | `google-vertex-ai` | ADC-based OAuth via `--from-gcloud-adc`. Gateway refreshes tokens automatically. Inference routed through `inference.local`. | -| Atlassian | `atlassian` | API token stored in gateway. Proxy resolves base64 Basic auth header. `JIRA_URL`/`JIRA_USERNAME` injected via `sandbox.env`. | -| GWS | Custom (file upload) | Decrypted OAuth credentials uploaded to `/sandbox/.config/openshell/`. Not proxy-managed. | - ---- - -## Sandbox Image - -The sandbox image extends `ghcr.io/nvidia/openshell-community/sandboxes/base:latest` with: -- `mcp-atlassian` — Jira/Confluence MCP server -- `gws` CLI — Google Workspace -- `policy.yaml` — network egress rules -- `CLAUDE.md` — agent instructions -- `settings.json` — Claude Code permissions (`defaultMode: bypassPermissions`) -- `.mcp.json` — MCP server configuration (auto-loaded by Claude Code) -- `startup.sh` — runtime env setup - -The image is multi-arch (`linux/amd64` + `linux/arm64`), built with `docker buildx`. - ---- - -## Network Policy - -`sandbox/policy.yaml` controls which processes can reach which hosts: - -| Policy | Hosts | Binaries | -|--------|-------|----------| -| `claude_telemetry` | `*.anthropic.com`, `downloads.claude.ai`, `platform.claude.com`, `sentry.io` | claude, node | -| `github_git` | `github.com` (GET info/refs, POST git-upload-pack only) | git | -| `github_downloads` | `*.githubusercontent.com`, `codeload.github.com` | curl, gh, git, uv | -| `google_workspace` | `*.googleapis.com`, `oauth2.googleapis.com` | gws | -| `pypi` | `pypi.org`, `files.pythonhosted.org` | python, pip, uv | -| `npm` | `registry.npmjs.org` | npm, node | - -Git push (`git-receive-pack`) is blocked by default. - ---- - -## OCP-Specific - -### Gateway Deployment - -- Helm chart from `oci://ghcr.io/nvidia/openshell/helm-chart` (version pinned in `openshell.toml`) -- TLS passthrough route at `gateway-openshell.` -- mTLS: client certs copied from `openshell-client-tls` K8s secret to `~/.config/openshell/gateways//mtls/` -- `allowUnauthenticatedUsers: true` (mTLS is the auth layer) - -### In-Cluster Launcher - -For OCP sandboxes, a Kubernetes Job runs the launcher image (`sandbox/launcher/`): -1. Register gateway via `http://` trick (avoids cert generation probe), patch to `https://` + mTLS -2. Parse profile from mounted ConfigMap -3. Build provider flags, create sandbox with retry -4. Upload files (GWS creds, sandbox.env) -5. Run startup.sh via `sandbox exec` - -The launcher connects to the gateway at `https://openshell.openshell.svc.cluster.local:8080` using mounted mTLS certs from `openshell-client-tls`. - ---- - -## Environment Variables - -| Variable | Default | Description | -|----------|---------|-------------| -| `OPENSHELL_CLI` | `openshell` | Path or name of the openshell CLI binary | -| `OPENSHELL_MODEL` | `claude-sonnet-4-6` | Inference model for `harness providers` | -| `HARNESS_DIR` | auto-detected | Root directory of the harness project | -| `OPENSHELL_NAMESPACE` | `openshell` | Kubernetes namespace for OCP deployments | - ---- - -## Testing - -### Bats Tests (`test/preflight.bats`) - -29 bats tests covering the preflight check engine. Runs against both the Python (`lib/providers.py`) and Go (`harness preflight`) implementations via `USE_GO=true`. Uses stubbed CLI, isolated temp dirs, and no gateway dependency. Tests: -- env inputs (set/missing/secret/masked) -- file inputs (exists/missing/metadata extraction) -- check inputs (pass/fail/env expansion) -- provider status (all pass/any fail/required/optional) -- config filtering (enabled/disabled/no config) -- CLI detection (present/missing) -- Gateway detection (podman/k8s) - -### Go Unit Tests - -- `internal/gateway/cli_test.go` — stub-based tests for CLI output parsing and argument building -- `internal/profile/profile_test.go` — TOML parsing, env generation, provider validation with mock gateway -- `cmd/new_test.go` — orchestration tests for `up` and `create`: no gateway, missing providers, retry logic, create opts -- `sandbox/launcher/main_test.go` — launcher config parsing and file staging - -### Integration Tests (`test/test-flow.sh`) - -End-to-end validation requiring a live gateway: -- Quick mode: deploy → providers → gateway check → teardown -- Full mode: + sandbox create → verify env/GWS/MCP/Claude → delete → teardown -- Error scenarios: bad profile, teardown idempotency, missing providers -- Targets: `podman`, `ocp`, `all` - -Flags: -- `--go` — run the Go binary instead of bash scripts -- `--full` — include sandbox lifecycle tests -- `--reuse-gateway` — skip helm deploy/teardown-k8s, reuse existing gateway (49s vs 137s for OCP) - -### Test Matrix (`make validate`) - -Full validation across all paths. Run before every commit: -1. Go unit tests (harness + launcher) -2. Bats preflight (Python + Go paths) -3. Integration: `{bash, go}` × `{podman, ocp}` +`sandbox.env` is sourced as the sandbox's initial command, making env vars available in all subsequent `sandbox exec` sessions. `run.sh` is the entrypoint for interactive mode. diff --git a/TODO.md b/TODO.md index 2ea2edd..cf58479 100644 --- a/TODO.md +++ b/TODO.md @@ -1,79 +1,57 @@ -# TODO — Go Migration & Roadmap +# TODO — Roadmap -## Migration Status - -| Command | Go Status | Notes | -|---------|-----------|-------| -| `up --local` | Native | Full flow: deploy + providers + sandbox create with retry | -| `up --remote` | Native | K8s Job YAML via internal/k8s, prerequisite chain (deploy+providers+creds) | -| `create` | Native | Sandbox creation only (no deploy/providers) | -| `connect` | Native | exec into `openshell sandbox connect` | -| `deploy --local` | Native | Podman check, gateway find/select/verify | -| `deploy --remote` | Native | Helm install, Route, mTLS, RBAC, SCCs via internal/k8s | -| `teardown --sandboxes` | Native | SandboxList + SandboxDelete via Gateway | -| `teardown --providers` | Native | ProviderList + ProviderDelete + InferenceRemove | -| `teardown --k8s` | Native | Helm uninstall, CRDs, SCCs, secrets, namespace via internal/k8s | -| `preflight` | Native | All 29 bats tests pass against Go | -| `providers` | Native | Eliminates jq dependency | -| `test` | Bash | test-flow.sh orchestration (intentionally stays bash) | -| **Launcher** | Native | In-cluster Go binary, UBI9 + openssh | - -**Score: 12/13 paths native Go.** Only `test` stays bash (test orchestration, not a user command). - -## Architecture Improvements - -### Image registry as gateway config vs env override -- gateway.toml `[images]` section sets sandbox/launcher image refs -- `SANDBOX_IMAGE`/`LAUNCHER_IMAGE` env vars override config (for dev/CI) -- Two sources of truth: gateway.toml hardcodes a registry, env vars override it -- Consider: gateway.toml uses a `registry` field and images are relative to it, - or gateway.toml supports variable expansion (`${REGISTRY}:sandbox`) -- Not urgent — env override approach works as a bridge +## Architecture ### Direct gRPC (future) - OpenShell gateway exposes 54 gRPC RPCs (proto files in NVIDIA/OpenShell repo) -- Generate Go stubs from proto files → `gateway.GRPC` implementation -- Swap `gateway.NewCLI(cli)` → `gateway.NewGRPC(conn)` — one line change +- Generate Go stubs from proto files -> `gateway.GRPC` implementation +- Swap `gateway.NewCLI(cli)` -> `gateway.NewGRPC(conn)` -- one line change - Eliminates: openshell CLI binary dependency, output parsing fragility - Prerequisite: proto files stabilize (OpenShell is alpha) -### Remove Python dependency -- `providers.py` and `parse-profile.py` still in repo, called by bash path (`bin/harness`) -- Go implementations exist: `internal/preflight/` and `internal/profile/` -- Remove when: bash path is no longer needed for dual-testing -- Blocked by: decision to stop maintaining bash path - -### Launcher consolidation — DONE (#50) -- ~~`sandbox/launcher/` is a separate Go module~~ → deleted, replaced by `harness launch` -- ~~Has its own `parseConfig` duplicating `internal/profile/`~~ → uses `internal/agent/` -- Single binary, single image (`:runner`), single config format (`agents/*.yaml`) +### Image registry as gateway config vs env override +- gateway.toml `[images]` section sets sandbox/runner image refs +- `SANDBOX_IMAGE`/`RUNNER_IMAGE` env vars override config (for dev/CI) +- Two sources of truth: gateway.toml hardcodes a registry, env vars override it +- Consider: gateway.toml uses a `registry` field and images are relative to it -## Agent Schema +### Consolidate internal/profile into internal/agent +- `internal/profile/` has dead code: `Parse()`, `ParseFile()`, `BuildSandboxEnv()` +- Only `Config` struct, `ValidateProviders`, and `StageHarnessDir` still used +- Move `ValidateProviders` to agent or gateway package, inline `Config` into sandbox.go +- Remove TOML dependency -The agent config format is `agents/*.yaml` (YAML). TOML profiles are removed. +## Agent Config ### Future fields - -- [ ] `description` — one line of human-readable context per agent config -- [ ] `repo` — git URL to clone into the sandbox at start -- [ ] `secrets` — non-provider secrets to inject, cleaner than stuffing credentials into `env:` - -## Low-Priority Cleanup (from audit) - -- [ ] Unexport internal-only functions in `internal/preflight/` and `internal/profile/` -- [ ] `SandboxExec`/`SandboxUpload` on Gateway interface — no callers yet (premature abstraction, but harmless) +- [ ] `description` -- one line of human-readable context per agent config +- [ ] `repo` -- git URL to clone into the sandbox at start +- [ ] `secrets` -- non-provider secrets to inject ## Testing ### Current coverage -- 38 Go unit tests (gateway, profile, cmd) -- 7 launcher tests -- 29 bats preflight tests (Python + Go paths) -- Integration: `{bash, go}` × `{podman, ocp}` via `make validate` -- `--reuse-gateway` for fast OCP cycles (49s vs 137s) +- Go unit tests across cmd/, internal/agent, internal/gateway, internal/k8s +- 29 bats preflight tests +- Integration: local + kind + OCP via `make dev-test-all` -### Gaps to fill -- [ ] Integration test for `providers --force` (currently no test exercises force mode) -- [ ] Go + OCP integration (`test-flow.sh ocp --full --go`) — not yet validated +### Gaps +- [ ] Tests for cmd/launch.go (configureGateway mTLS, in-cluster flow) +- [ ] Integration test for `providers --force` - [ ] Preflight Go unit tests (internal/preflight/ has no _test.go) -- [ ] Kind gateway integration — `gateways/kind/gateway.toml` exists with full config (direct mode, nodeport) but is not exercised by test-flow.sh or CI. Add `test-flow.sh kind` and a GHA workflow with `kind create cluster`. +- [ ] Add bats step to `.github/workflows/ci.yml` (Makefile ci runs bats, GHA doesn't) + +## Release + +- [ ] Add CHANGELOG.md for 0.1 +- [ ] Add LICENSE file +- [ ] `harness init` command for standalone binary distribution (no repo clone) + +## Deferred (post-0.1) + +- [ ] Rename K8s SA `openshell-launcher` -> `openshell-runner` (breaking for deployed OCP clusters) +- [ ] Rename `LauncherSection` -> `RunnerSection` in gateway config TOML +- [ ] Gateway-level LLM proxy/logging (gateway.toml `[proxy]` section) +- [ ] Multi-agent workflow support (fleet.yaml / workflow.yaml) +- [ ] `harness policy suggest` (DenialEvent stream -> policy proposals) +- [ ] Fleet management (multi-gateway kubectl-context style) diff --git a/cmd/deploy_test.go b/cmd/deploy_test.go index 04ea332..887a433 100644 --- a/cmd/deploy_test.go +++ b/cmd/deploy_test.go @@ -14,7 +14,7 @@ func setupDeployHarnessDir(t *testing.T) string { t.Helper() dir := t.TempDir() os.WriteFile(filepath.Join(dir, "values-ocp.yaml"), []byte("image:\n pullPolicy: Always\n"), 0o644) - os.MkdirAll(filepath.Join(dir, "profiles"), 0o755) + os.MkdirAll(filepath.Join(dir, "agents"), 0o755) return dir } @@ -44,7 +44,7 @@ values = "values.yaml" manifests = ["addons/rbac.yaml", "addons/route.yaml"] [images] -launcher = "test-launcher:v1" +runner = "test-runner:v1" [ocp] scc-privileged = ["sa1", "sa2"] diff --git a/cmd/providers.go b/cmd/providers.go index ecb73d9..7e37e13 100644 --- a/cmd/providers.go +++ b/cmd/providers.go @@ -288,7 +288,7 @@ func registerGWS(harnessDir string, gw gateway.Gateway, enabled func(string) boo return nil } -// gwsProfileScopes reads the refresh.scopes list from sandbox/profiles/gws.yaml +// gwsProfileScopes reads the refresh.scopes list from agents/providers/profiles/gws.yaml // and returns them as a space-separated string for use as OAuth scope material. func gwsProfileScopes(harnessDir string) string { profilePath := filepath.Join(harnessDir, "sandbox", "profiles", "gws.yaml") diff --git a/cmd/providers_test.go b/cmd/providers_test.go index fdf8fd1..210cd28 100644 --- a/cmd/providers_test.go +++ b/cmd/providers_test.go @@ -13,7 +13,7 @@ import ( func setupProvidersTest(t *testing.T) string { t.Helper() dir := t.TempDir() - os.MkdirAll(filepath.Join(dir, "sandbox", "profiles"), 0o755) + os.MkdirAll(filepath.Join(dir, "agents", "providers", "profiles"), 0o755) return dir } diff --git a/cmd/sandbox.go b/cmd/sandbox.go index aa7a7e8..bfd53d0 100644 --- a/cmd/sandbox.go +++ b/cmd/sandbox.go @@ -11,8 +11,8 @@ import ( ) // sandboxOpts holds the parameters that vary between callers of -// createSandbox (upLocal vs createDirect). Everything else is -// derived from the profile.Config passed alongside. +// createSandbox (upLocal vs create). Everything else is derived +// from the agent config passed alongside. type sandboxOpts struct { harnessDir string gw gateway.Gateway diff --git a/credentials.md b/credentials.md deleted file mode 100644 index 8356606..0000000 --- a/credentials.md +++ /dev/null @@ -1,58 +0,0 @@ -# Credentials - -How credentials flow from your machine into sandboxes. - -## Overview - -| Credential | Provider type | Mechanism | Setup | -|------------|--------------|-----------|-------| -| GitHub token | `github` (built-in) | Bearer auth, proxy-resolved | `setup-providers.sh` | -| Vertex AI | `google-vertex-ai` (built-in) | `inference.local` gateway routing, OAuth refresh | `setup-providers.sh` | -| Atlassian API token | `atlassian` (custom profile) | Basic auth, proxy decodes base64 + resolves | `setup-providers.sh` | -| Atlassian URL/username | — | Uploaded as `atlassian.json` (not secrets) | `sandbox-podman.sh` / `sandbox-ocp.sh` | -| GWS OAuth | — | Decrypted export uploaded as files | `sandbox-podman.sh` / `setup-creds.sh` | - -## How it works - -Providers v2 (`providers_v2_enabled = true`) manages API tokens. The gateway -stores credentials and injects opaque placeholder tokens into sandboxes. The -sandbox proxy resolves placeholders in HTTP headers at request time — the -sandbox process never sees real values. - -**Inference routing:** Claude Code sends requests to `https://inference.local`. -The gateway proxies to Vertex AI using the `vertex-local` provider's -credentials. No Anthropic API key needed — `ANTHROPIC_API_KEY` is a dummy. - -**Basic auth (Atlassian):** The proxy decodes the base64 Basic auth header, -resolves the placeholder token inside, and re-encodes. - -## Access control - -- **GitHub:** read-only at proxy level (`access: read-only` in profile). Push requires `openshell policy set` with explicit `git-receive-pack` rules. -- **Atlassian:** read-only via `READ_ONLY_MODE=true` in mcp-atlassian config. - -## Rotation - -```bash -# GitHub — update provider -openshell provider update github --credential GITHUB_TOKEN="ghp_new" - -# Vertex AI — re-auth and recreate -gcloud auth application-default login -./setup-providers.sh --force - -# Atlassian — update provider + re-export env vars -openshell provider update atlassian --credential JIRA_API_TOKEN="new" - -# GWS — re-auth, new sandboxes get fresh export -gws auth login -``` - -## Revocation - -| Credential | Revoke at | -|------------|-----------| -| GCP ADC | `gcloud auth application-default revoke` | -| Atlassian | https://id.atlassian.com/manage-profile/security/api-tokens | -| GitHub | https://github.com/settings/tokens | -| GWS OAuth | https://myaccount.google.com/permissions | diff --git a/docs/profile-concepts.md b/docs/profile-concepts.md deleted file mode 100644 index 0c9a2fe..0000000 --- a/docs/profile-concepts.md +++ /dev/null @@ -1,228 +0,0 @@ -# Profile Concepts: OpenShell, Kaiden, and harness-openshell - -Three projects use the word "profile" to mean fundamentally different things. This document maps each concept, identifies where they overlap and diverge, and notes implications for harness-openshell. - ---- - -## 1. OpenShell — Provider Type Profiles - -**Source:** [NVIDIA/OpenShell#896](https://github.com/NVIDIA/OpenShell/issues/896) ("Enhanced Provider Management") - -### What it is - -A **Provider Type Profile** is a declarative YAML file that bundles everything OpenShell needs to know about a single external service (GitHub, Claude, Slack, Vertex AI, etc.): - -| Section | Purpose | -|---------|---------| -| `credentials` | Expected secrets, env vars, injection style (bearer, header, query, path) | -| `endpoints` | Host/port allowlist using the existing network policy language | -| `deny` | Deny rules the user cannot override (e.g., block branch-protection endpoints) | -| `binaries` | Allowed executables for this provider | -| `verify` | Optional probe endpoint for creation-time connectivity checks | -| `inference` | Base URL, protocol, default headers (inference-capable providers only) | -| `refresh` | Credential refresh strategy (static, oauth2, external) | - -### Key design decisions - -- **Policy composition:** Three layers composed JIT — base sandbox policy, auto-generated provider policy, user-authored policy. Deny wins over allow; provider deny rules cannot be bypassed. -- **Attach/detach lifecycle:** Providers can be added/removed from running sandboxes because credential injection is proxy-side, not env-var-at-start. -- **Custom registry:** Users can `export`, fork, and `import` their own profiles, enabling custom provider types without upstream changes. -- **Credential scoping:** Credentials bound to `(credential, endpoint, binary)` triples — a GitHub PAT can only reach `api.github.com`, not be exfiltrated to an arbitrary endpoint. - -### Implementation status (as of June 2026) - -**Shipped:** Profile schema, built-in profiles, custom registry (import/export/lint), JIT policy composition, attach/detach lifecycle, `providers_v2_enabled` feature gate, docs. - -**Remaining:** Profile-driven credential injection (replacing placeholder env scanning), credential scoping enforcement, credential verification, credential refresh lifecycle, inference automation (`inference.local` routing), HA hardening. - -### Open sub-issues (blockers for full vision) - -- [#894](https://github.com/NVIDIA/OpenShell/issues/894) — Placeholder model breaks SDKs that validate token format before any network call (Slack `xoxb-` prefix check) -- [#913](https://github.com/NVIDIA/OpenShell/issues/913) — WSS payloads bypass L7 rewriting (Discord gateway opcode 4004) - -Both motivate the move from placeholder env vars to proxy-side credential injection declared in provider profiles. - -### Related active work - -- [#1306](https://github.com/NVIDIA/OpenShell/issues/1306) — Gateway-owned credential refresh for short-lived tokens -- [#1622](https://github.com/NVIDIA/OpenShell/pull/1622) — Path-based `auth_style` for provider profiles -- [#1638](https://github.com/NVIDIA/OpenShell/pull/1638) — AWS SigV4 credential signing at the proxy -- [#1681](https://github.com/NVIDIA/OpenShell/pull/1681) — Okta OBO token exchange -- [#1736](https://github.com/NVIDIA/OpenShell/issues/1736) — Dynamic identity sources (SPIFFE/K8s) for OAuth2TokenExchange - ---- - -## 2. Kaiden — Projects (Workspace Configuration Profiles) - -**Source:** [openkaiden/kaiden#1272](https://github.com/openkaiden/kaiden/issues/1272) ("Projects management"), [Milestone 0.3 — Mycelium](https://github.com/openkaiden/kaiden/milestone/4) - -### What it is - -A **Project** is a reusable configuration template that defines everything needed to spin up an agent workspace. It sits between "raw resources" (models, MCP servers, skills, knowledge bases) and "running sandboxed workspaces." - -```typescript -interface WorkspaceProjectInfo { - id: string; - name: string; - folder: string; // local directory or git URL - skills: string[]; // enabled skill names - mcpServers: string[]; // enabled MCP server names - knowledges: string[]; // enabled RAG knowledge base names - secrets: string[]; // enabled secret names - filesystem: FilesystemConfiguration; // mode + mount points - network: NetworkConfiguration; // mode + allowed hosts -} -``` - -### Key design decisions - -- **Local-directory-first:** A project is anchored to a folder on the host machine (or a Git URL). The folder is the source code the agent will work in. -- **Resource selection, not definition:** Projects reference skills, MCP servers, knowledges, and secrets by name — they don't define them. The resources exist independently; projects toggle which ones are active. -- **Template → instance separation:** Projects are reusable templates; workspaces are ephemeral running instances created from projects. -- **UI-driven creation:** Two-step wizard (source → review) with per-resource checklists for skills, MCP servers, etc. -- **Stored as JSON** on disk under a `workspace-projects` directory. - -### Implementation status (as of June 2026, milestone due 2026-06-17) - -| Sub-issue | Status | What | -|-----------|--------|------| -| [#1925](https://github.com/openkaiden/kaiden/issues/1925) | Merged | Backend CRUD, JSON storage, reference validation | -| [#1926](https://github.com/openkaiden/kaiden/issues/1926) | Merged | Projects list page (searchable table) | -| [#1927](https://github.com/openkaiden/kaiden/issues/1927) | Merged | Project details page (Overview + Settings tabs) | -| [#1928](https://github.com/openkaiden/kaiden/issues/1928) | PR open | Create wizard (local folder or Git URL) | -| [#1929](https://github.com/openkaiden/kaiden/issues/1929) | Open | Wire MCP servers to create wizard | -| [#1930](https://github.com/openkaiden/kaiden/issues/1930) | PR open | Wire skills to create wizard | -| [#2066](https://github.com/openkaiden/kaiden/issues/2066) | Open | Feed project config into workspace creation | - -### What Kaiden's "project" is NOT - -- It is not about provider type definitions — Kaiden delegates that entirely to OpenShell -- It does not define credential injection, endpoint allowlists, or deny rules -- It does not manage credential refresh or verification -- It is a higher-level orchestration concern: "which combination of pre-existing resources does this workspace need?" - ---- - -## 3. harness-openshell — Profiles + Custom Providers - -### What it is - -**Profiles** in harness-openshell are TOML files (`profiles/*.toml`) that define per-sandbox configuration: - -```toml -image = "ghcr.io/org/agent-image:latest" -command = "claude --bare" -name = "agent" -keep = true -providers = ["github", "vertex-local", "atlassian"] - -[env] -SOME_VAR = "value" -``` - -**Custom providers** are a workaround layer for integrations OpenShell doesn't natively support yet. Defined in `providers.toml` with type `"custom"` (vs. `"openshell"` for gateway-managed providers). Today only `gws` (Google Workspace) is custom — its credentials are decrypted locally and uploaded directly into the sandbox, bypassing the gateway's credential store. - -### How the pieces fit - -``` -providers.toml profiles/*.toml sandbox/profiles/*.yaml -───────────── ────────────── ────────────────────── -Catalog of all known Per-sandbox config: OpenShell-native provider -providers + their image, command, env, type profiles (YAML) that -prerequisites (inputs) which providers to get imported into the - attach gateway via `openshell - provider profile import` -``` - -1. `harness providers` registers providers with the gateway (using `providers.toml` inputs) -2. `harness up --profile NAME` (or `harness create --profile NAME`) reads `profiles/NAME.toml`, validates its `providers` list against the gateway, creates the sandbox -3. Custom providers (like `gws`) bypass the gateway entirely — files are uploaded directly - -### Current limitations that motivated this analysis - -- Only one profile exists today: `default.toml` -- Custom providers are workarounds that should shrink as OpenShell's provider v2 matures -- The `sandbox/profiles/*.yaml` directory (OpenShell provider type profiles) only has `atlassian.yaml` — as more custom providers get native support upstream, more would move here -- Profile config doesn't express the resource-selection semantics Kaiden projects add (skills, MCP servers, knowledge bases) - ---- - -## Comparison Matrix - -| Dimension | OpenShell Provider Profile | Kaiden Project | harness-openshell Profile | -|-----------|---------------------------|----------------|---------------------------| -| **Unit of** | A service/integration type | A workspace template | A sandbox configuration | -| **Defines** | Credentials, endpoints, deny rules, binaries, inference, refresh | Which skills/MCP/secrets/knowledge are active, source folder, filesystem/network mode | Image, command, env vars, which providers to attach | -| **Scope** | Single provider (GitHub, Claude, etc.) | Entire workspace (many providers, tools, data sources) | Single sandbox (one image, one command, N providers) | -| **Stored as** | YAML in gateway registry | JSON on disk | TOML on disk | -| **Lifecycle** | Registered once, attached/detached per sandbox | Created once, instantiated as workspaces | Read at sandbox creation time | -| **Credential handling** | Proxy-side injection, refresh, scoping | References secrets by name (delegates to OpenShell) | Delegates to gateway (openshell providers) or uploads directly (custom providers) | -| **Network policy** | Auto-generated from endpoint declarations + deny rules | Mode + hosts array (delegates enforcement to OpenShell) | Not expressed — relies on gateway defaults + manual policy | -| **Custom extensibility** | export/fork/import registry | Add resources to the project's resource lists | Add TOML files to `profiles/` or YAML to `sandbox/profiles/` | - ---- - -## Key Observations - -### 1. Three layers of the same stack - -These three concepts operate at different layers of the same stack: - -``` -┌─────────────────────────────────────────────┐ -│ Kaiden Project │ "What combination of resources -│ (workspace template) │ does this workspace need?" -├─────────────────────────────────────────────┤ -│ harness-openshell Profile │ "What image, command, env, and -│ (sandbox configuration) │ providers does this sandbox use?" -├─────────────────────────────────────────────┤ -│ OpenShell Provider Type Profile │ "What does this single service -│ (service integration definition) │ need to work inside a sandbox?" -└─────────────────────────────────────────────┘ -``` - -They are complementary, not competing. Kaiden selects which providers a workspace gets; harness-openshell attaches those providers to a sandbox; OpenShell's provider profiles define what each provider actually means. - -### 2. Kaiden's "project" adds the resource-selection layer harness-openshell doesn't have - -harness-openshell profiles today are focused narrowly: image + command + env + provider list. Kaiden adds: -- **Skills selection** (enable/disable per workspace) -- **MCP server selection** (enable/disable per workspace) -- **Knowledge base selection** (RAG sources per workspace) -- **Secrets selection** (which secrets are accessible) -- **Filesystem configuration** (mount mode + mount points) -- **Network configuration** (mode + host allowlist) - -This is relevant if harness-openshell ever needs to support multi-agent or multi-tool configurations where different sandboxes need different subsets of available resources. - -### 3. harness-openshell's custom providers are a bridge - -The `providers.toml` custom provider mechanism (type: `"custom"`) exists specifically to fill gaps in OpenShell's provider v2 system. As #896's remaining work lands (especially profile-driven credential injection and credential refresh), custom providers like `gws` should migrate to native OpenShell provider type profiles. The `upstream` field in `providers.toml` already tracks which upstream issue would obsolete each workaround. - -### 4. OpenShell's credential model is evolving toward what harness-openshell needs - -The two open sub-issues under #896 (#894 and #913) are exactly the problems that make custom providers necessary in harness-openshell — SDKs that can't work with placeholder tokens need real credentials, which means bypassing the gateway. Once proxy-side injection and credential refresh land upstream, the custom provider workaround layer can shrink. - -### 5. Kaiden's worktree/local-dir focus is orthogonal to harness-openshell's image-first approach - -Kaiden projects start from a local folder (the code you want the agent to work on). harness-openshell profiles start from a container image (the pre-built agent environment). These are different entry points to the same result (a running sandbox), but they reflect different usage patterns: -- **Kaiden:** "I have source code, give me a configured agent to work on it" -- **harness-openshell:** "I have a pre-built agent image, configure the sandbox it runs in" - ---- - -## Implications for harness-openshell - -### Short term (now) -- Continue using custom providers as the bridge for services OpenShell doesn't natively support -- Track OpenShell #896 progress — each shipped sub-feature is a chance to remove a custom workaround -- Keep profiles simple (TOML, image + command + env + providers) — don't add Kaiden-style resource selection unless needed - -### Medium term (as OpenShell provider v2 matures) -- Migrate custom providers to native OpenShell provider type profiles (`sandbox/profiles/*.yaml`) -- Adopt profile-driven credential injection when it ships, eliminating direct credential upload -- Consider whether `providers.toml` should shrink to just a preflight validation catalog (no longer distinguishing custom vs. openshell types) - -### Long term (if scope expands) -- If harness-openshell needs multi-tool or multi-agent workspace configuration, Kaiden's resource-selection model (skills, MCP servers, knowledge bases per profile) is a reasonable pattern to follow -- The three-layer stack (provider type → sandbox config → workspace template) is a natural factoring if the project grows beyond single-sandbox use cases diff --git a/internal/status/status.go b/internal/status/status.go index 7e54175..02877f7 100644 --- a/internal/status/status.go +++ b/internal/status/status.go @@ -20,7 +20,7 @@ func Cmd(name string, args ...string) { redactNext = false continue } - if a == "--credential" { + if a == "--credential" || a == "--material" || a == "--secret-material-key" { redactNext = true fmt.Fprintf(os.Stderr, " %s", a) continue diff --git a/main.go b/main.go index 0f183e0..4762d4a 100644 --- a/main.go +++ b/main.go @@ -10,6 +10,8 @@ import ( "github.com/spf13/cobra" ) +var version = "dev" + func main() { harnessDir := detectHarnessDir() @@ -18,6 +20,7 @@ func main() { root := &cobra.Command{ Use: "harness", Short: "OpenShell Harness — deploy and manage AI agent sandboxes", + Version: version, SilenceErrors: true, SilenceUsage: true, PersistentPreRun: func(cmd *cobra.Command, args []string) { diff --git a/openshell.toml b/openshell.toml index b0f4ce9..32e83f8 100644 --- a/openshell.toml +++ b/openshell.toml @@ -5,7 +5,7 @@ # # Copy this file and customize per deployment: # cp openshell.toml my-team.toml -# ./openshell-harness-preflight.sh +# harness preflight # Which providers to register and attach to sandboxes. # Must match names defined in providers.toml or imported provider profiles. diff --git a/sandbox/startup.sh b/sandbox/startup.sh index 79444c6..9f5b6a1 100644 --- a/sandbox/startup.sh +++ b/sandbox/startup.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Runtime setup for the sandbox. Runs once via launcher's sandbox exec. +# Runtime setup for the sandbox. Runs once via harness launch. set -euo pipefail # ── Source env vars from agent config ───────────────────────────────── diff --git a/todo-improve.md b/todo-improve.md deleted file mode 100644 index cb4f5be..0000000 --- a/todo-improve.md +++ /dev/null @@ -1,97 +0,0 @@ -# harness-openshell improvement backlog - -Generated from 12-expert review (2 workflows, 6 agents each). -Triaged against design.md rearchitect plan. -Effort: S (<1hr), M (1-4hr), L (4hr+). - ---- - -## Bugs - -- [x] **quiet-mode kubectl never retries on transient errors** [S] -- [x] **non-seekable stdin retries replay empty input** [S] -- [x] **gateway timeout returns exit 0** [S] -- [x] **buggy contains()/indexOf() test helpers** [S] -- [x] **kubectl error message uses opts.Args not actual executed args** [S] -- [x] **runQuiet goroutine leak on timeout** [S] - -## Done — safety and quality - -- [x] **teardown no-args defaults to maximum destruction** [S] -- [x] **--local/--remote not mutually exclusive on up (formerly new)** [S] -- [x] **deployRemote prints no guidance on partial failure** [S] -- [x] **ProviderList/SandboxList dedup** — `parseFirstColumn` helper [S] -- [x] **RunHelm returns (string, error) but string is always empty** [S] -- [x] **runQuiet should use context.WithTimeout** [S] -- [x] **Merge preflight.go + check.go** [S] -- [x] **Merge parse.go into profile.go** [S] -- [x] **shared test mock in wrong file** → `cmd/helpers_test.go` [S] -- [x] **detectHarnessDir silently falls back to "."** [S] -- [x] **detectHarnessDir loop duplicated** [S] -- [x] **Makefile lacks lint/vet targets** [S] - -## Done — dead code - -- [x] `SandboxUpload` + `SandboxExec` — zero callers -- [x] `status.Warnf` — zero callers -- [x] `decodeBase64` — trivial wrapper, inlined -- [x] `cmd.Stderr = nil` in `runOutput` — no-op -- [x] `internal/util` package — inlined at call site -- [x] Four Gateway sub-interfaces — zero consumers - ---- - -## Do now — quick independent fixes - -These are safe to do before the rearchitect. Each is independent and -won't conflict with design.md changes. - -- [x] **error wrapping inconsistency** — audited, all already use `%w` [S] -- [x] **env-var-with-fallback** — added `envOr()` helper, applied to 4 cases [S] -- [x] **launcher binary swallows errors** — check cmd.Run(), json.Unmarshal, copyFile close, mTLS 0700 [S] -- [ ] **duplicated Config struct in launcher** — separate go.mod, can't import internal/. Deferred. [S] -- [x] **test writes to hardcoded /tmp path** — use `t.TempDir()` [S] -- [x] **secret name string literals** — shared `secretNames` slice [S] -- [x] **hardcoded sk-ant- placeholder and personal email** — removed PII from default.toml [S] -- [x] **CheckInput file case** — flattened with early returns [S] -- [x] **deployLocal unnecessary podmanPath** — inlined [S] -- [x] **force-deleting pods with --grace-period=0** — changed to `--grace-period=30` [S] -- [x] **ConfigFile.ChartVersion alias** — removed, use `cfg.Upstream.ChartVersion` [S] -- [x] **pickKeys/pickKeysExcept/formatMeta** — deleted, inlined at call site [S] -- [x] **extractYAMLID** — simplified with `strings.CutPrefix` [S] -- [ ] **unreachable return nil** — `cmd/new.go` (already annotated with comment) [S] -- [ ] **swallowed errors in deploy.go and new.go** — remaining discarded errors [S] -- [ ] **sandbox CRD from unversioned latest URL** — pin version [S] -- [x] **SCC grant/revoke lists** — shared `sccPrivilegedSAs` slice [S] -- [ ] **launcher image single-arch** — use `docker buildx` for launcher. `Makefile` [S] - -## Config improvements - -- [ ] **Provider env vars (JIRA_URL, JIRA_USERNAME) should come from environment** — these are validated as `kind = "env"` inputs in `providers.toml` but hardcoded in `profiles/default.toml`. Profile should use `$JIRA_URL` / `$JIRA_USERNAME` references expanded at build time via `os.ExpandEnv`, so changing a Jira URL doesn't require editing the profile. Needs `BuildSandboxEnv` to expand env var references in values. [S] -- [ ] **Provider-specific env vars should flow from provider config, not profile** — longer-term, JIRA_URL and JIRA_USERNAME shouldn't be in the profile at all. Provider registration should carry config values that get injected into the sandbox automatically. See design.md "Config files" section. [M] - -## Defer to rearchitect (design.md) - -These will be restructured or replaced during the command/code reorg. - -- [ ] **Move orchestration out of cmd/** → the rearchitect IS this [L] -- [ ] **Table-driven provider registration** → redesigned with `providers register` [M] -- [ ] **Preflight subcommands** → redesigned in new command structure [S] -- [ ] **Cobra examples** → add after commands are renamed [S] -- [ ] **Hardcoded values → harness.toml config** → addressed by config redesign [M] -- [x] **Context propagation + cancellation** → split into create/up is done [L] -- [ ] **Gateway CLI timeout** → comes with context propagation [M] -- [ ] **kubectl log tailing bypasses k8s.Runner** → restructured code [S] -- [ ] **Configmap creation dedup** → restructured code [S] -- [ ] **No test coverage for registerProviders/RunCheck** → rewritten functions [M] -- [ ] **ProviderGet → ProviderExists rename** → interface redesign [S] -- [ ] **CLIVersion removal** → interface cleanup [S] -- [ ] **Inline Job spec → YAML template** → move to deploy/ [M] -- [ ] **Gateway CLI output parsing** → depends on openshell `--output json` [M] -- [ ] **GWS credential export flow duplicated** → restructured code [M] -- [ ] **Provider registration order** → `inference_provider` config field [S] - -## Critical — address during rearchitect - -- [ ] **cluster-admin ClusterRoleBinding + privileged SCC on default SA** — scoped ClusterRole in `deploy/`, use `kubectl apply`, remove `default` from SCC grants [M] -- [ ] **credentials visible in ps aux and verbose logs** — pass via subprocess env vars or stdin, add credential redaction to `status.Cmd` [M]