docs: rewrite README and PROVIDERS-SPEC for current architecture

PROVIDERS-SPEC.md

@@ -1,123 +1,102 @@
-# Providers Configuration Spec
+# Configuration Spec
 
-Two files define the harness configuration:
+Three config files drive the harness:
 
-- **`providers.toml`** — catalog of provider definitions with their inputs (checked into repo)
-- **`openshell.toml`** — deployment config selecting which providers to enable (per-user)
+- **`providers.toml`** — provider definitions (inputs, types, checks)
+- **`openshell.toml`** — which providers to enable, inference model
+- **`agents/*.toml`** — per-agent sandbox config (image, command, env vars)
 
 ## providers.toml
 
 ### Provider entry
 
-Each provider is a `[[providers]]` array entry with these fields:
-
 | Field | Required | Description |
 |-------|----------|-------------|
-| `name` | yes | Unique identifier (used in openshell.toml to enable) |
+| `name` | yes | Unique identifier |
 | `type` | yes | `"openshell"` (registered with gateway) or `"custom"` (harness workaround) |
-| `description` | yes | Human-readable description shown in preflight |
-| `required` | no | If `true`, preflight `--strict` fails when inputs are missing. Default: `false` |
+| `description` | yes | Shown in preflight output |
+| `required` | no | If `true`, preflight `--strict` fails when inputs missing |
 | `method` | no | Registration method (e.g., `"from-gcloud-adc"`) |
-| `upstream` | custom only | Link to upstream issue that would make this provider native |
+| `upstream` | no | Link to upstream issue (custom providers only) |
 
 ### Input entry
 
-Each provider has an `[[providers.inputs]]` sub-array. Each input has:
+Inline tables in the `inputs` array:
 
 | Field | Required | Description |
 |-------|----------|-------------|
 | `key` | yes | Env var name, file path, or shell command |
-| `kind` | no | `"env"` (default), `"file"`, or `"check"` |
-| `secret` | no | If `true`, value is masked in preflight output and delivered via the provider system. If `false` (default), env vars are written to sandbox.env for the sandbox to source. |
+| `kind` | yes | `"env"`, `"file"`, or `"check"` |
+| `secret` | no | Mask value in preflight output. Default: `false` |
 
 ### Input kinds
 
-- **`env`** — environment variable. Preflight checks if set, shows value (or masked if `secret = true`)
-- **`file`** — filesystem path. Preflight checks existence, extracts metadata from known formats (ADC project, GWS client_id)
-- **`check`** — shell command. Preflight runs it and reports pass/fail
+- **`env`** — checks if env var is set. Shows `✓ local env: VAR=value` or masked if secret.
+- **`file`** — checks file exists. Extracts metadata from known formats (ADC project, GWS client_id).
+- **`check`** — runs shell command. Shows `✓ check: command` or `✗ check: command`.
 
 ### Example
 
 ```toml
 [[providers]]
 name = "github"
 type = "openshell"
-
-description = "GitHub API and git operations (read-only)"
+description = "GitHub API and git operations"
 required = true
 inputs = [
-  { key = "GITHUB_TOKEN", kind = "env", secret = true, description = "GitHub PAT" },
-]
-
-[[providers]]
-name = "vertex-local"
-type = "openshell"
-
-method = "from-gcloud-adc"
-description = "Google Vertex AI inference via gateway-managed OAuth"
-required = true
-inputs = [
-  { key = "ANTHROPIC_VERTEX_PROJECT_ID", kind = "env", description = "GCP project ID" },
-  { key = "CLOUD_ML_REGION", kind = "env", description = "Vertex AI region" },
-  { key = "~/.config/gcloud/application_default_credentials.json", kind = "file", description = "GCP ADC" },
+  { key = "GITHUB_TOKEN", kind = "env", secret = true },
 ]
 
 [[providers]]
 name = "atlassian"
 type = "openshell"
-
-description = "Jira and Confluence (read-only, Basic auth resolved by proxy)"
-inputs = [
-  { key = "JIRA_API_TOKEN", kind = "env", secret = true, description = "Atlassian API token" },
-]
-
-[[providers]]
-name = "atlassian-config"
-type = "custom"
-description = "Atlassian URL and username (non-secret, uploaded to sandbox)"
-inputs = [
-  { key = "JIRA_URL", kind = "env", description = "Atlassian site URL" },
-  { key = "JIRA_USERNAME", kind = "env", description = "Atlassian username" },
-]
-
-[[providers]]
-name = "gws"
-type = "custom"
-description = "Google Workspace (Gmail, Calendar, Drive)"
-upstream = "https://github.com/NVIDIA/OpenShell/issues/1268"
+description = "Jira and Confluence"
 inputs = [
-  { key = "~/.config/gws/client_secret.json", kind = "file", description = "GWS OAuth client config" },
-  { key = "gws auth status", kind = "check", description = "GWS authentication" },
+  { key = "JIRA_API_TOKEN", kind = "env", secret = true },
+  { key = "JIRA_URL", kind = "env" },
+  { key = "JIRA_USERNAME", kind = "env" },
+  { key = "curl -sf ${JIRA_URL}/rest/api/2/serverInfo -o /dev/null", kind = "check" },
 ]
 ```
 
 ## openshell.toml
 
-Selects which providers from the catalog to enable:
-
 ```toml
 providers = ["github", "vertex-local", "atlassian"]
-providers-custom = ["gws", "atlassian-config"]
+providers-custom = ["gws"]
+
+[inference]
+model = "claude-sonnet-4-6"
+```
 
-[sandbox]
+If absent, all providers are enabled.
+
+## agents/*.toml
+
+Per-agent sandbox configuration. `sandbox-podman.sh` and `sandbox-ocp.sh` read these.
+
+```toml
+name = "agent"
 image = "quay.io/rcochran/openshell:sandbox"
 command = "claude --bare"
+keep = true
+providers = ["github", "vertex-local", "atlassian"]
 
-[inference]
-model = "claude-sonnet-4-6"
+[env]
+ANTHROPIC_BASE_URL = "https://inference.local"
+ANTHROPIC_API_KEY = "sk-ant-openshell-proxy-managed"
+JIRA_URL = "https://mysite.atlassian.net"
+JIRA_USERNAME = "user@example.com"
 ```
 
-If `openshell.toml` is absent, all providers are enabled.
+The `[env]` section is uploaded as `sandbox.env` and sourced inside the sandbox.
 
-## Preflight behavior
+## Preflight
 
-`openshell-harness-preflight.sh` reads both files and for each enabled provider:
+`openshell-harness-preflight.sh` reads `providers.toml` + `openshell.toml` and checks:
 
-1. Checks all inputs (env vars set? files exist? commands pass?)
-2. Prints status per input with details:
-   - `env` + `secret = true` → masked value (`ghp_***`)
-   - `env` + `secret = false` → full value
-   - `file` → existence + extracted metadata (project ID, client ID)
-   - `check` → pass/fail
-3. Reports provider as ✓ (all inputs present), ✗ (required + missing), or - (optional + missing)
-4. With `--strict`, exits non-zero if any required provider has missing inputs
+1. OpenShell CLI installed
+2. Gateway reachable (podman or k8s, based on active gateway)
+3. Each enabled provider's inputs (env vars, files, commands)
+4. Reports `✓`/`✗` per input with `local env:`, `local file:`, `check:` prefixes
+5. With `--strict`, exits non-zero if any required provider has missing inputs

README.md

@@ -1,107 +1,98 @@
-# OpenShell Harness for OpenShift
+# OpenShell Harness
 
-Deploy OpenShell sandboxes on OpenShift with Claude Code (Vertex AI), Atlassian MCP, Google Workspace, and GitHub integrations.
-
-## What This Is
-
-A deployment harness for running AI agent sandboxes on OpenShift using [OpenShell](https://github.com/NVIDIA/OpenShell). Each sandbox gets:
+Deploy AI agent sandboxes on Podman (local) or OpenShift using [OpenShell](https://github.com/NVIDIA/OpenShell). Each sandbox gets:
 
 - **Claude Code** via Google Vertex AI (`inference.local` routing)
-- **Jira/Confluence** via mcp-atlassian MCP server (read-only)
+- **Jira/Confluence** via mcp-atlassian MCP server
 - **Gmail, Calendar, Drive** via gws CLI
-- **GitHub** via gh CLI (read-only at proxy level)
+- **GitHub** via gh CLI
 - Network policy enforcement per sandbox
-- Persistent workspace across reconnects
-
-## Prerequisites
 
-- OpenShift cluster with `KUBECONFIG` set
-- `kubectl`, `helm` on PATH
-- OpenShell CLI (`openshell`) built from source (>= 0.0.55-dev for `google-vertex-ai` provider)
-- NVIDIA/OpenShell repo cloned (for the Helm chart)
-- `gcloud auth application-default login` completed
-- Custom images pushed to `quay.io/rcochran/openshell` (sandbox + gateway)
+## Quick Start (Local)
 
-## Quick Start (Local — Podman/Docker)
-
-```shell
-# 1. Install OpenShell (auto-starts the gateway)
+```bash
+# 1. Install OpenShell
 curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
 
-# 2. Verify gateway is running
+# 2. Verify gateway
 ./deploy-podman.sh
 
-# 3. Register providers (one-time per gateway)
+# 3. Register providers (one-time)
 export GITHUB_TOKEN="ghp_..."
 export JIRA_API_TOKEN="..."
+export ANTHROPIC_VERTEX_PROJECT_ID="my-project"
+export CLOUD_ML_REGION="us-east5"
 ./setup-providers.sh
 
 # 4. Launch a sandbox
-export JIRA_URL="https://mysite.atlassian.net"
-export JIRA_USERNAME="user@example.com"
 ./sandbox-podman.sh
 ```
 
 ## Quick Start (OpenShift)
 
-```shell
-# 1. Build and push images (one-time, or on version bumps)
-make push-sandbox push-launcher push-gateway push-supervisor
-
-# 2. Deploy to the cluster
+```bash
+# 1. Deploy gateway to cluster
 ./deploy-ocp.sh
 
-# 3. Store credentials in cluster + register providers
+# 2. Store credentials + register providers
 ./setup-creds.sh
 ./setup-providers.sh
 
-# 4. Launch a sandbox
-# or: ./sandbox-ocp.sh
+# 3. Launch a sandbox
+./sandbox-ocp.sh
 ```
 
-## Files
+## Agent Configs
 
-| File | Purpose |
-|------|---------|
-| `deploy-podman.sh` | Verify local gateway is running (Podman/Docker) |
-| `deploy-ocp.sh` | Deploy OpenShell to OpenShift (Helm, SCCs, route) |
-| `setup-providers.sh` | Register credential providers — works on any gateway |
-| `sandbox-podman.sh` | Launch sandbox on local gateway (direct CLI) |
-| `sandbox-ocp.sh` | Launch sandbox on OpenShift (kubectl apply) |
-| `sandbox/Dockerfile` | Custom sandbox image (extends community base) |
-| `sandbox/policy.yaml` | Network policy (endpoints not covered by provider profiles) |
-| `sandbox/startup.sh` | Runtime env, GWS, MCP config |
-| `sandbox/profiles/atlassian.yaml` | Custom provider v2 profile for Atlassian |
-| `sandbox/CLAUDE.md` | Agent instructions baked into sandbox image |
-| `sandbox/settings.json` | Claude permissions baked into sandbox image |
-| `credentials.md` | Credential flows, mechanisms, and rotation guide |
-| `AGENTS.md` | Project principles and workaround tracking |
+Sandboxes are configured via `agents/*.toml`:
 
-## Credentials
+```toml
+# agents/default.toml
+name = "agent"
+image = "quay.io/rcochran/openshell:sandbox"
+command = "claude --bare"
+providers = ["github", "vertex-local", "atlassian"]
 
-See [credentials.md](credentials.md) for the full reference.
+[env]
+ANTHROPIC_BASE_URL = "https://inference.local"
+JIRA_URL = "https://mysite.atlassian.net"
+```
 
-| Credential | Mechanism | Setup |
-|------------|-----------|-------|
-| GitHub | Provider v2 (`github` profile, read-only) | `./setup-providers.sh` |
-| Vertex AI | Provider v2 (`google-vertex-ai`, `inference.local`) | `./setup-providers.sh` |
-| Atlassian | Provider v2 (`atlassian` profile, Basic auth resolved by proxy, read-only) | `./setup-providers.sh` |
-| Google Workspace | File upload (encrypted local files) | Pre-authenticate with `gws auth login` |
+Launch with a specific config: `./sandbox-podman.sh research` (uses `agents/research.toml`).
 
-## Sandbox Usage
+## Testing
+
+```bash
+bats test/preflight.bats        # unit tests (29 tests)
+./test-flow.sh podman --full    # full local validation
+./test-flow.sh ocp --full       # full OCP validation
+make test                       # build images + test both
+```
+
+## Files
 
-```shell
-# Local
-./sandbox-podman.sh                          # launch
-./sandbox-podman.sh --name dev               # named sandbox
+| File | Purpose |
+|------|---------|
+| `agents/default.toml` | Agent config (image, command, providers, env vars) |
+| `providers.toml` | Provider definitions (env/file/check inputs) |
+| `openshell.toml` | Which providers to enable, inference model |
+| `deploy-podman.sh` | Verify local gateway is running |
+| `deploy-ocp.sh` | Deploy OpenShell to OpenShift (Helm + route) |
+| `setup-providers.sh` | Register providers with the gateway |
+| `setup-creds.sh` | Store GWS + Atlassian config in cluster (OCP only) |
+| `sandbox-podman.sh` | Launch sandbox locally |
+| `sandbox-ocp.sh` | Launch sandbox on OpenShift |
+| `teardown.sh` | Tear down sandboxes, providers, k8s resources |
+| `test-flow.sh` | End-to-end validation |
+| `openshell-harness-preflight.sh` | Pre-flight environment check |
+| `AGENTS.md` | Project principles and workaround tracking |
 
-# OpenShift
-./sandbox-ocp.sh my-config.yaml             # use a custom config
+## Sandbox Usage
 
-# Either platform
-openshell sandbox connect <name>            # reconnect to running sandbox
-openshell sandbox list                      # list sandboxes
-openshell sandbox delete <name>             # delete a sandbox
+```bash
+openshell sandbox connect <name>    # reconnect
+openshell sandbox list              # list running
+openshell sandbox delete <name>     # delete
 ```
 
 ## Architecture
@@ -110,17 +101,17 @@ openshell sandbox delete <name>             # delete a sandbox
 Your Mac                         OpenShift Cluster
 ┌──────────┐                   ┌──────────────────────────────┐
 │ openshell│   OpenShift Route │ Gateway (StatefulSet)         │
-│ CLI      ├──────────────────▶│   ├─ gRPC API                 │
+│ CLI      ├──────────────────►│   ├─ gRPC API                 │
 │          │   TLS passthrough │   ├─ inference.local proxy     │
 │          │   mTLS :443       │   ├─ Provider credential store │
 └──────────┘                   │   └─ OAuth token refresh       │
                                │                               │
                                │ Sandbox Pods                  │
                                │   ├─ Claude Code → inference  │
                                │   │   .local → Vertex AI      │
-                               │   ├─ mcp-atlassian (read-only)│
+                               │   ├─ mcp-atlassian            │
                                │   ├─ gws CLI                  │
-                               │   ├─ gh CLI (read-only)       │
+                               │   ├─ gh CLI                   │
                                │   └─ Network proxy            │
                                └──────────────────────────────┘
 ```