feat: kubectl-style apply command with --dry-run and -o yaml

.agents/skills/validate/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: validate
+description: Run the full test matrix for harness-openshell. Use when asked to "validate", "run tests", "check everything", or before any commit/push/PR.
+---
+
+# Validate
+
+Run the full test matrix. Skip steps that require unavailable infrastructure.
+
+## Steps
+
+Run each step sequentially. Report pass/fail for each.
+
+### 1. Build
+
+```bash
+go build ./...
+```
+
+### 2. Unit tests
+
+```bash
+CGO_ENABLED=0 go test ./...
+```
+
+### 3. Vet
+
+```bash
+go vet ./...
+```
+
+### 4. Local integration (full providers)
+
+Requires: `openshell` running locally, provider credentials configured.
+
+```bash
+make test-local
+```
+
+Skip if `openshell` is not on PATH or the gateway is not running.
+
+### 5. Local integration (CI mode)
+
+Requires: `openshell` running locally. No credentials needed.
+
+```bash
+make test-local
+```
+
+Pass `--ci` to `test-flow.sh` (auto-detected when `CI=true`).
+
+### 6. OCP integration
+
+Requires: `KUBECONFIG` set, cluster accessible.
+
+```bash
+make test-remote
+```
+
+Skip if `KUBECONFIG` is not set or `kubectl cluster-info` fails.
+
+### 7. Kind integration
+
+Requires: `kind` on PATH.
+
+```bash
+make test-kind
+```
+
+Skip if `kind` is not on PATH.
+
+### 8. CI status
+
+Check if CI is green for the current branch:
+
+```bash
+gh run list --branch $(git branch --show-current) --limit 3
+```
+
+## Output
+
+Report a summary table:
+
+```
+Validation Results
+──────────────────
+  Build:          PASS
+  Unit tests:     PASS (6 packages)
+  Vet:            PASS
+  Local (full):   PASS (22/22)
+  Local (CI):     PASS (14/14)
+  OCP:            PASS (10/10)
+  Kind:           SKIP (kind not installed)
+  CI:             GREEN (3/3 workflows)
+```

AGENTS.md

@@ -62,6 +62,56 @@ For multi-step tasks, state a brief plan:
 
 3. **Upstream alignment** — OpenShell is alpha and developing quickly. Don't fight the framework. Use native patterns (providers v2, profiles, inference.local, policy composition) and adapt when upstream changes.
 
+## Upstream Conventions
+
+Follow these conventions from the OpenShell ecosystem. Do not invent alternatives.
+
+### Output format: `-o table|json|yaml`
+Every list/get command must support `-o` with three formats. `table` is the default
+for human consumption, `json` and `yaml` for machine consumption. Define a shared
+`OutputFormat` type used by all commands. Match the convention from OpenShell
+issues #1745 and #1750.
+
+### Credential exclusion from structured output
+Never serialize credential values into `-o json` or `-o yaml` output. Expose key
+names only. This is a security invariant, not a nice-to-have. See OpenShell
+PR #1830 `provider_to_json()` for the pattern.
+
+### Flag resolution order
+Explicit flag > `OPENSHELL_*` env var > config file > default. This matches the
+plugin host contract from issue #1851. The env vars `OPENSHELL_GATEWAY`,
+`OPENSHELL_GATEWAY_ENDPOINT`, and verbosity flags propagate from the plugin host.
+
+### Policy schema
+`kind: policy` documents must use the upstream OpenShell policy YAML schema
+verbatim (from the `openshell-policy` crate). Do not invent a harness-specific
+policy format. A policy written for the harness should be byte-compatible with
+what `openshell-image-builder` generates.
+
+### Provider abstraction
+`kind: provider` is an abstraction layer, not a thin wrapper around
+`openshell provider create`. The backend may change to gateway.toml entries
+(#1886) or K8s CRDs (#1719) as upstream settles. Implement the imperative
+CLI backend today. Do not hard-code the execution strategy.
+
+### Plugin compatibility
+The binary may eventually be discoverable as an OpenShell plugin via
+`openshell-<name>` PATH-based discovery (#1851). Design for standalone first.
+Plugin compatibility (binary naming, env var consumption) is additive. Do not
+depend on plugin host behavior that is not yet accepted upstream.
+
+### Do not cache or forward auth tokens
+Issue #1851 explicitly prohibits token forwarding to plugins. The harness must
+resolve credentials fresh via `openshell-bootstrap` or configured auth. Never
+store, cache, or relay gateway auth tokens.
+
+### Delegate image building
+Do not replicate Dev Container Feature fetching or complex image composition.
+The `openshell-image-builder` handles agent installation, settings bake-in,
+policy composition, and OCI artifact fetching. If the harness needs advanced
+image support, generate config the image-builder consumes rather than
+reimplementing in Go.
+
 ## The harness should shrink, not grow
 
 This harness exists to bridge gaps in OpenShell's current capabilities. As OpenShell matures, custom code should be replaced by upstream features. Every workaround should reference the upstream issue that would eliminate it.

TODO.md

@@ -30,76 +30,101 @@
 - [ ] Document non-secret provider env vars (what `providers[].env` captures
       and why it exists alongside secret credentials)
 
-## CLI
-
-- [ ] Flows that support agent.yaml (`create`, `up`) should also support
-      `--provider-profile` and provider config overrides
+## CLI — kubectl-style refactor
+
+Sequenced implementation plan. Each phase builds on the previous.
+
+### Phase 1: `apply` command (replaces `up` and `create`)
+- [ ] `cmd/apply.go` — unified deploy command, delegates to existing `upLocal()`
+- [ ] `-f` flag as primary interface (replaces `--agent-profile`)
+- [ ] `--attach` flag (default false) for interactive TTY (flips old `up` default)
+- [ ] `--dry-run` — resolve everything, report pass/fail per step, don't deploy
+- [ ] `-o yaml` — output fully resolved harness YAML with source annotations
+- [ ] `-o json` — machine-readable resolved config
+- [ ] Env var fallbacks: `OPENSHELL_GATEWAY`, `OPENSHELL_GATEWAY_ENDPOINT` (#1851)
+- [ ] Bare `harness apply` uses default agent config (same as old `harness up`)
+- [ ] Mark `up` and `create` as hidden deprecated aliases via cobra `Deprecated` field
+
+### Phase 2: `get` and `describe` commands (replaces `status`)
+- [ ] `cmd/get.go` — parent command with subcommands:
+  - `harness get agents` (aliases: `sandboxes`) — list running sandboxes
+  - `harness get providers` — list registered providers
+  - `harness get gateways` — list gateways
+- [ ] Shared `OutputFormat` type: `table|json|yaml` via `-o` flag on all `get` subcommands
+- [ ] Credential exclusion: `-o json/yaml` never includes secret values (#1830 pattern)
+- [ ] `cmd/describe.go` — `harness describe <name>` for detailed sandbox status
+- [ ] Mark `status` as hidden deprecated alias
+
+### Phase 3: `delete` command (replaces `teardown`)
+- [ ] `cmd/delete.go` — targeted + bulk deletion:
+  - `harness delete <name>` — delete specific sandbox
+  - `harness delete --all` — full teardown (sandboxes + providers + k8s)
+  - `harness delete --providers` — providers only
+  - `harness delete --k8s` — k8s resources only
+- [ ] Reuses existing `teardownSandboxes()`, `teardownProviders()`, `teardownK8s()`
+- [ ] Mark `teardown` as hidden deprecated alias
+
+### Phase 4: Integration + docs
+- [ ] Update `test/test-flow.sh` to use new verbs
+- [ ] Update SPEC.md command reference
+- [ ] Update README.md command reference
+- [ ] `render` becomes hidden alias for `apply -o yaml`
+- [ ] `deploy` stays as-is (infrastructure action)
+- [ ] `start`/`stop` stay as-is (lifecycle actions)
 
 ## Agent Config
 
-### Self-contained agent YAML (multi-document, k8s-style)
-- [ ] Support multi-document YAML (`---` separated) where all objects live in one file
-- Agent still references everything by name (`profile: github`), but provider/gateway/policy
-  definitions can be co-located in the same file instead of separate files in profiles/
-- Parser reads all documents via `yaml.Decoder` loop, indexes by `kind`+`name`, resolves
-  references against local set first, falls back to profiles/ tree
-- Composes naturally: split the file up and drop objects into `profiles/` to share across agents
-- Example:
-  ```yaml
-  ---
-  kind: agent
-  name: my-agent
-  entrypoint: claude
-  gateway: local
-  providers:
-    - profile: github
-    - profile: vertex
-  env:
-    ANTHROPIC_BASE_URL: https://inference.local
-  ---
-  kind: provider
-  name: github
-  type: github
-  credentials: [GITHUB_TOKEN]
-  endpoints:
-    - { host: "api.github.com", port: 443 }
-  ---
-  kind: provider
-  name: vertex
-  type: google-vertex-ai
-  credentials: [GOOGLE_APPLICATION_CREDENTIALS]
-  ---
-  kind: gateway
-  name: local
-  type: local
-  insecure: true
-  ---
-  kind: policy
-  network_policies:
-    github:
-      endpoints:
-        - { host: "api.github.com", port: 443 }
-  ```
-- Goal: `harness up -f agent.yaml` with one file. Zero to working sandboxed agent.
-- Existing single-document agent YAMLs (no `kind` field) continue to work unchanged
-
-### `harness render` as live config snapshot
-- [ ] `harness render` queries the running gateway for effective state, not just YAML files
-- Outputs what is actually configured: registered providers, active gateway, inference
-  config, sandbox policy, env structure
-- Credentials replaced with `${VAR}` placeholders -- the snapshot is shareable
-- Replay with different creds: `GITHUB_TOKEN=theirs harness up -f snapshot.yaml`
-- Like `kubectl get -o yaml` -- captures the running shape, not the source config
-- Round-trip: `harness render > snapshot.yaml && harness up -f snapshot.yaml` should
-  reproduce the same agent setup (with different credentials from env)
-- [ ] `harness render preview -f harness.yaml` -- dry-run that resolves all references
-  (providers, gateway, env vars) and shows the fully resolved config without deploying.
-  Like `terraform plan` or `helm template`. Shows what `harness up` would do.
+### Multi-document harness YAML [DONE]
+- [x] `kind: agent/provider/gateway/policy` dispatch via `yaml.Decoder` loop
+- [x] `Harness` type with `ParseHarness`/`ParseHarnessFile`
+- [x] `RenderHarness` with built-in vs custom provider labeling
+- [x] Resolution: harness-local definitions > profiles/ tree > embedded defaults
+- [x] Backwards compat: single-doc agent YAMLs without `kind` still work
+
+### Config reconciliation (`apply -o yaml`)
+- [ ] Resolves agent YAML against profiles/, defaults, and running gateway
+- [ ] Shows where each value came from (default, profile, harness file, env var)
+- [ ] Credentials rendered as `${VAR}` placeholders — shareable, replayable
+- [ ] Round-trip: `apply -o yaml > snapshot.yaml && apply -f snapshot.yaml`
+- [ ] `--dry-run` without `-o` reports pass/fail (gateway available? providers
+      resolvable? image exists? env vars resolved?)
+
+### `kind: config` — embed sandbox files in harness YAML (future)
+- [ ] `kind: config` documents for `claude.json`, `CLAUDE.md`, `mcp.json`, etc.
+- [ ] Rendered to payload directory instead of baking into sandbox image
+- [ ] Keeps sandbox image minimal — all agent-specific config in the harness YAML
+
+### Provider abstraction layer
+- [ ] `kind: provider` targets `openshell provider create` today (imperative)
+- [ ] Abstraction supports future backends: gateway.toml (#1886), K8s CRDs (#1719)
+- [ ] Do not hard-code execution strategy — upstream is undecided
 
 ### Future fields
-- [ ] `description` -- one line of human-readable context per agent config
-- [ ] `repo` -- git URL to clone into the sandbox at start
-- [ ] `secrets` -- non-provider secrets to inject
+- [ ] `description` — one line of human-readable context per agent config
+- [ ] `repo` — git URL to clone into the sandbox at start
+- [ ] `secrets` — non-provider secrets to inject
+
+## Upstream alignment
+
+### Plugin compatibility (#1851)
+- [ ] Binary naming: plan for `openshell-harness` (PATH-based plugin discovery)
+- [ ] Dual invocation: standalone `openshell-harness` and plugin `openshell harness`
+- [ ] Env var fallbacks for gateway/endpoint/verbosity when running as plugin
+- [ ] Auth via `openshell-bootstrap` APIs (no token forwarding)
+- [ ] Status: #1851 is `question` label, not accepted. Design standalone first.
+
+### Image building delegation
+- [ ] Evaluate delegating to `openshell-image-builder` for advanced image composition
+- [ ] Harness generates config (`.kaiden/workspace.json` + `config.toml`), builder consumes
+- [ ] Layered policy composition: base + agent-specific + user overlay
+- [ ] Programmatic Containerfile generation from config (stop maintaining static Dockerfiles)
+
+### Upstream issues to track
+- #1719 — K8s Operator design (affects provider CRDs, declarative config)
+- #1851 — Plugin system (affects binary naming, env var contract)
+- #1886 — Declarative provider config in gateway.toml (affects `kind: provider`)
+- #1922 — Portable sandbox log collection (affects observability)
+- #1933 — Centralized audit/event log (affects run recorder)
 
 ## Testing