feat(harness): @fro.bot/harness — patched-OpenCode build + publish pipeline#752
Merged
Conversation
Add the @fro.bot/harness published package: a forwarding CLI that wraps a patched OpenCode binary. v1 scaffold provides the passthrough + provenance commands (info/patches/doctor) and a host-binary resolver, with placeholder provenance until the integration engine and native build land. The CLI passes any non-reserved command through to the resolved opencode binary (stdio + exit code preserved); info/patches/doctor are harness-owned. Published dot-scope (@fro.bot/harness) with a bin entry and public publish config — the first publishable package in the workspace. Build output is gitignored like the other workspace packages; the npm tarball ships dist via the files field at publish time.
…e tag) Add the integration engine that carries non-mainline OpenCode refs onto a pinned release. sources.ts maps each configured ref (GitHub PR URL, branch URL, or local branch) to its git fetch refs; integrate.ts clones the base at the release tag, fetches the refs, runs an LLM merge to resolve them onto the tag, builds the native CLI, verifies the version, and freezes the integration commit plus a provenance manifest. Every step fails hard and freezes nothing on a merge/build/version failure; the merge and build steps are dependency- injected so the orchestration contract is unit-tested without a live merge. harness.config.json carries pull/30182 as the sole integration ref (a merged-to-dev Anthropic thinking-block fix not yet in 1.15.13). prompt.txt adapts the merge instruction with non-interactive CI guardrails. The base release pin is a tracked constant. info/patches now report the real manifest.
Add the build/distribution layer. build-platform.ts checks out the full upstream repo at the frozen integration commit, pins the upstream Bun version, runs upstream's real build (embedded app + native deps), and emits a native binary per target, verifying the version before emit. verify-binary.ts is the publish gate (version, integration marker, boot smoke); its assertion logic is unit-tested against stubs. platform.ts maps host os/arch to the per-platform package name; resolve-binary now finds the host binary by that computed name with a PATH fallback for local dev. harness-release.yaml builds each platform on its own native runner, assembles the main + per-platform packages, and publishes all-or-nothing to npm with provenance attestation — dispatch- and tag-gated, never on PRs. Per-platform optionalDependencies are injected into the published manifest at publish time rather than listed in source, so the workspace frozen-lockfile install stays clean. Harness scripts are TypeScript, run via bun.
Switch the harness release workflow from an NPM_TOKEN secret to npm trusted publishing over OIDC: drop the token env, upgrade npm to an OIDC-capable version, and publish with a bare npm publish (provenance is automatic; access comes from publishConfig). The workflow already carries id-token: write. Document the one-time per-package npmjs.com trusted-publisher setup (fro-bot org, agent repo, harness-release.yaml workflow) and the first-publish bootstrap, since trusted publishing requires a package to already exist before it accepts OIDC publishes.
Address review findings on the harness package: - Type-check and lint now cover scripts/ (was src/ only), so the build scripts are checked. - Scope npm trusted-publishing OIDC (id-token: write) to the publish job only; the build job that runs the LLM merge + upstream build is read-only and cannot obtain a publish token. Pass tag/input-derived version and commit values through env vars (not string-interpolated into node -e/shell) and validate them; pin npm to 11.5.1. - Resolve the per-platform binary via Node module resolution so pnpm/npm hoisting works, and fail closed when the platform binary is absent (PATH fallback only behind an explicit dev escape hatch) — the harness is the default OpenCode, so its absence is an error, not a silent stock fallback. - Embed the integration commit into the built binary's version (<base>+harness.<sha>) so it self-reports provenance; verify that structured marker (not a loose substring) and compute the expected version from one shared helper. Emit the provenance manifest the release flow consumes. - Replace the platform-error class with a discriminated result (functions only), validate parsed provenance/config JSON, make doctor fail on a dev-fallback or version-mismatched binary, tighten strict-boolean checks, route invalid integration refs through the fail-hard path, and de-duplicate the provenance manifest shape.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
fro-bot
requested changes
Jun 4, 2026
| * @param probeOutput - Combined output from the binary probe (--version + info). | ||
| * @param integrationCommit - The frozen integration commit SHA to look for, or null/empty. | ||
| */ | ||
| export function assertIntegrationMarker(probeOutput: string, integrationCommit: string | null): VerifyResult { |
Owner
There was a problem hiding this comment.
Blocking (pipeline correctness): assertIntegrationMarker requires the full integrationCommit SHA on a structured integration commit: <sha> line in the binary probe output. But the built binary is a stock upstream opencode (built from anomalyco/opencode), which has no info subcommand and never emits formatProvenance output. In verify-binary.ts, probe 2 runs binary info on that stock binary; it cannot produce this line. So whenever --integration-commit is passed (i.e. every real release), this assertion fails and the publish is blocked.
Additionally, the only commit identity actually baked into the binary is the short 8-char SHA via OPENCODE_VERSION=<base>+harness.<short8> (see version.ts), whereas this check looks for the full SHA. Even if info worked, the lengths wouldn't match.
Recommend either: (a) assert the marker against the --version output +harness.<short8> substring instead of an info line, or (b) embed a real provenance line into the upstream build. The check should be reconciled with what the built binary can actually report.
|
|
||
| // Probe 2: info (for integration marker — harness-own subcommand) | ||
| // Only attempt if the --version probe succeeded. | ||
| if (exitCode === 0) { |
Owner
There was a problem hiding this comment.
The info probe swallows all failures silently (empty catch). Since the stock opencode binary has no info subcommand, probeOutput will never contain the structured marker line, so assertIntegrationMarker fails for any release with --integration-commit. This is the runtime side of the same correctness gap flagged in verify.ts.
| bun-version: 1.3.13 | ||
|
|
||
| - name: Install harness dependencies | ||
| run: pnpm install --filter @fro.bot/harness --frozen-lockfile |
Owner
There was a problem hiding this comment.
Non-blocking: this step runs pnpm install but no pnpm setup step (e.g. pnpm/action-setup or corepack enable) appears before it in the build job. Confirm pnpm is on PATH on the matrix runners (ubuntu/macos images), otherwise the build job fails at install. Same applies to the publish job (line ~204).
| const manifestPath = path.join(dir, MANIFEST_FILENAME) | ||
| try { | ||
| const raw = await fs.readFile(manifestPath, 'utf8') | ||
| return JSON.parse(raw) as ProvenanceManifest |
Owner
There was a problem hiding this comment.
Non-blocking: readProvenanceManifest casts JSON.parse(raw) as ProvenanceManifest with no shape validation, unlike provenance.ts which uses an isValidProvenance type guard. A malformed/partial manifest would be returned as valid. Consider reusing the same validation guard here for consistency with the fail-hard contract.
Owner
Verdict: CONDITIONALCONDITIONAL = can merge after addressing the listed blocking issue. The package design is solid: clean fail-hard contract in the integration engine, DI-based testing, fail-closed binary resolution, scoped OIDC publishing (build job has no Blocking issues
Non-blocking concerns
Missing tests
Risk assessment (MED)
Run Summary
|
Address review on the publish pipeline: - The built artifact is stock upstream opencode, which has no info subcommand and never emits a provenance line, and only the short 8-char commit is baked into its version via OPENCODE_VERSION=<base>+harness.<short8>. Drop the dead info-subcommand probe and verify the +harness.<short8> marker in the binary's --version output instead of looking for a full SHA the binary never reports, so a real release can actually pass verification. A stock binary or a build of a different commit fails; the full provenance lives in the package manifest plus npm attestation. - Make parseArgs return null on missing args (instead of exiting itself) so the null guard in main() is reachable, clearing two CodeQL unneeded-defensive-code alerts. - Set up pnpm in both the build and publish jobs before pnpm install. - Validate the provenance manifest shape on read instead of an unchecked cast.
fro-bot
approved these changes
Jun 4, 2026
Owner
fro-bot
left a comment
fro-bot
left a comment
There was a problem hiding this comment.
Verdict: PASS
Re-review after 43be0af ("reconcile binary verification with what the binary reports"). The prior blocking issue and all three non-blocking concerns are fully resolved. Verified the design and tests directly:
- Binary verification (prior blocker) — fixed.
assertIntegrationMarkernow checks for+harness.<short8>in the binary's--versionoutput (verify.ts:64), matching whatbuildHarnessVersionactually bakes into the build. The deadbinary infoprobe is gone (verify-binary.ts:110), andassertVersionMatchenforces the full<base>+harness.<short8>string so a stock binary or a build of a different commit fails. A real release can now pass verification, and the verdict-vs-reality gap is closed. - Test coverage — added.
verify.test.ts:68-140now covers the contract end to end: correct harness build passes, stock bare-version fails, different-commit short8 fails, and an unstructured SHA substring fails.runVerificationscombined cases are covered too. - pnpm setup — fixed. Both build and publish jobs now run
./.github/actions/setupbeforepnpm install(harness-release.yaml:78,:197). The secondsetup-nodein the publish job (:202) is intentional — it configures the npm registry/OIDC for trusted publishing, distinct from the pnpm setup. - Manifest validation — fixed.
readProvenanceManifestnow validates shape viaisValidProvenanceManifest(integrate.ts:105-132) instead of an unchecked cast, consistent with the fail-hard contract. - CodeQL defensive-code — addressed.
parseArgsnow returnsnullon missing args so themain()null guard is reachable.
Blocking issues
None
Non-blocking concerns
build-platform.tsresolveCliPath/resolveBuiltBinaryPathstill keepwindows/.exebranches even though the matrix andplatform.tsexclude Windows. Harmless dead code; clean up opportunistically on the next touch.
Missing tests
None blocking. Optional: a focused unit test for readProvenanceManifest returning null on malformed/partial JSON would lock in the new validation guard, but the guard logic mirrors the already-tested isValidProvenance in provenance.ts.
Risk assessment (LOW)
- Regression likelihood: LOW — the verification logic is now exercised by unit tests and reconciled with the binary's actual self-report. The release workflow remains dispatch/tag-gated and never runs on PRs.
- Security exposure: LOW — OIDC publishing stays scoped to the maintainer-gated
publishjob; the untrusted build job is read-only with noid-token. Shell-interpolated inputs are validated against strict semver/SHA regexes. - Blast radius: LOW — net-new package + isolated workflow; the action does not yet consume the harness, so no existing runtime path changes.
Run Summary
| Field | Value |
|---|---|
| Event | pull_request |
| Repository | fro-bot/agent |
| Run ID | 26935129059 |
| Cache | hit |
| Session | ses_16eb8f4c5ffePCAm40m8c8xA4u |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
@fro.bot/harness: a published package that ships a patched build of upstream OpenCode behind a forwarding CLI, distributed as native per-platform binaries. It gives Fro Bot durable, reproducible control over a small set of non-mainline OpenCode changes while tracking a deliberately-pinned recent release.What's here
The package (
packages/harness/)harnessCLI that passes any command through to the resolved OpenCode binary (stdio + exit code preserved), plusinfo/patches/doctorfor provenance and diagnostics.bunx @fro.bot/harness-runnable.optionalDependencieswith a host-resolvingpostinstall.Publishing — npm trusted publishing (OIDC): no long-lived token, automatic provenance, with OIDC scoped to a maintainer-gated publish job. The build job that runs the merge is read-only.
Initial carry set — one ref: anomalyco/opencode#30182 (preserve signed Anthropic thinking during reorder). It's merged upstream to
devbut not in the pinned release, so the merge carries it onto the tag; it drops automatically once a release includes it. A carry policy in the package AGENTS.md keeps the set small.Scope
This is the package + pipeline only. The action does not consume the harness binary yet — wiring the harness in as the default OpenCode is a deliberate follow-up after the package is published, so the action is never pointed at an unpublished package. The release workflow is dispatch/tag-gated and does not run on PRs.
Notes for review
harness-release.yamlwon't run here (gated); it builds, verifies, and publishes only on deliberate dispatch.packages/harness/AGENTS.md), since trusted publishing needs a package to exist before it accepts OIDC publishes.