[B4] ci(bench): run benchmarks on pull_request + push results to Bencher for trend tracking

[MichaelFisher1997]

Part of #834 (Production-Readiness Audit) — Batch B4 (make bench/visual actually fail, parallel-safe).
Replaces audit items #3 + #19 (merged because both touch benchmark.yml). Parallel with: audit-b4 siblings — this issue owns benchmark.yml.

Problem

Two related gaps sharing one file, so they ship as one PR:

1. feat: voxel engine with worldgen, async streaming, and render stability #3 — bench doesn't run on PRs. benchmark.yml triggers on push: dev + workflow_dispatch only. Performance regressions land on dev and are detected after merge, when bisecting is harder.
2. 🔍 Conduct Comprehensive Code Audit - Architecture, Performance, SOLID, and Quality Standards #19 — no historical trend store. Each benchmark run overwrites the last. There's no Bencher / Conbench history, so month-over-month drift is invisible.

Acceptance criteria

• pull_request: trigger added to .github/workflows/benchmark.yml, gated by a path filter (src/**, modules/**, assets/shaders/**, build.zig) or a run-benchmark label to control cost. Document the chosen gate.
• Benchmark results pushed to Bencher (or equivalent OSS trend store) for historical tracking.
• Bencher project + API token configured (repo secret).
• Trend dashboard link posted as a PR comment on benchmark-triggered runs.
• Nightly scheduled run also pushed to Bencher for stable long-term baseline.

Files in scope

• .github/workflows/benchmark.yml
• Bencher config (bencher.yml or similar) if needed

Dependencies

• After: B1-3 ([B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861, real baseline.json) — Bencher baseline should be consistent with the committed baseline.
• Coordinate with B4-3 (SLO + draw-call ceilings) which touches the benchmark harness source (not the workflow) — parallel-safe.

Out of scope

• SLO absolute thresholds (B4-3 / audit feat: worm cave system with connected underground networks #4).
• Golden-image diff (B4-2 / audit Refactor: Split main.zig into modular game components #17 — different workflow).

Branch / PR

• Branch: ci/bench-pr-trigger-bencher · Targets: dev

[github-actions]

Triage: B4-1 — bench PR trigger + Bencher trend store

Classification: Feature request (CI workflow + trend-store integration). Well-scoped to a single workflow file plus an optional bencher.yml. No code-path changes outside .github/workflows/benchmark.yml. Safe to ship as one PR.

Current state of benchmark.yml

• Trigger block lives at .github/workflows/benchmark.yml:3-6 — push: dev + workflow_dispatch only. pull_request: and schedule: are absent.
• Permissions at .github/workflows/benchmark.yml:24-26 are contents: read + statuses: write. A PR comment step will need pull-requests: write added.
• No existing bencher.yml / bencher.toml in the repo — needs to be authored.
• No existing PR for this branch (ci/bench-pr-trigger-bencher) in the open/merged list.

Implementation pointers (for the eventual PR)

1. Trigger block — extend .github/workflows/benchmark.yml:3-6 to add pull_request: (branches: [dev]) and schedule: (e.g. cron: '0 6 * * *'). Concurrency group at lines 13-15 should likely key on PR number for pull_request runs so a push+PR don't fight.
2. Path filter — pull_request.paths: with src/**, modules/**, assets/shaders/**, build.zig matches the acceptance criteria. Optionally layer a run-benchmark label gate via if: github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'run-benchmark') — see design question below.
3. Bencher step — insert after the "Compare against baseline" step (line 64). The bencherdev/bencher@main composite action is the canonical entry point; it can wrap the existing run-with-log invocation or consume the JSON output via gh actions/cache of benchmark-results/.
4. PR comment step — needs pull-requests: write permission. Post the Bencher dashboard URL (project + branch + testbed) using actions/github-script@v9, gated if: github.event_name == 'pull_request'.
5. Repo secret — BENCHER_API_TOKEN (and optionally BENCHER_HOST if self-hosted). Document the setup in docs/benchmarks/README.md (which is the same file [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861 is extending — coordinate so it lands consistently).

Design questions to resolve before implementation

• Gate choice: the AC says "path filter or run-benchmark label". Recommendation: both, AND'd — path filter as the cheap default, label as the opt-out override so perf-unrelated PRs (docs, CI-only) don't auto-trigger. Confirm or push back?
• Bencher host: Bencher Cloud (free OSS plan, project slug <org>/zigcraft) or self-hosted? This affects the secret surface and the dashboard URL format.
• Baseline pinning strategy: Bencher has its own baseline concept (--start-point / branches). Should dev be the Bencher baseline branch, or should it mirror docs/benchmarks/baseline.json from [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861? The latter keeps a single source of truth; recommend mirroring [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861's baseline file.
• Nightly schedule cost: blacksmith-2vcpu-ubuntu-2404 × 90 min × nightly is meaningful CI spend. Confirm budget is OK before enabling the cron, or start with weekly.

Coordination notes (parallel-safe as stated)

• [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861 (B1-3, real baseline.json) — soft dep: Bencher will work against placeholder data, but the trend won't be meaningful until [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861 lands. Acceptable to ship this PR first; the baseline can be re-captured once [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861's numbers are committed.
• [B4] bench: add absolute p1-FPS / spike / draw-call / vertex / GPU-memory ceilings per preset #875 (B4-3, SLO ceilings) — touches benchmark harness source only, no workflow conflict. Once both land, PR-triggered bench runs will actually fail on SLO breach (the workflow change is what wires it up).
• [B4] ci(visual): nightly/label trigger + deterministic golden-image diff as primary gate #874 (B4-2, visual-test nightly + golden diff) — different workflow file (visual-test.yml), no conflict.

Suggestion

Scope is appropriate as-is. No need to split into sub-issues — the file boundary (benchmark.yml + bencher.yml) is clean and the AC list is testable end-to-end in a single PR. Recommend landing after #861 merges so Bencher starts ingesting real numbers from run 1.

[github-actions]

Posted triage comment on #873: #873 (comment)

Classification: Feature request (CI workflow change), well-scoped to a single file.

Key findings from the audit:

• benchmark.yml:3-6 is the only trigger block — no pull_request: or schedule:.
• Permissions at lines 24-26 need pull-requests: write added for the dashboard-link comment.
• No existing bencher.yml — needs authoring.
• No duplicate PRs for branch ci/bench-pr-trigger-bencher.

Design questions raised (so author can answer before implementing):

1. Path filter + label gate (AND) vs either-alone?
2. Bencher Cloud vs self-hosted (affects BENCHER_API_TOKEN + dashboard URL format)?
3. Bencher baseline branch mirrors dev, or mirrors [B1] ci(bench): populate docs/benchmarks/baseline.json and enable regression gating #861's docs/benchmarks/baseline.json?
4. Nightly cron cost on blacksmith-2vcpu — weekly first?

Coordination confirmed parallel-safe with #875 (harness source) and #874 (visual-test.yml). Soft dep on #861 noted; recommend landing after #861 so Bencher ingests real numbers from run 1.

opencode session  |  github run

[MichaelFisher1997]

Completed by merged PR #888; follow-up runtime CI stabilization landed in #891.