From 9a86fa909e5bb7d3b907165b320e192045f11217 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 21:38:14 +0900 Subject: [PATCH 01/69] docs: define automation control-plane architecture --- .../workflows/pr-review-merge-scheduler.yml | 2 +- .github/workflows/strix.yml | 8 +- AGENTS.md | 5 + ARCHITECTURE.md | 17 + CHANGELOG.md | 5 + CLAUDE.md | 17 +- PR_GOVERNANCE_AUDIT.md | 20 +- README.md | 38 +- docs/CWL-MASTER-CONTEXT.md | 7 +- docs/automation/ARCHITECTURE.md | 98 +++++ docs/automation/DATA_MODEL.md | 356 ++++++++++++++++++ docs/automation/ERD.md | 29 ++ docs/automation/INCIDENT_RUNBOOK.md | 110 ++++++ docs/automation/OPERABILITY.md | 87 +++++ docs/automation/PRD.md | 112 ++++++ docs/automation/README.md | 48 +++ docs/automation/RUNBOOK.md | 337 +++++++++++++++++ docs/automation/SECURITY.md | 99 +++++ docs/automation/TEST_STRATEGY.md | 111 ++++++ docs/automation/THREAT_MODEL.md | 189 ++++++++++ docs/automation/TRACEABILITY.md | 65 ++++ docs/automation/TRD.md | 230 +++++++++++ docs/automation/UML.md | 215 +++++++++++ ...ranch-writer-leases-and-read-only-audit.md | 57 +++ ...0002-exact-source-and-live-base-binding.md | 57 +++ .../adr/0003-classified-bounded-retries.md | 57 +++ .../adr/0004-explicit-secret-contracts.md | 57 +++ .../adr/0005-independent-review-authority.md | 56 +++ ...6-protected-main-operational-acceptance.md | 57 +++ .../adr/0007-work-conserving-maintenance.md | 57 +++ ...entral-control-plane-and-thin-consumers.md | 58 +++ ...009-sandbox-evidence-redaction-boundary.md | 69 ++++ ...-mention-routing-and-idempotency-ledger.md | 86 +++++ ...ovider-routing-and-credential-isolation.md | 83 ++++ ...olchains-and-exact-base-materialization.md | 78 ++++ ...-autofix-and-merge-authority-separation.md | 84 +++++ ...adata-event-and-default-branch-dispatch.md | 81 ++++ ...0015-direct-or-auto-merge-state-machine.md | 79 ++++ ...6-fail-closed-security-gate-composition.md | 82 ++++ docs/automation/adr/README.md | 28 ++ .../automation-control-plane-standards.md | 91 +++++ docs/org-required-workflow-rollout.md | 9 +- docs/scorecard-governance.md | 11 +- .../test_automation_documentation_contract.py | 272 +++++++++++++ tests/test_pr_governance_audit_contract.py | 12 + 45 files changed, 3692 insertions(+), 34 deletions(-) create mode 100644 ARCHITECTURE.md create mode 100644 docs/automation/ARCHITECTURE.md create mode 100644 docs/automation/DATA_MODEL.md create mode 100644 docs/automation/ERD.md create mode 100644 docs/automation/INCIDENT_RUNBOOK.md create mode 100644 docs/automation/OPERABILITY.md create mode 100644 docs/automation/PRD.md create mode 100644 docs/automation/README.md create mode 100644 docs/automation/RUNBOOK.md create mode 100644 docs/automation/SECURITY.md create mode 100644 docs/automation/TEST_STRATEGY.md create mode 100644 docs/automation/THREAT_MODEL.md create mode 100644 docs/automation/TRACEABILITY.md create mode 100644 docs/automation/TRD.md create mode 100644 docs/automation/UML.md create mode 100644 docs/automation/adr/0001-branch-writer-leases-and-read-only-audit.md create mode 100644 docs/automation/adr/0002-exact-source-and-live-base-binding.md create mode 100644 docs/automation/adr/0003-classified-bounded-retries.md create mode 100644 docs/automation/adr/0004-explicit-secret-contracts.md create mode 100644 docs/automation/adr/0005-independent-review-authority.md create mode 100644 docs/automation/adr/0006-protected-main-operational-acceptance.md create mode 100644 docs/automation/adr/0007-work-conserving-maintenance.md create mode 100644 docs/automation/adr/0008-central-control-plane-and-thin-consumers.md create mode 100644 docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md create mode 100644 docs/automation/adr/0010-agent-mention-routing-and-idempotency-ledger.md create mode 100644 docs/automation/adr/0011-provider-routing-and-credential-isolation.md create mode 100644 docs/automation/adr/0012-hash-pinned-toolchains-and-exact-base-materialization.md create mode 100644 docs/automation/adr/0013-autofix-and-merge-authority-separation.md create mode 100644 docs/automation/adr/0014-trusted-metadata-event-and-default-branch-dispatch.md create mode 100644 docs/automation/adr/0015-direct-or-auto-merge-state-machine.md create mode 100644 docs/automation/adr/0016-fail-closed-security-gate-composition.md create mode 100644 docs/automation/adr/README.md create mode 100644 docs/doctoring/automation-control-plane-standards.md create mode 100644 tests/test_automation_documentation_contract.py diff --git a/.github/workflows/pr-review-merge-scheduler.yml b/.github/workflows/pr-review-merge-scheduler.yml index 8e1157060..7c4a14a5b 100644 --- a/.github/workflows/pr-review-merge-scheduler.yml +++ b/.github/workflows/pr-review-merge-scheduler.yml @@ -553,7 +553,7 @@ jobs: # published after the scheduler pass, required merge-preview checks landing # late, a base-branch policy blocker clearing) stays approved-but-unmerged # with no later trigger. This job re-runs the same trusted scheduler against - # every organization repository on an hourly heartbeat so each such PR is + # every organization repository on a 15-minute heartbeat so each such PR is # merged, branch-updated, or leaves a concrete per-PR blocker reason in this # log. It never bypasses policy: all mutations go through the same guarded # scheduler contract as the per-repository runs. diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml index 03ec23257..60294905b 100644 --- a/.github/workflows/strix.yml +++ b/.github/workflows/strix.yml @@ -38,10 +38,10 @@ on: # path filters against the PR's full base..head diff, so a PR is skipped only # when EVERY changed file is a non-executable doc/image asset; any code, # config, build, or workflow change still triggers the scan. Concurrency is - # PR-number based for status grouping, but Strix runs intentionally do not - # cancel in progress because a pre-job cancellation leaves no scanner log to - # review. Queue pressure should be handled by stale-run cleanup outside this - # current-head evidence path. For PRs the merge scheduler manages, same-head + # event-class/repository/PR-number scoped and cancels obsolete work within + # that class. Separate event classes keep a default-branch dispatch from + # cancelling the required pull_request_target context. For PRs the merge + # scheduler manages, same-head # Strix evidence is still forced at merge time via repository_dispatch (which # paths-ignore does not affect), so merged code never loses evidence. paths-ignore: diff --git a/AGENTS.md b/AGENTS.md index 688b33035..c7e02404b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,3 +2,8 @@ > **Agents: read the master context FIRST.** Before any work, read [`docs/CWL-MASTER-CONTEXT.md`](docs/CWL-MASTER-CONTEXT.md) (mission · naruon-as-platform + inter-component UML · cross-cutting disciplines · conventions · roadmap · current state), the live **GitHub Project #1** (work/roadmap source of truth), the full spec **ContextualWisdomLab/naruon#974**, and operate the Project per [`docs/agent-github-project-protocol.md`](docs/agent-github-project-protocol.md). The repo/Project — not any private agent memory — is the source of truth. + +For central review, security, merge, fix, sandbox, or autonomous-maintenance work, +also read the authoritative [automation documentation graph](docs/automation/README.md) +and update its traceability/ADR entries whenever an authority, trust, failure, +secret, evidence, compatibility, or operational-acceptance boundary changes. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..9f2f32420 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,17 @@ +# Architecture index + +This repository is both the ContextualWisdomLab organization profile and the +shared automation control plane. + +- The ecosystem and naruon platform context live in + [docs/CWL-MASTER-CONTEXT.md](docs/CWL-MASTER-CONTEXT.md). +- The central automation architecture, product and technical requirements, + component/sequence/state/deployment UML, conceptual evidence ERD, trust + boundaries, failure domains, operations, and decisions start at + [docs/automation/README.md](docs/automation/README.md). +- Live repository-by-repository governance observations remain in + [PR_GOVERNANCE_AUDIT.md](PR_GOVERNANCE_AUDIT.md). + +The durable boundary is central shared policy and trusted execution with thin, +versioned consumers. Product repositories remain independently operable and own +their domain code, data, tests, releases, deployments, and migrations. diff --git a/CHANGELOG.md b/CHANGELOG.md index bf30091dd..ff8341881 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,11 @@ Semantic Versioning where the repository publishes a release. ### Added +- Added an authoritative automation documentation graph covering PRD, TRD, + architecture, conceptual ERD, UML behavior views, security and threat model, + test strategy, operability, incident response, requirement traceability, sixteen + detailed ADRs, current standards doctoring, and machine-checkable index/source + contracts. - Added a trusted pull-request comment router for `@cwl-noema-review` and review-only `@opencode-agent` dispatches, with an organization sweep, exact-head receipts, repository allowlisting, fixed runners, immutable checkout pins, and a permanent 100% statement/branch/docstring quality gate. - Added exact-base `uv.lock` materialization that reconstructs standalone nested projects with a checksum-pinned official `uv` exporter, isolated frozen/offline execution, strict exact-pin and SHA-256 output validation, and complete Python 3.10/3.14 quality evidence. diff --git a/CLAUDE.md b/CLAUDE.md index 1c7bdb2f6..26f527893 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,20 +21,22 @@ This is the ContextualWisdomLab **organization-wide `.github` special repository `profile/assets/`. Org-wide defaults `SECURITY.md`, `.github/CODEOWNERS`, and `.github/dependabot.yml` also live here. 2. **Central PR governance and CI hub** — the workflows in `.github/workflows/` are the canonical - implementation of PR review, security scanning, and merge automation for **every sibling repo**. + implementation of PR review, security scanning, and merge automation for enrolled sibling repos. An organization required-workflow ruleset (`CWL Central required workflows`, id `18156473`) runs Strix, OpenCode Review, and the PR Review Merge Scheduler from this repo in each target repository's context. Repository-local copies of these workflows are drift sources, not repo-specific contracts. See `README.md` (policy summary) and `PR_GOVERNANCE_AUDIT.md` - (live audit + per-repo DX/UX transfer decisions). + (historical snapshot + per-repo DX/UX transfer decisions). Durable current contracts start at + [`docs/automation/README.md`](docs/automation/README.md); fetch GitHub live state before acting. 3. **Infrastructure as code** — `infra/cloudflare/` manages the org's DNS zones and Cloudflare Pages hosting declaratively (`zones.json` + `reconcile.sh`, curl + jq only; dry-run by default, writes only on explicit manual `mode = apply`). ## Governance model in one paragraph -**OpenCode judges PRs; GitHub Actions performs mechanical updates and merges.** OpenCode approval is -evidence-gated (changed files, CodeGraph evidence, Change Flow DAG, test/coverage/docstring evidence, +**OpenCode, Noema, Strix, and deterministic checks produce separate evidence; GitHub rulesets and +qualifying reviewers authorize, and GitHub Actions performs guarded mechanical updates and merges.** +OpenCode approval is evidence-gated (changed files, CodeGraph evidence, Change Flow DAG, test/coverage/docstring evidence, an actually-executed PoC via `scripts/ci/sandboxed_verify.py` or `scripts/ci/sandboxed_web_e2e.py`, split `Developer experience:` / `User experience:` sections). The scheduler updates a PR branch only when the latest review is approved, no current-head check has failed, and GitHub reports the PR as @@ -57,14 +59,17 @@ Details: `README.md` and `PR_GOVERNANCE_AUDIT.md`. - `scripts/ci/` — Python/bash helpers the workflows execute (schedulers, review normalization and gates, sandboxed verification, prompt template rendering). `tests/` covers them. - `opencode.jsonc` + `ci-review-prompt.md` + `code-reviewer-prompt.md` — the OpenCode reviewer - configuration (GitHub Models provider, CodeGraph/DeepWiki/Context7/web-search MCP). All reviewer + configuration (NVIDIA NIM first, with a bounded GitHub Models fallback pool and + CodeGraph/DeepWiki/Context7/web-search MCP). All reviewer agents have `"edit": "deny"`: they are reviewers, never implementers. Keep it that way. - `requirements-{bandit,pip-audit,strix,opencode-review}-ci.txt` + `*-hashes.txt` — pinned CI dependency sets (see below). - `fuzz/` + `.clusterfuzzlite/` — Atheris fuzz targets for the review-output normalizer and the ClusterFuzzLite discovery marker. - `docs/` — master context, Project protocol, `org-required-workflow-rollout.md`, - `scorecard-governance.md`, SBOM inventory. + `scorecard-governance.md`, SBOM inventory, and the authoritative + [`docs/automation/`](docs/automation/README.md) PRD/TRD/architecture/UML/ERD, + security, test, operations, traceability, and ADR graph. - `.jules/` — recorded performance (`bolt.md`) and security (`sentinel.md`) learnings from past work on `scripts/ci/`; worth scanning before optimizing or hardening those scripts. diff --git a/PR_GOVERNANCE_AUDIT.md b/PR_GOVERNANCE_AUDIT.md index bc0e31a6a..78738ca89 100644 --- a/PR_GOVERNANCE_AUDIT.md +++ b/PR_GOVERNANCE_AUDIT.md @@ -1,8 +1,22 @@ -# PR Governance Audit +# PR Governance Audit — historical snapshot -Live check: 2026-06-26 17:53 KST, GitHub API via `gh` as `seonghobae`. +> **Historical evidence, not current authority.** Observed 2026-06-26 17:53 +> KST through the GitHub API. Repository state, rulesets, workflows, providers, +> review counts, merge modes, and open PRs have changed since this capture. +> Re-fetch live GitHub state before acting. Durable current contracts start at +> [`docs/automation/README.md`](docs/automation/README.md); this file preserves +> the dated fleet observation and decision history for auditability. -## Canonical Policy +Snapshot check: 2026-06-26 17:53 KST, GitHub API via `gh` as `seonghobae`. + +- Snapshot status: historical and stale for mutation decisions +- Freshness policy: re-fetch after 24 hours or any relevant repository/ruleset event +- Regeneration input: live organization repositories, rulesets, PR heads, + checks, reviews, threads, workflow sources, and permissions +- Receipt limitation: the original raw API bundle was not retained; the dated + observations below are evidence, not a reproducible current-state digest + +## Policy recorded at snapshot time OpenCode decides; GitHub Actions mutates. diff --git a/README.md b/README.md index 332324301..120f1ac8a 100644 --- a/README.md +++ b/README.md @@ -6,19 +6,28 @@ The public GitHub organization profile lives in [profile/README.md](profile/READ Homepage: https://contextualwisdomlab.github.io/ -PR governance live audit: [PR_GOVERNANCE_AUDIT.md](PR_GOVERNANCE_AUDIT.md). -The audit includes repository-by-repository DX/UX transfer decisions: what the -central workflow borrows because it reduces friction, and what it rejects -because it adds noise or misleading review experience. +Historical PR-governance snapshot: [PR_GOVERNANCE_AUDIT.md](PR_GOVERNANCE_AUDIT.md). +It records repository-by-repository DX/UX observations as of its stated date; +live decisions must re-fetch GitHub state and use the durable contracts below. + +The authoritative product, technical, architecture, UML, conceptual ERD, +security, testing, operability, traceability, and ADR graph for the organization +automation control plane starts at +[docs/automation/README.md](docs/automation/README.md). Historical audits and +rollout ledgers are evidence, while the automation documents define durable +intended contracts. ## PR review and merge policy -OpenCode judges PRs; GitHub Actions performs mechanical updates and merges. -The scheduler updates a same-repository PR branch only when the latest OpenCode -review is approved, no current-head failed check is present, and GitHub reports -the PR as behind. After that update, the new head must pass OpenCode, Strix, -required checks, and review-thread gates again before auto-merge or -`--match-head-commit` merge can proceed. +OpenCode, Noema, Strix, and deterministic checks produce separate evidence; +GitHub rulesets and qualifying reviewers authorize, and GitHub Actions performs +only guarded mechanical updates and merges. The live organization contract +requires exactly two approving reviews, dismissal after a push, last-pusher +protection, resolved threads, and current-head required checks. The scheduler +updates a same-repository PR branch only when its exact-head evidence and live +policy allow that mutation. After any update, the new head must pass all review, +security, required-check, and thread gates again before auto-merge or an +expected-head direct merge can proceed. Branch updates and merges run through the central scheduler mutation credential: `PR_REVIEW_MERGE_TOKEN`, `OPENCODE_APPROVE_TOKEN`, the exchanged OpenCode GitHub App token, or finally the target workflow token. The scheduler reports the @@ -78,9 +87,12 @@ push, and commits as `github-actions[bot]` only when a conservative OpenCode autofix produces a validated diff. A repository-local autofix worker remains an explicit compatibility override through `--autofix-repository`; it is no longer the default contract. -Strix keeps `cancel-in-progress: false` so old evidence is not cancelled by a -force-push, but PR-scoped concurrency includes the head SHA so an obsolete scan -does not serialize newer current-head evidence. +Strix concurrency is scoped by event class, target repository, and PR number (or +ref for non-PR work), with `cancel-in-progress: true`. A newer event therefore +cancels obsolete work in its own class, while `repository_dispatch` and +`pull_request_target` remain separate so default-branch retry evidence cannot +cancel the required PR context. Only the completed current-head result is +authoritative. OpenCode approval is evidence-gated. Before approval, the review summary must name changed files, CodeGraph or structural MCP evidence, a Change Flow DAG, diff --git a/docs/CWL-MASTER-CONTEXT.md b/docs/CWL-MASTER-CONTEXT.md index bd5e6c0c4..cbd848ae5 100644 --- a/docs/CWL-MASTER-CONTEXT.md +++ b/docs/CWL-MASTER-CONTEXT.md @@ -2,7 +2,7 @@ > Purpose: a single, durable, agent-readable brief so ANY agent (Claude with a fresh context, Codex, Grok, Gemini) can reconstruct and continue this work WITHOUT the originating conversation. Private assistant memory is NOT the source of truth — this repo is. Keep this file current. > -> Durable sources of truth (in priority order): (1) **GitHub Project #1** "naruon Platform Roadmap" https://github.com/orgs/ContextualWisdomLab/projects/1 — live work/roadmap; (2) **naruon `docs/planning/naruon-platform-plan.md`** (PR ContextualWisdomLab/naruon#974) — full IA/User-Stories/Use-Cases/Architecture spec; (3) **`docs/agent-github-project-protocol.md`** (this repo, PR #363) — how agents operate the Project + cross-repo-ref convention; (4) this file. +> Durable sources of truth (in priority order): (1) **GitHub Project #1** "naruon Platform Roadmap" https://github.com/orgs/ContextualWisdomLab/projects/1 — live work/roadmap; (2) **naruon `docs/planning/naruon-platform-plan.md`** (PR ContextualWisdomLab/naruon#974) — full IA/User-Stories/Use-Cases/Architecture spec; (3) **`docs/agent-github-project-protocol.md`** (this repo, PR #363) — how agents operate the Project + cross-repo-ref convention; (4) **`docs/automation/README.md`** — the authoritative central automation PRD/TRD/architecture/UML/ERD/security/operations/ADR graph; (5) this ecosystem reconstruction brief. ## 0. Origin / core job-to-be-done (READ THIS — everything serves it) naruon's genesis (the user's own words): **"I can't find my emails, and the schedules that arrive by email keep changing so they're hard to track."** So the two founding jobs are: @@ -124,7 +124,10 @@ A **source-agnostic artifact-analysis service**: `submit(artifact, context) → ## 9. How work is tracked (dogfood the traceability) GitHub **Project #1** is the shared source of truth. Structure: real **Issues** (roadmap/backlog, in owning repos, custom fields Phase P0–P5/Ops/Decision + Component) and real **PRs** (delivered work, native Repository). Native workflows are ON (item added→Todo, PR merged→Done, item closed→Done). Chain: roadmap **Issue** → agent sets In Progress on pickup → implementing **PR** `Closes #N` → merge → auto Done. Operate the Project per `docs/agent-github-project-protocol.md`. Group by Phase / Component / Repository. -## 10. Current state (2026-07-08) +## 10. Historical state snapshot (2026-07-08 — not live work state) + +This section is retained as dated reconstruction evidence. Always use live GitHub +Project, repository, PR, issue, ruleset, and workflow state for current decisions. - Renames done (keyverse/wardnet/inkspan). Planning spec = naruon#974. Project #1 populated (68 issues + 60 PRs). Protocol = .github#363. - **BLOCKER B1**: org GitHub Actions effectively HALTED (~86 queued, ~0 in_progress org-wide) — likely the Actions monthly SPENDING CAP. Blocks ALL PR checks/merges + the Cloudflare DNS run (nameservers). Fix (org-admin): raise the Actions spending limit OR add a self-hosted runner. Nothing merges until then. - **Decisions pending**: (D1) Code Security enablement vs the CodeQL-only code_scanning ruleset (osv/trivy/scorecard SARIF upload) — a private repo needs GHAS seats; reconcile or make those checks non-required. (D2) trivy `limit-severities-for-sarif: true` (gate only CRITICAL/HIGH) — held pending the user's strict-security preference. diff --git a/docs/automation/ARCHITECTURE.md b/docs/automation/ARCHITECTURE.md new file mode 100644 index 000000000..f62ebb485 --- /dev/null +++ b/docs/automation/ARCHITECTURE.md @@ -0,0 +1,98 @@ +# Architecture — CWL automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 + +## 1. Architecture description approach + +This description follows stakeholder/viewpoint separation from ISO/IEC/IEEE 42010:2022. It documents context, component, information, security, execution, and operations viewpoints. Standards and primary sources are listed in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). + +## 2. Context and bounded responsibilities + +```mermaid +flowchart TB + Human["Maintainer / independent reviewer"] + GitHub["GitHub API, rulesets, checks, reviews"] + Central["CWL central automation control plane"] + Product["Independent product repositories"] + Provider["Review and security providers"] + + Human --> GitHub + GitHub --> Central + Central --> GitHub + Product --> Central + Central --> Provider + Provider --> Central + Central --> Product +``` + +The central control plane owns shared governance policy, trusted workflow source, dispatch validation, evidence normalization, and safe scheduling. GitHub owns repository state, rule enforcement, formal review identities, checks/status APIs, and merge primitives. Product repositories own product code, product data, domain tests, release/deployment, and thin central integrations. Providers supply advisory or configured machine evidence but do not acquire merge or human-review authority. + +## 3. Logical components + +```mermaid +flowchart TB + Entry["Trusted entry workflows"] + Identity["Identity and revision validator"] + Evidence["Evidence collectors and normalizers"] + Decision["Review / merge / fix schedulers"] + Sandbox["Sandbox and redaction boundary"] + + Entry --> Identity + Identity --> Evidence + Evidence --> Sandbox + Sandbox --> Decision + Decision --> Entry +``` + +| Component | Current implementation examples | Responsibility | +|---|---|---| +| Trusted entry workflows | `opencode-review.yml`, `opencode-review-dispatch.yml`, `noema-review.yml`, `strix.yml` | Materialize stable contexts; keep privileged execution on trusted default-branch source. | +| Identity validator | workflow metadata checks, `pr_head_replay_guard.py`, dispatch validators | Bind repository, PR, head, base, actor, workflow source, and run attempt. | +| Evidence collectors | `collect_failed_check_evidence.sh`, review context and normalizers | Gather bounded current evidence without merging authority classes. | +| Sandbox/redaction | `sandboxed_verify.py`, `sandboxed_web_e2e.py`, `redact_sensitive_log.py` | Execute bounded proof paths and publish safe, useful diagnostics. | +| Decision/schedulers | `pr_review_merge_scheduler.py`, `pr_review_fix_scheduler.py`, autofix workflows | Select allowed actions under expected-head, policy, and writer constraints. | +| Mention router | `agent_mention_router.py`, `agent_mention_sweep.py`, exact-name artifact ledger | Authenticate, deduplicate, and forward explicit review-agent requests. | +| Security/supply-chain gates | CodeQL, Semgrep, OSV, Scorecard, SBOM, secret scan, Strix | Produce independent security evidence. | + +## 4. Control plane and data plane + +The **control plane** reads metadata, classifies evidence, selects work, validates authority, and requests GitHub operations. The **evidence data plane** carries source archives-as-data, test output, logs, findings, review bodies, status/check records, and artifacts. Product application data is outside both planes. + +Untrusted PR content may enter the evidence data plane only through a bounded low-privilege execution or inert inspection path. It must not cross into a privileged `pull_request_target` or default-branch write path as executable code. A privileged action consumes validated identity and bounded evidence, not arbitrary PR shell text. + +## 5. Trust boundaries + +| Boundary | Untrusted side | Trusted side | Required control | +|---|---|---|---| +| PR to base workflow | PR source, filenames, metadata, artifacts | protected workflow source | No untrusted checkout/execution in privileged event; strict input validation. | +| Dispatch caller to central worker | payload and actor claims | default-branch dispatcher | Actor allowlist, canonical schema, live PR comparison, replay/idempotency receipt. | +| Provider to review gate | model text and tool output | normalized gate record | Schema validation, head/run binding, adversarial receipts, no raw authority transfer. | +| Sandbox to public evidence | child stdout/stderr/logs/commands | CI log, summary, comment, result JSON | Complete-boundary redaction, canonicalization, bounded output, stable schema. | +| Central to product repository | central App/token and mutation request | product branch/ruleset | Least privilege, expected head, branch-local lease, protected review. | +| Check/review evidence to merge | heterogeneous evidence | GitHub merge primitive | Authority separation, required gate inventory, qualifying approval, thread state. | + +## 6. Failure domains + +- **GitHub control-plane failure:** API, Actions queue, artifact service, or ruleset availability. Actions remain non-passing; maintenance rotates to other safe work. +- **Provider failure:** OpenCode, Noema, Strix, or upstream model capacity/transport. Provider evidence remains absent or failed; deterministic work continues. +- **Runner/toolchain failure:** DNS, package index, compiler, container, or toolchain. Classified transient failures may retry within a budget; integrity failures do not. +- **Revision race:** head/base changes after evidence collection. Expected-head checks abort mutations and invalidate stale acceptance. +- **Evidence failure:** malformed, oversized, secret-bearing, ambiguous, or contradictory data. Fail closed while preserving bounded non-sensitive diagnosis. +- **Governance failure:** no eligible independent reviewer or required permission. Record the exact external prerequisite and rotate work; never weaken policy. +- **Documentation failure:** implementation and authoritative contracts diverge. Treat as repository debt and repair with machine-checked links/terms. + +## 7. Deployment topology + +The central repository is the organization policy/source repository. Organization required workflows and thin `workflow_call` consumers materialize stable check contexts in target repositories. Default-branch `repository_dispatch` workers perform validated privileged operations. GitHub-hosted runners are disposable execution environments; durable state is GitHub repository/PR/check/review/artifact state and, conceptually, the evidence entities in [DATA_MODEL.md](DATA_MODEL.md). + +No new always-on service or database is required by this architecture baseline. Persisting the conceptual evidence model would be a separate architecture decision. + +## 8. Evolution rules + +- Add central capability behind a versioned thin contract; do not copy a thick workflow into every product repository. +- Preserve independent product operation and product-owned gates. +- Prefer immutable action/workflow pins and short-lived identity. +- Separate deterministic gates from optional model execution so missing model credentials do not prevent unrelated validation. +- Change one authority boundary at a time and require consumer proof after protected integration. +- Keep diagrams, ADRs, tests, operations, and traceability synchronized with source changes. diff --git a/docs/automation/DATA_MODEL.md b/docs/automation/DATA_MODEL.md new file mode 100644 index 000000000..3bb3d7f0e --- /dev/null +++ b/docs/automation/DATA_MODEL.md @@ -0,0 +1,356 @@ +# Conceptual data model and ERD + +Status: accepted conceptual model; no new persistence is implied +Last reviewed: 2026-08-09 + +## 1. Purpose and persistence boundary + +The control plane already exchanges these concepts through GitHub API objects, workflow payloads, artifacts, comments, local JSON, and in-memory structures. This document gives them stable names so exact revision identity and authority do not collapse into an informal boolean such as `passed`. + +The model is **conceptual and logical**. It is not a claim that ContextualWisdomLab operates a database with these tables. GitHub remains the durable system of record today. A future materialized evidence store would require a separate ADR, retention/privacy design, migrations, tenancy model, backup/restore, and rollback plan. + +All entity names contain at least two words and use `snake_case`, matching the organization naming rule. + +## 2. Identity and run ERD + +```mermaid +erDiagram + repository_target ||--o{ automation_run : executes_for + repository_target ||--o{ pull_request_snapshot : contains + pull_request_snapshot }o--|| source_revision : observes_head + pull_request_snapshot }o--|| base_revision : observes_base + automation_run }o--o| source_revision : evaluates + automation_run }o--o| base_revision : compares + automation_run }o--o| merge_revision : may_integrate + repository_target ||--o{ merge_revision : protects + source_revision ||--o{ merge_revision : integrates_as + base_revision ||--o{ merge_revision : integrates_against + + repository_target { + string repository_uid PK + string repository_full_name UK + string default_branch_name + string repository_visibility + } + automation_run { + string automation_run_uid PK + string run_mode + string trigger_kind + datetime started_at + datetime completed_at + string run_outcome + } + pull_request_snapshot { + string pull_request_snapshot_uid PK + int pull_request_number + datetime observed_at + string mergeability_state + string review_decision + } + source_revision { + string source_revision_uid PK + string head_commit_sha UK + string head_branch_name + string head_repository_name + } + base_revision { + string base_revision_uid PK + string live_base_commit_sha + string base_branch_name + string pr_snapshot_base_sha + } + merge_revision { + string merge_revision_uid PK + string merge_commit_sha UK + string merge_kind + string protected_ref_name + datetime integrated_at + } +``` + +`pr_snapshot_base_sha` and `live_base_commit_sha` are deliberately separate. The former explains what GitHub recorded with the PR snapshot; the latter is resolved from the current protected ref at decision time. + +## 3. Evidence ERD + +```mermaid +erDiagram + automation_run ||--o{ workflow_evidence : produces + source_revision ||--o{ check_evidence : binds + source_revision ||--o{ status_evidence : binds + source_revision ||--o{ review_evidence : binds + source_revision ||--o{ model_evidence : binds + source_revision ||--o{ workflow_evidence : binds + source_revision ||--o{ dependency_evidence : depends_on + workflow_evidence ||--o{ check_evidence : reports + workflow_evidence ||--o{ model_evidence : reports + + check_evidence { + string check_evidence_uid PK + string check_name + string conclusion_name + bool required_flag + string source_kind + datetime observed_at + } + status_evidence { + string status_evidence_uid PK + string context_name + string state_name + string creator_login + datetime observed_at + } + review_evidence { + string review_evidence_uid PK + string reviewer_login + string review_state + string reviewed_commit_sha + bool qualifying_human_flag + } + model_evidence { + string model_evidence_uid PK + string provider_name + string model_name + string normalized_outcome + string evidence_digest + } + workflow_evidence { + string workflow_evidence_uid PK + string workflow_name + string workflow_commit_sha + bigint workflow_run_id + int workflow_run_attempt + string workflow_conclusion + } + dependency_evidence { + string dependency_evidence_uid PK + string dependency_locator + string dependency_revision + string dependency_state + datetime observed_at + } +``` + +Evidence classes remain separate because their issuers and permissions differ. +`check_evidence.source_kind` distinguishes a Check Run from another configured +check source; a Commit Status remains `status_evidence`. Neither is silently +converted into `review_evidence`, and normalized model output remains +`model_evidence` even when a workflow uses it as one configured gate input. + +## 4. Operations and governance ERD + +```mermaid +erDiagram + automation_run ||--o{ incident_hypothesis : investigates + automation_run ||--o{ handoff_record : defers + repository_target ||--o{ writer_lease : protects + merge_revision ||--o{ operational_acceptance : validates + automation_run ||--o{ secret_requirement : evaluates + incident_hypothesis ||--o{ operational_acceptance : closes_with + + incident_hypothesis { + string incident_hypothesis_uid PK + string symptom_summary + string root_cause_claim + string owner_boundary + string hypothesis_state + } + handoff_record { + string handoff_record_uid PK + string blocked_action + string external_prerequisite + string continuation_action + datetime recorded_at + } + operational_acceptance { + string operational_acceptance_uid PK + string acceptance_scenario + string protected_commit_sha + string consumer_repository_name + string acceptance_outcome + } + secret_requirement { + string secret_requirement_uid PK + string secret_name + string purpose_name + string consumer_job_name + bool required_flag + } + writer_lease { + string writer_lease_uid PK + string branch_ref_name + string expected_head_sha + string writer_identity + datetime acquired_at + datetime expires_at + } +``` + +## 5. Governance and dispatch ERD + +```mermaid +erDiagram + organization_target ||--o{ repository_target : governs + organization_target ||--o{ orchestration_run : schedules + orchestration_run ||--o{ automation_run : contains + organization_target ||--o{ ruleset_snapshot : observes + repository_target ||--o{ ruleset_snapshot : applies_to + automation_run ||--o{ dispatch_envelope : emits + dispatch_envelope ||--o{ invocation_claim : claims + automation_run ||--o{ scheduler_decision : records + scheduler_decision }o--|| source_revision : binds + + organization_target { + string organization_target_uid PK + string organization_login UK + datetime observed_at + } + orchestration_run { + string orchestration_run_uid PK + string orchestration_mode + datetime started_at + datetime completed_at + string orchestration_outcome + } + ruleset_snapshot { + string ruleset_snapshot_uid PK + bigint ruleset_id + string ruleset_digest + string enforcement_state + datetime observed_at + } + dispatch_envelope { + string dispatch_envelope_uid PK + string event_name + string payload_schema_version + string expected_head_sha + string authenticated_actor + } + invocation_claim { + string invocation_claim_uid PK + string idempotency_key UK + string claim_state + datetime claimed_at + datetime expires_at + } + scheduler_decision { + string scheduler_decision_uid PK + string decision_name + string reason_code + string expected_head_sha + datetime decided_at + } +``` + +A ruleset snapshot is a dated observation, not an eternal property of a +repository. An invocation claim gives at-least-once GitHub delivery an +idempotent side-effect boundary. A scheduler decision records a proposed or +completed action; it is not review evidence. + +## 6. Review, sandbox, and supply-chain ERD + +```mermaid +erDiagram + source_revision ||--o{ review_thread : discusses + source_revision ||--o{ security_finding : affects + source_revision ||--o{ sandbox_evidence : validates + source_revision ||--o{ sbom_snapshot : inventories + workflow_evidence ||--o{ sandbox_evidence : publishes + workflow_evidence ||--o{ sbom_snapshot : produces + + review_thread { + string review_thread_uid PK + string thread_state + bool outdated_flag + datetime observed_at + } + security_finding { + string security_finding_uid PK + string scanner_name + string severity_name + string finding_state + string source_location + } + sandbox_evidence { + string sandbox_evidence_uid PK + string result_schema_version + int child_exit_code + string redaction_policy_version + string evidence_digest + } + sbom_snapshot { + string sbom_snapshot_uid PK + string format_name + string format_version + string artifact_digest + datetime generated_at + } +``` + +These entities store bounded metadata and digests, not unrestricted logs, +credentials, source archives, or personal data. A scanner finding, review +thread, sandbox result, and SBOM are different evidence authorities and must not +be collapsed into one pass/fail row. + +## 7. Entity definitions + +| Entity | Meaning | Current physical representation | +|---|---|---| +| `automation_run` | One finite maintainer, scheduler, reviewer, or recovery execution | GitHub Actions run or external automation invocation; not centrally persisted as this schema | +| `repository_target` | Repository and protected-branch policy context | GitHub repository/ruleset APIs and configured allowlists | +| `pull_request_snapshot` | Time-bounded observation of PR state | GitHub PR API/GraphQL response | +| `source_revision` | Immutable source head under evaluation | Git commit SHA and head repository/ref | +| `base_revision` | Both PR snapshot base and independently resolved live base tip | PR API plus Git ref/API lookup | +| `merge_revision` | Protected integrated commit or merge-group revision, distinct from the PR source head | Pull request merge response and protected ref/workflow evidence | +| `check_evidence` | Check Run result with source kind and revision | GitHub Check Runs API | +| `status_evidence` | Commit status context, creator, state, and revision | Commit Status API | +| `review_evidence` | Formal review submission with author, state, and commit | Pull Request Review API/GraphQL | +| `model_evidence` | Normalized provider/model output with digest and revision | Bounded review/security artifact or job output | +| `workflow_evidence` | Workflow/run/job/attempt and artifact provenance | Actions APIs and bounded artifacts | +| `dependency_evidence` | Cross-PR, package, workflow, or release prerequisite | GitHub/package/attestation evidence | +| `incident_hypothesis` | Falsifiable RCA statement and owner boundary | Incident notes, PR body, or run artifact | +| `handoff_record` | Precise external prerequisite plus autonomous continuation | Maintainer ledger, issue, or PR comment when needed | +| `operational_acceptance` | Protected-main or real-consumer scenario proof | Workflow run, check, artifact receipt, and dated traceability row | +| `secret_requirement` | Explicit secret-to-purpose-to-job contract | `workflow_call` declaration, job environment, and security docs | +| `writer_lease` | Branch-scoped exclusive mutation intent | Live Project/issue assignment, branch/head observation, or automation ledger | +| `organization_target` | Organization scope and observation time for fleet governance | GitHub organization API identity | +| `orchestration_run` | One fleet/hourly invocation grouping repository-scoped child runs | External continuation ledger or top-level Actions sweep receipt | +| `ruleset_snapshot` | Dated ruleset parameters, targets, exclusions, and digest | GitHub ruleset API response plus audit artifact | +| `dispatch_envelope` | Canonical authenticated event fields and schema version | `repository_dispatch`, workflow inputs, or validated local JSON | +| `invocation_claim` | Idempotency claim and completion state for one routed request | Bounded GitHub Actions artifact ledger | +| `scheduler_decision` | Exact-head action, reason, and outcome selected by a scheduler | Job summary/result JSON and workflow evidence | +| `review_thread` | Current/outdated and resolved/unresolved review conversation state | Pull request review-thread GraphQL nodes | +| `security_finding` | Scanner-specific finding with severity, state, and source location | SARIF, dependency alerts, or bounded scanner report | +| `sandbox_evidence` | Versioned result, exit semantics, redaction policy, and evidence digest | `SANDBOXED_*_RESULT` plus scrubbed bounded log/artifact | +| `sbom_snapshot` | Revision-bound software inventory artifact and digest | SPDX/CycloneDX artifact and attestation metadata | + +## 8. Invariants + +1. PR evidence cannot exist without a named source revision. A non-PR scheduled + run may omit source/base relations only with an explicit `not_applicable` + reason and a protected workflow revision. +2. `review_evidence.qualifying_human_flag` is false for authors, automated identities, dismissed reviews, comment-only records, and reviews not bound to the required current head. +3. A merge decision references the current `pull_request_snapshot`, + `source_revision`, `base_revision`, required evidence inventory, and writer + lease. A successful integration creates a distinct `merge_revision`. +4. `operational_acceptance` belongs to `merge_revision`, names the protected + commit and concrete scenario, and cannot be attached directly to a PR + `source_revision`; a source-branch run is insufficient. +5. A `secret_requirement` names one purpose and consumer job; wildcard purpose is invalid. +6. A writer lease covers one repository/ref/expected-head tuple, never the whole organization by implication. +7. Handoff records always name a continuation action unless every work lane is freshly proven non-actionable. +8. Credential values and unrestricted raw logs are never data-model attributes. +9. A ruleset snapshot always has an observation time and digest; stale snapshots + cannot authorize a current mutation. +10. One invocation idempotency key has at most one completed claim; retries link + rather than overwrite their predecessor. +11. Checks, statuses, reviews, model results, scheduler decisions, security + findings, threads, sandbox results, and SBOMs + retain their distinct issuer and authority class. +12. An organization-wide invocation is one `orchestration_run` with + repository-scoped `automation_run` children. Atomic evidence and mutations + remain bound to one `repository_target`; a fleet parent does not imply a + cross-repository writer lease. + +## 9. Future persistence decision gate + +Materialization is justified only if GitHub-native evidence cannot meet query latency, cross-repository history, retention, or audit requirements. Before implementation, measure those gaps and decide tenancy, access control, purpose limitation, legal retention, encryption, deletion, schema migration, disaster recovery, and reconciliation with GitHub as source of truth. diff --git a/docs/automation/ERD.md b/docs/automation/ERD.md new file mode 100644 index 000000000..878b25cba --- /dev/null +++ b/docs/automation/ERD.md @@ -0,0 +1,29 @@ +# Evidence relationship diagrams + +Status: accepted logical ERD; no physical database is implied +Last reviewed: 2026-08-09 + +This exact-path artifact answers the ERD requirement without inventing a SQL +database. GitHub APIs, Actions runs/artifacts, Git commits, rulesets, checks, +statuses, reviews, and threads remain the physical systems of record. The full +attribute catalog, Mermaid ERDs, cardinalities, invariants, retention boundary, +and future-persistence decision gate are normative in +[DATA_MODEL.md](DATA_MODEL.md). + +## Relationship groups + +| Group | Aggregate roots | Revision/authority rule | +|---|---|---| +| Identity and runs | `organization_target`, `orchestration_run`, `repository_target`, `automation_run`, `pull_request_snapshot` | A fleet invocation contains repository-scoped child runs; many observations/runs may bind the same immutable source/base revision. | +| Evidence | `check_evidence`, `status_evidence`, `review_evidence`, `model_evidence`, `workflow_evidence`, `dependency_evidence` | Issuer classes remain separate and every PR record binds an exact source revision. | +| Decisions and dispatch | `ruleset_snapshot`, `dispatch_envelope`, `invocation_claim`, `scheduler_decision`, `writer_lease` | Dated live-state observations and idempotency/expected-head identity precede side effects. | +| Security and operation | `merge_revision`, `review_thread`, `security_finding`, `sandbox_evidence`, `sbom_snapshot`, `incident_hypothesis`, `operational_acceptance`, `handoff_record` | Operational acceptance attaches to the protected integrated revision, never directly to the PR source; bounded evidence/digests are retained. | + +## Physical-model decision + +ERD status is intentionally **N/A for a deployed relational database**. Any +proposal to materialize these logical entities must first add an ADR covering +tenancy, access control, purpose and retention, deletion, encryption, schema +migration, reconciliation with GitHub, backup/restore, and rollback. Until then, +the diagrams describe relationships that implementations and evidence receipts +must preserve, not tables that operators should provision. diff --git a/docs/automation/INCIDENT_RUNBOOK.md b/docs/automation/INCIDENT_RUNBOOK.md new file mode 100644 index 000000000..1221a7f9c --- /dev/null +++ b/docs/automation/INCIDENT_RUNBOOK.md @@ -0,0 +1,110 @@ +# Incident runbook — automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 + +Use this document for severity, RCA, repair, and closure decisions. The executable read-only diagnostics, guarded rerun/disable/rollback templates, ownership placeholders, evidence-deletion safeguards, and canary receipt are in [RUNBOOK.md](RUNBOOK.md). + +## 1. Use this runbook for + +False-green or stale gates; unauthorized or raced writes; secret/log disclosure; privileged untrusted-code execution; scanner/reviewer failure; queue starvation; artifact/provenance mismatch; ruleset drift; broken central consumers; and documentation that materially misstates an authority or recovery boundary. + +## 2. Severity + +| Severity | Examples | Initial response objective | +|---|---|---| +| SEV-1 | credential exposure, unauthorized protected write/merge, privileged execution of malicious PR code | contain immediately; rotate/revoke affected authority; preserve minimal evidence | +| SEV-2 | false-green required gate, organization-wide consumer breakage, widespread queue starvation | stop affected mutation path; establish scope and safe fallback | +| SEV-3 | one PR/provider/toolchain blocked with other safe paths available | classify, repair or defer, continue queue | +| SEV-4 | documentation/telemetry drift without current unsafe behavior | record and repair in bounded docs/control increment | + +## 3. First response + +1. Record repository, PR, exact head, live base tip, workflow source SHA, run ID/attempt, event, actor, and first observed time. +2. Stop only the affected mutation/credential path. Do not disable unrelated security gates or the whole fleet without evidence. +3. For suspected credential disclosure, restrict the evidence, revoke/rotate the capability, and preserve only bounded redacted receipts. +4. Re-fetch live state; do not trust screenshots, remembered SHAs, or predecessor-run summaries. +5. Identify the first failing boundary and classify the affected authority: source, workflow, credential, check, status, review, merge, artifact, or operations. +6. Move unrelated executable work to another branch/repository/lane. + +## 4. RCA and feasibility + +State separately: + +- symptom; +- immediate cause; +- root-cause hypothesis and evidence that could falsify it; +- owner boundary; +- affected and unaffected paths; +- materially distinct remedies; +- feasibility of each remedy under current permissions, policy, writer state, dependencies, rollback, and acceptance test; and +- smallest root-cause-changing repair. + +Do not label `failed`, `pending`, `cancelled`, `rate_limited`, or `action_required` as a root cause. Do not propose a credential, reviewer, bypass, or branch rewrite that does not exist or cannot be safely verified. + +## 5. Repair and recovery + +1. Add a fail-first regression at the closest durable contract. +2. Apply the smallest cohesive production repair on a branch with a clear writer lease. +3. Run the original reproduction, focused suite, full suite, coverage/docstrings, security/supply-chain gates, and documentation checks. +4. Push without rewriting shared history; bind every hosted result to the new head. +5. Obtain current-head automated and qualifying independent review; resolve only addressed threads. +6. Merge through protection and expected-head semantics. +7. Execute protected-main central acceptance and an affected real-consumer positive/negative scenario where applicable. +8. Update traceability, ADR, operations, and change history. + +## 6. Playbooks + +### 6.1 Suspected credential disclosure + +- Restrict or delete public evidence only through supported GitHub controls; do not copy the secret into comments or tickets. +- Rotate/revoke first when the value may be live. +- Determine every publication path: stdout, stderr, timeout, exception, service file/tail, command, result JSON, summary, comment, artifact, cache. +- Test the exact encoding/evasion form and neighboring non-sensitive diagnostics. +- Redact complete content before truncation and validate that no stream/result concatenation reconstructs the value. +- Close only after protected-main/consumer output proves redaction and ordinary diagnosis remains intact. + +### 6.2 Stale or false-green evidence + +- Capture the claimed and actual source/workflow/base/run identities. +- Mark mismatched evidence non-authorizing; do not rerun before understanding the binding defect. +- Repair the producer/consumer identity contract and add moved-head/live-base/synthetic-merge negative tests. +- Re-run all evidence on the new exact head; predecessor approvals do not transfer. + +### 6.3 Queue starvation or provider outage + +- Read queue age and provider failure once; deduplicate dispatches by PR/head/run identity. +- Defer the affected action and work another ledger lane. +- Retry only documented transient classes within a budget. +- Escalate capacity/permission only when it is the sole remaining fleet prerequisite. + +### 6.4 Concurrent writer or moved branch + +- Stop source writes immediately. +- Fetch exact remote head/base and compare trees/lineage; preserve both writers' work. +- Move to a disjoint task or prepare a non-destructive reconciliation. +- Never force-push, select `ours/theirs` wholesale, or manufacture a repair workflow. + +### 6.5 Ruleset or reviewer-governance drift + +- Fetch live ruleset/branch protection, required contexts, review count, last-push dismissal, CODEOWNERS, and bypass actors. +- Keep automation checks and formal human approval separate. +- Restore policy through an independently reviewed administrative change; do not compensate in source with fake approval. + +## 7. Rollback + +Rollback selects a reviewed known-good version and names the security trade-off. If the previous version contains the incident cause, disable the affected optional path or introduce a narrow fail-closed guard instead of restoring it. Re-run exact protected-main and consumer acceptance after rollback. + +## 8. Closure and reopening + +An incident closes only when: + +- the root cause and affected scope are evidence-backed; +- the durable regression fails before and passes after the repair; +- the integrated protected commit is identified; +- required exact-head gates and qualifying review passed without bypass; +- protected-main or real-consumer positive/negative scenarios passed; +- rollback and monitoring are documented; and +- traceability and authoritative docs are current. + +Reopen on any contradictory protected-main/consumer result, recurrence, newly discovered publication path, ruleset drift, or evidence that the accepted revision was not the one executed. diff --git a/docs/automation/OPERABILITY.md b/docs/automation/OPERABILITY.md new file mode 100644 index 000000000..f580df634 --- /dev/null +++ b/docs/automation/OPERABILITY.md @@ -0,0 +1,87 @@ +# Operability and service objectives + +Status: accepted baseline; telemetry gaps are explicit +Last reviewed: 2026-08-09 + +## 1. Service model + +The control plane is a set of GitHub-hosted, event-driven and scheduled workflows rather than one continuously running service. Its availability depends on GitHub APIs/Actions, runner capacity, external review providers, package/tool sources, organization policy, and eligible human reviewers. Degradation of one dependency must not idle unrelated maintenance work. + +## 2. Service-level indicators and targets + +Targets are design objectives, not claims of achieved performance. [TRACEABILITY.md](TRACEABILITY.md) marks missing telemetry. + +| SLI | Proposed target | Measurement | +|---|---|---| +| Exact-revision binding | 100% of gate and mutation records name exact source and workflow revision | contract tests plus run receipts | +| Unauthorized/stale mutation | 0 accepted writes after expected head/base mismatch | scheduler/autofix rejection counters and tests | +| Credential disclosure | 0 known credentials in published evidence | adversarial tests, secret scanning, incident reports | +| Deterministic gate completion | 99% within 30 minutes excluding declared GitHub outage/queue saturation | run timestamps by workflow and repository | +| Model review completion | 95% within 6 hours; no correctness trade for latency | dispatch-to-terminal duration by provider/class | +| Queue starvation | 0 executable item left untouched across two hourly sweeps | maintainer continuation ledger | +| Incident closure quality | 100% of operational closures have protected-main or real-consumer receipt | traceability audit | +| Documentation freshness | 100% of boundary-changing PRs update linked docs/ADR/tests | documentation contract and review checklist | + +## 3. Telemetry model + +Correlate logs, metrics, checks, reviews, and artifacts by: + +- repository full name and PR number; +- source head SHA and live base SHA; +- workflow repository/SHA, run ID, and run attempt; +- trigger and event delivery identity; +- evidence class and gate name; +- provider and failure class, never secret value; and +- writer lease and expected-head outcome. + +Recommended metrics include queue age, queued/running/terminal counts, dispatch-to-start latency, duration, retry class/count, provider exhaustion, stale-evidence rejection, expected-head abort, redaction setup failure, artifact expiry, ruleset drift, and protected-main acceptance age. + +Raw evidence may contain business PII. Access and retention are purpose-bound; metrics use identifiers/hashes and classifications rather than raw content where possible. + +## 4. Queue and capacity management + +- Use concurrency keys scoped to repository, PR, and event class so unrelated work can proceed. +- Cancellation may supersede stale current-head work only when cancellation still produces unambiguous evidence; otherwise preserve the run and bound queue growth elsewhere. +- Long-running model reviews enter a deferred set after one state read. The maintainer selects another safe lane rather than polling. +- Dispatch budgets prevent storms but must not permanently starve older eligible PRs; age and fairness are observable. +- GitHub-wide runner saturation triggers local verification, docs, RCA, or other disjoint work and an operator alert when age crosses the objective. + +## 5. Provider and platform outages + +| Failure | Immediate action | Continued work | Recovery evidence | +|---|---|---|---| +| GitHub API/Actions outage | Preserve non-passing state; stop writes whose live preconditions cannot be checked | local deterministic tests/docs; read-only analysis from already fetched exact source | fresh API state and exact-head rerun | +| Runner queue saturation | Avoid duplicate dispatch; inspect queue age and org capacity | other branches/repos/local proof | queued run starts and completes on intended head | +| Model provider outage/rate limit | Classify and defer or use reviewed fallback | deterministic gates and other queue items | valid current-head model result with provider identity | +| OIDC/App exchange failure | Fail privileged operation closed | read-only/local work | successful scoped exchange and intended operation | +| Package/DNS failure | Retry only if classified transient within budget | cached/disjoint work | integrity-verified install and original test | +| Eligible reviewer unavailable | Preserve governance wait | all other code/docs/operations/product lanes | counted independent current-head approval | + +## 6. Deployment and rollback + +Central workflow changes deploy when merged to the protected default branch and then affect required/thin consumers according to GitHub source semantics. Rollout therefore uses: + +1. exact-head PR gates; +2. protected merge without bypass; +3. central protected-main run; +4. one low-risk real consumer and negative control; +5. wider fleet observation; and +6. incident closure only after receipts are recorded. + +Rollback reverts to a reviewed known-good protected commit or disables only the affected optional route. It must not restore a known trust-boundary vulnerability, remove unrelated gates, change credential identity, or erase evidence. Emergency rollback is followed by a normal reviewed reconciliation PR and consumer proof. + +## 7. Operator experience + +Each terminal failure should state the first failing boundary, evidence identity, classification, what was preserved, the smallest operator action, and the exact rerun/acceptance path. It should not emit a URL without diagnosis, claim success for a skip, or hide ordinary failure context under over-broad redaction. + +Routine scheduled runs suppress status-only narration. Notifications are reserved for a required external permission/governance action with no autonomous alternative, an irreconcilable decision, a substantive protected merge/release, or a safety boundary. + +## 8. Current operability gaps + +- No single persisted cross-repository continuation ledger implements the conceptual `automation_run` model. +- Proposed SLI rollups are not all emitted as metrics. +- Total sandbox output and service-file quotas remain separate work from redaction. +- Legacy inherited deploy-secret usage needs an explicit-secret migration. +- Real-consumer acceptance receipts are distributed across runs and PRs rather than a single index. + +These gaps remain open in [TRACEABILITY.md](TRACEABILITY.md); they are not hidden by the target SLOs. diff --git a/docs/automation/PRD.md b/docs/automation/PRD.md new file mode 100644 index 000000000..08ea28145 --- /dev/null +++ b/docs/automation/PRD.md @@ -0,0 +1,112 @@ +# Product requirements — CWL automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 +Owners: ContextualWisdomLab maintainers and repository operators + +## 1. Problem + +ContextualWisdomLab operates many independently useful products that reuse organization-wide review, security, coverage, and merge controls. Without one control plane, repository-local workflow copies drift, a result from the wrong revision can be mistaken for current evidence, long-running reviewers can idle the whole maintenance loop, and operators must reconstruct decisions from comments and conversations. + +The product is an automation control plane that converts live repository state into safe, reviewable actions while preserving human merge authority and product-repository autonomy. Its buyer-visible value is shorter and more reliable change lead time, fewer false-green gates, lower incident-reconstruction cost, and auditable organization-wide governance. + +## 2. Users and stakeholders + +| Persona | Primary need | Harm to prevent | +|---|---|---| +| Product maintainer | Fix and merge valid changes without babysitting every check | Stale evidence, queue starvation, destructive branch repair | +| Independent reviewer | See exact code, tests, risk, and diagnostics for the reviewed revision | Approval reuse after a push; automated approval impersonation | +| Security operator | Enforce shared security boundaries and investigate failures | Secret leakage, untrusted-code execution, fail-open scanner results | +| Product team | Reuse thin central contracts while remaining independently operable | Central coupling, credential sprawl, repository-specific policy drift | +| Buyer or auditor | Verify governance, provenance, rollback, and operational proof | Unverifiable claims based only on prose or predecessor runs | +| Automation agent | Select and execute the next safe action from fresh evidence | Private-memory authority, report-only completion, concurrent writers | + +## 3. Product outcomes + +- Every merge decision is bound to the exact current source head and independently resolved live base state. +- Checks, commit statuses, formal reviews, model findings, and merge authority remain separate evidence classes. +- One pending reviewer or check blocks only its action; other safe work continues. +- Shared workflows are trusted, minimally privileged, and reusable without thick copies in product repositories. +- CI evidence is useful for diagnosis but does not disclose credentials or silently erase ordinary failure context. +- Code integration is not incident closure: affected behavior is demonstrated again from protected main or a real consumer. +- Product, security, architecture, operations, and decision documents can reconstruct the system without private conversation. + +## 4. Operating modes + +### 4.1 PR-maintenance mode + +The automation repeatedly refreshes every open pull request, exact head, live base tip, review, unresolved thread, required check, workflow run, ruleset, and writer state. It then merges a genuinely clean exact head, fixes a valid current defect test-first, removes a repository-owned blocker, advances a disjoint pull request, or records a precise non-actionable external prerequisite and moves on. + +### 4.2 Product-development mode + +When no PR or accepted issue is safely executable, the automation selects exactly one bounded, highest-impact control-plane or buyer-visible gap. It researches current primary standards where material, implements the slice test-first, updates authoritative documents and change history, opens or updates one reviewable pull request, and returns to the live queue. + +The modes share one work-conserving ledger. Completing one merge, document, review dispatch, or product slice never implies that the remaining queue is empty. + +## 5. Functional requirements + +| ID | Requirement | +|---|---| +| PRD-01 | Re-fetch current repository, PR, issue, revision, gate, ruleset, and writer evidence before acting. | +| PRD-02 | Bind acceptance to an exact source head and the independently resolved current base tip; invalidate prior evidence after movement. | +| PRD-03 | Keep check runs, commit statuses, formal reviews, model output, branch protection, and merge actions as distinct authorities. | +| PRD-04 | Require qualifying non-author human approval when repository policy requires it; no agent may synthesize or impersonate approval. | +| PRD-05 | Maintain branch-local writer leases and move to disjoint work when a live writer owns the target. | +| PRD-06 | Continue useful work while checks, reviews, providers, or external governance actions are pending. | +| PRD-07 | Perform RCA, compare materially distinct remedies, prove feasibility, and implement the smallest root-cause-changing repair. | +| PRD-08 | Preserve ordinary stdout/stderr, timeout, service-tail, exit-code, and structured-result diagnostics while redacting credentials at every publication boundary. | +| PRD-09 | Treat documentation drift across PRD, TRD, architecture, UML, ERD/data model, security, tests, operations, ADRs, and traceability as executable repository debt. | +| PRD-10 | Require protected-main or real-consumer evidence before closing an operational incident. | +| PRD-11 | Preserve standalone product operation and expose stable, thin, versioned central interfaces. | +| PRD-12 | Use model credentials only for actual model calls; autonomous model-backed development uses `NVIDIA_NIM_API_KEY`, never `COPILOT_GITHUB_TOKEN`. | +| PRD-13 | Protect business PII through access control, purpose limitation, retention, audit, and controlled evidence scope; do not apply blanket PII masking that makes the workflow unusable. | +| PRD-14 | Maintain beginner-readable operator output, public documentation, and 100% owned production statement/branch/docstring evidence. | + +## 6. Non-functional requirements + +- **Correctness:** no stale, absent, skipped-required, queued, cancelled, synthetic-merge-only, or predecessor-head evidence is promoted to exact-head success. +- **Security:** least privilege, immutable trusted sources, fail-closed validation, bounded output, credential redaction, and no untrusted source execution in privileged events. +- **Reliability:** bounded retries only for classified transient failures; permanent integrity, authorization, TLS, ref, and schema failures fail immediately. +- **Operability:** every terminal decision has a reason, evidence identity, recovery action, and observable acceptance test. +- **Performance:** linear or bounded processing for attacker-influenced evidence; provider latency is not traded for weaker correctness. +- **Compatibility:** thin consumers survive central internal refactoring as long as versioned inputs, outputs, check names, and authority semantics remain compatible. +- **Auditability:** decisions, deviations, rollback, and current implementation status are linked through the traceability matrix. + +## 7. Degraded behavior + +| Condition | Required behavior | +|---|---| +| Reviewer or model provider unavailable | Keep the affected gate non-passing, defer it, and execute another safe item. | +| GitHub Actions queue saturated | Preserve queued state, avoid duplicate dispatch storms, advance local or disjoint work, and alert on queue age. | +| OIDC or App-token exchange unavailable | Do not invent a credential or broaden `GITHUB_TOKEN`; fail the privileged action closed and continue read-only work. | +| Exact head or live base changes during an action | Abort the write, discard stale acceptance, refresh state, and re-plan. | +| Redaction context cannot be established safely | Emit no potentially colliding evidence and exit with the documented setup-failure code. | +| Documentation contradicts implementation | Record the mismatch as a traceability gap and repair code or docs in the same bounded increment. | +| Independent approval is the only PR gate | Preserve expected-head-safe merge intent if policy allows, then rotate to other work. | + +## 8. Success measures + +The initial product acceptance requires all of the following: + +- zero merges authorized solely by model, status-only, stale-head, or author-self evidence; +- zero accepted check/review records without exact revision identity; +- zero known credential disclosures through completed, timeout, exception, service-tail, command, or structured-result evidence; +- 100% owned production statement and branch coverage plus public docstrings; +- a machine-checked authoritative documentation index, ADR index, diagrams, and requirement traceability; +- a protected-main or real-consumer receipt for every operational incident declared closed; and +- a work-conserving exit sweep showing no safe remaining item before a scheduled run ends for reasons other than its real execution budget. + +Proposed service-level indicators and targets are defined in [OPERABILITY.md](OPERABILITY.md). Targets without implemented telemetry are explicitly marked as gaps rather than reported as achieved. + +## 9. Out of scope + +- Replacing GitHub's branch protection, rulesets, review model, or merge queue. +- Letting an automated reviewer count as a qualifying human approval. +- Hosting product data or implementing product-specific release and deployment logic centrally. +- Persisting the conceptual evidence ERD as a new database without a separate approved design and migration. +- Unlimited log capture or sandbox resource quotas; those require their own implementation slice. +- Blanket masking of names, email bodies, or other business PII in a way that destroys the operator's task. Access and disclosure controls are preferred. + +## 10. Release and closure + +A documentation or source pull request is merge-ready only when its exact head passes deterministic tests, security and supply-chain checks, coverage/docstring gates, automated review, qualifying independent approval, thread resolution, and repository protection without bypass. A runtime incident closes only after the integrated protected head or an affected real consumer produces the acceptance evidence named in the runbook. diff --git a/docs/automation/README.md b/docs/automation/README.md new file mode 100644 index 000000000..dbd60fd10 --- /dev/null +++ b/docs/automation/README.md @@ -0,0 +1,48 @@ +# Automation control-plane documentation + +Status: authoritative documentation index +Last reviewed: 2026-08-09 +Scope: `ContextualWisdomLab/.github` organization automation and its thin repository consumers + +This directory is the durable specification for the ContextualWisdomLab automation control plane. It turns decisions that were previously distributed across pull-request bodies, incident comments, workflow prose, and private conversation into a reviewable documentation graph. The implementation remains authoritative for observed behavior; these documents are authoritative for intended behavior. Any mismatch is a defect and must be recorded in [TRACEABILITY.md](TRACEABILITY.md) until repaired. + +## Reading order + +1. [PRD.md](PRD.md) — operator and buyer outcomes, scope, and acceptance. +2. [TRD.md](TRD.md) — normative event, evidence, identity, retry, secret, and compatibility contracts. +3. [ARCHITECTURE.md](ARCHITECTURE.md) — viewpoints, bounded contexts, trust boundaries, and failure domains. +4. [DATA_MODEL.md](DATA_MODEL.md) and [ERD.md](ERD.md) — conceptual evidence + model and exact-path logical ERD; neither claims a deployed database. +5. [UML.md](UML.md) — component, sequence, state, authority, deployment, and incident flows. +6. [SECURITY.md](SECURITY.md) and [THREAT_MODEL.md](THREAT_MODEL.md) — security objectives, controls, abuse cases, and residual risk. +7. [TEST_STRATEGY.md](TEST_STRATEGY.md) — deterministic, security, compatibility, documentation, and protected-main acceptance gates. +8. [OPERABILITY.md](OPERABILITY.md), [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md), + and executable [RUNBOOK.md](RUNBOOK.md) — service objectives, observability, + diagnosis, ownership, recovery, retention, rollback, and acceptance receipts. +9. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. +10. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. + +## Authority and change control + +The sources of truth are ordered as follows: + +1. live GitHub repository, pull-request, ruleset, check, review, and workflow evidence for current state; +2. protected default-branch workflow and script source for implemented behavior; +3. this documentation set for intended cross-component contracts; +4. dated incident receipts and pull-request bodies for historical evidence only. + +Historical SHAs, run IDs, check counts, and performance numbers belong in dated evidence or pull-request records, not timeless architecture. A head change makes predecessor-head checks and reviews historical. A base-branch change requires independently resolving the new live base tip and regenerating evidence. + +Changes to triggers, permissions, secret names, evidence identity, retry classes, reviewer eligibility, writer authority, merge behavior, sandbox output, or protected-main closure require: + +- a linked ADR update or a new ADR; +- corresponding tests and traceability rows; +- an operability and rollback review; +- current-head checks and qualifying independent review; and +- protected-main operational acceptance when the change affects runtime behavior. + +## Scope boundary + +The central repository owns shared policy, trusted workflow entrypoints, evidence normalization, review dispatch, and merge/fix scheduling. Product repositories own their source, product-specific tests, release gates, deployments, data, and thin calls into central contracts. Central automation must not silently become an application runtime, a product database, an approval impersonator, or a repository-specific business workflow. + +The hourly commercial-maintenance automation is an orchestration policy outside GitHub Actions. GitHub Actions remains the event-driven execution and evidence plane. The hourly policy must use live repository evidence and must not treat its private memory as authority. diff --git a/docs/automation/RUNBOOK.md b/docs/automation/RUNBOOK.md new file mode 100644 index 000000000..c020410cd --- /dev/null +++ b/docs/automation/RUNBOOK.md @@ -0,0 +1,337 @@ +# Operator runbook — CWL automation control plane + +Status: accepted executable baseline; placeholders must be resolved from live authority +Last reviewed: 2026-08-09 + +This is the command companion to [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md). It never supplies an operator identity, approval, credential, retention period, rollback SHA, or escalation destination. Resolve every `` from live GitHub state, the incident record, repository policy, and the authorized organization roster. Do not paste secret values into a shell history, issue, comment, artifact, or receipt. + +## 1. Safety contract + +1. Start with read-only queries. A missing permission or `404` is not proof that a rule, private repository, or artifact does not exist. +2. Record the repository, PR, source head, live base, workflow source, run ID/attempt, event, and actor before any rerun or mutation. +3. Do not rerun a permanent integrity, authorization, TLS, ref, schema, policy, or product-test failure. Repair its cause first. +4. Do not disable a required workflow, ruleset, branch protection, security gate, or independent-review requirement as incident mitigation. +5. Rerun, disable, rollback, deletion, and credential rotation require the named authority in §3 and an incident receipt. These templates grant no authority by themselves. +6. Never force-push, rewrite shared history, delete a repository/ref, or run a broad recursive cleanup from this runbook. +7. Re-fetch live state immediately before every mutating command. If the head, base, writer, policy, or approved scope changed, stop and re-plan. + +## 2. Read-only diagnosis + +### 2.1 Resolve the target without inventing values + +Set only non-secret identifiers. Keep the literal placeholders until the incident record or live API supplies each value. + +```bash +CWL_TARGET_REPO='' +CWL_PR_NUMBER='' +CWL_HEAD_SHA='' +CWL_BASE_REF='' +CWL_RUN_ID='' +CWL_WORKFLOW_FILE='' +CWL_CONTROL_REPO='ContextualWisdomLab/.github' +CWL_CONTROL_BASE_REF='' +``` + +Confirm authentication and repository identity without printing a token: + +```bash +gh auth status +gh repo view "$CWL_TARGET_REPO" \ + --json nameWithOwner,defaultBranchRef,visibility,url +``` + +### 2.2 Re-fetch PR, exact head, and live base + +```bash +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/pulls/${CWL_PR_NUMBER}" \ + --jq '{number,state,draft,mergeable,mergeable_state,head:{repo:.head.repo.full_name,ref:.head.ref,sha:.head.sha},base:{repo:.base.repo.full_name,ref:.base.ref,snapshot_sha:.base.sha},updated_at}' + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/git/ref/heads/${CWL_BASE_REF}" \ + --jq '{ref,live_base_sha:.object.sha}' +``` + +Treat the PR's `.base.sha` as a snapshot and the ref lookup as the live base tip. Do not substitute one for the other. + +### 2.3 Inspect checks, statuses, reviews, and threads + +```bash +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/commits/${CWL_HEAD_SHA}/check-runs" \ + --jq '.check_runs[] | {name,status,conclusion,app:.app.slug,head_sha,started_at,completed_at,html_url}' + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/commits/${CWL_HEAD_SHA}/status" \ + --jq '{sha,state,statuses:[.statuses[] | {context,state,creator:.creator.login,created_at,target_url}]}' + +gh api --method GET --paginate \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/pulls/${CWL_PR_NUMBER}/reviews" \ + --jq '.[] | {id,user:.user.login,state,commit_id,submitted_at,html_url}' +``` + +Review threads require GraphQL. Split the verified repository name only after `gh repo view` succeeds: + +```bash +CWL_OWNER="${CWL_TARGET_REPO%%/*}" +CWL_REPOSITORY="${CWL_TARGET_REPO#*/}" + +gh api graphql \ + -f owner="$CWL_OWNER" \ + -f name="$CWL_REPOSITORY" \ + -F number="$CWL_PR_NUMBER" \ + -f query='query($owner:String!,$name:String!,$number:Int!){repository(owner:$owner,name:$name){pullRequest(number:$number){reviewDecision,reviewThreads(first:100){nodes{isResolved,isOutdated,comments(first:20){nodes{author{login}path,createdAt,url}}}pageInfo{hasNextPage,endCursor}}}}}' +``` + +If `hasNextPage` is true, paginate before declaring zero unresolved threads. + +```bash +CWL_THREAD_CURSOR='' + +gh api graphql \ + -f owner="$CWL_OWNER" \ + -f name="$CWL_REPOSITORY" \ + -F number="$CWL_PR_NUMBER" \ + -f after="$CWL_THREAD_CURSOR" \ + -f query='query($owner:String!,$name:String!,$number:Int!,$after:String){repository(owner:$owner,name:$name){pullRequest(number:$number){reviewThreads(first:100,after:$after){nodes{isResolved,isOutdated,comments(first:20){nodes{author{login}path,createdAt,url}}}pageInfo{hasNextPage,endCursor}}}}}' +``` + +### 2.4 Inspect workflow and artifact identity + +```bash +gh run view "$CWL_RUN_ID" \ + --repo "$CWL_TARGET_REPO" \ + --json databaseId,attempt,event,headBranch,headSha,status,conclusion,workflowName,createdAt,startedTime,updatedAt,url + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/runs/${CWL_RUN_ID}" \ + --jq '{id,run_attempt,event,path,head_branch,head_sha,workflow_id,referenced_workflows,status,conclusion,created_at,updated_at,html_url}' + +gh run list \ + --repo "$CWL_TARGET_REPO" \ + --commit "$CWL_HEAD_SHA" \ + --limit 100 \ + --json databaseId,attempt,event,headSha,status,conclusion,workflowName,createdAt,updatedAt,url + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/runs/${CWL_RUN_ID}/artifacts" \ + --jq '.artifacts[] | {id,name,size_in_bytes,expired,created_at,expires_at,archive_download_url}' +``` + +`headSha` may identify a synthetic merge or merge-group revision for some events. Confirm the workflow's own source-receipt fields before treating it as source-head evidence. View failed logs only inside the approved evidence boundary; do not copy raw output into a public issue: + +```bash +gh run view "$CWL_RUN_ID" --repo "$CWL_TARGET_REPO" --log-failed +``` + +### 2.5 Inspect live governance + +```bash +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/rulesets?includes_parents=true" \ + --jq '.[] | {id,name,target,enforcement,source_type,source,conditions,rules}' + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/branches/${CWL_BASE_REF}/protection" + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/rules/branches/${CWL_BASE_REF}" +``` + +Inherited organization rules and repository-local branch protection are separate surfaces. Inspect both; a permission-limited empty response is an unresolved prerequisite, not a clean result. + +## 3. Ownership and escalation record + +Populate this table in the private incident record from the current authorized roster. Do not commit personal contact details to this public repository. + +| Role | Required live value | Authority | +|---|---|---| +| Incident commander | `` | Coordinates severity, scope, and closure; does not inherit technical bypass authority. | +| Control-plane owner | `` | Owns central workflow/script diagnosis and reviewed rollback PR. | +| Security owner | `` | Authorizes evidence restriction, credential rotation, and security reopening/closure. | +| Organization governance owner | `` | Owns rulesets, required workflows, Apps, and branch-protection changes. | +| Credential owner | `` | Revokes/rotates only the affected credential through its provider. | +| Consumer owner | `` | Approves and verifies the real product-repository canary. | +| Escalation destination | `` | Receives sensitive coordination; never substitute a public PR comment for a security channel. | +| Decision deadline | `` | Defines when to reassess, not when to weaken a gate. | + +If a required owner or channel cannot be resolved, keep the affected mutation fail-closed, record the missing authority, and continue unrelated read-only or deterministic work. + +## 4. Safe action templates + +### 4.1 Rerun a classified transient failure + +Before rerun, confirm all of the following in the incident record: + +- the run belongs to the intended repository/workflow and its evidence receipt names the current source identity; +- the failure is a documented transient class, not authentication, authorization, integrity, TLS, ref, schema, policy, or test failure; +- no equivalent current-head run is queued or running; +- the retry count and total budget remain; and +- `` authorized the rerun. + +Read the run again immediately before mutation: + +```bash +gh run view "$CWL_RUN_ID" \ + --repo "$CWL_TARGET_REPO" \ + --json databaseId,attempt,event,headSha,status,conclusion,workflowName,url +``` + +Then rerun only failed jobs, preserving the original run identity in the incident record: + +```bash +gh run rerun "$CWL_RUN_ID" --repo "$CWL_TARGET_REPO" --failed +``` + +Do not loop this command. Re-fetch the new attempt once, record its result, and reclassify before any further action. + +### 4.2 Disable only an optional affected workflow + +Disabling is never an acceptable way to make a required gate disappear. Use this only when live ruleset inspection proves the workflow is optional, the governance owner approves the exact scope, the safer fallback is documented, and a re-enable condition exists. + +```bash +gh workflow view "$CWL_WORKFLOW_FILE" --repo "$CWL_CONTROL_REPO" --yaml +``` + +After recording ``, ``, the current workflow state, and the re-enable condition, the authorized mutation is: + +```bash +gh workflow disable "$CWL_WORKFLOW_FILE" --repo "$CWL_CONTROL_REPO" +``` + +Re-enable after the reviewed repair or rollback reaches protected main: + +```bash +gh workflow enable "$CWL_WORKFLOW_FILE" --repo "$CWL_CONTROL_REPO" +``` + +If the path is required, deploy a narrow fail-closed guard through a protected pull request instead of disabling the workflow or changing the ruleset. + +### 4.3 Roll back through a protected revert pull request + +Resolve ``, ``, and `` from GitHub history and the incident decision. Prefer reverting the smallest causal commit. Do not use `reset`, a force-push, or a direct write to the protected branch. + +Run the template only from a verified clone of `$CWL_CONTROL_REPO`. `git status --short` must be empty before creating the rollback branch; preserve and stop for any existing work rather than discarding it. + +```bash +CWL_BAD_COMMIT_SHA='' +CWL_KNOWN_GOOD_SHA='' +CWL_INCIDENT_ID='' +CWL_ROLLBACK_BRANCH="incident/${CWL_INCIDENT_ID}-rollback" + +gh repo view "$CWL_CONTROL_REPO" --json nameWithOwner,defaultBranchRef,url +git remote get-url origin +git status --short +git check-ref-format --branch "$CWL_ROLLBACK_BRANCH" +git fetch origin "$CWL_CONTROL_BASE_REF" +git switch --create "$CWL_ROLLBACK_BRANCH" "origin/${CWL_CONTROL_BASE_REF}" +git show --stat --oneline "$CWL_BAD_COMMIT_SHA" +git show --stat --oneline "$CWL_KNOWN_GOOD_SHA" +git revert --no-commit "$CWL_BAD_COMMIT_SHA" +git diff --cached --check +``` + +Run the original reproduction, focused/full tests, security checks, and documentation contract before committing. Inspect the staged tree and then create a normal reviewed commit and PR: + +```bash +git status --short +git diff --cached +git commit -m "revert: contain ${CWL_INCIDENT_ID}" +git push --set-upstream origin "$CWL_ROLLBACK_BRANCH" +gh pr create \ + --repo "$CWL_CONTROL_REPO" \ + --base "$CWL_CONTROL_BASE_REF" \ + --head "$CWL_ROLLBACK_BRANCH" \ + --title "revert: contain ${CWL_INCIDENT_ID}" \ + --body-file '' +``` + +The PR body must name the causal commit, known-good comparison, security trade-off, test evidence, rollback-of-rollback plan, consumer canary, and reopen conditions. A known-vulnerable version is not a valid rollback target; disable only the affected optional path or add a narrow fail-closed guard instead. + +## 5. Evidence retention and deletion + +No retention duration is defined by this runbook. Resolve and record the applicable repository, organization, contractual, privacy, and incident policy values: + +| Evidence class | Retention value to resolve | Minimum handling rule | +|---|---|---| +| Raw workflow logs and service output | `` | Restrict access; retain only as long as the incident and governing policy require; never treat a credential as evidence. | +| Workflow artifacts/source archives | `` | Verify run/name/digest/source before access; delete expired-purpose copies. | +| Review/check/status records | `` | Preserve exact identity and decision history subject to platform and legal policy. | +| Minimal incident receipt | `` | Retain hashes, IDs, classifications, decisions, owners, and bounded redacted excerpts rather than raw sensitive data. | +| Business PII | `` | Purpose-limit access and deletion; involve the data owner before disclosure or removal. | + +For suspected credential disclosure, rotate/revoke first. Then enumerate the exact artifacts before deletion: + +```bash +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/runs/${CWL_RUN_ID}/artifacts" \ + --jq '.artifacts[] | {id,name,size_in_bytes,expired,created_at,expires_at}' +``` + +Only after `` approves the exact artifact ID and a bounded redacted receipt exists: + +```bash +CWL_ARTIFACT_ID='' + +gh api --method GET \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/artifacts/${CWL_ARTIFACT_ID}" \ + --jq '{id,name,size_in_bytes,expired,created_at,expires_at}' + +gh api --method DELETE \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/artifacts/${CWL_ARTIFACT_ID}" +``` + +Deleting run logs is separately destructive and removes diagnostic evidence. Use it only after rotation, approval, and receipt capture: + +```bash +gh api --method DELETE \ + -H 'Accept: application/vnd.github+json' \ + "repos/${CWL_TARGET_REPO}/actions/runs/${CWL_RUN_ID}/logs" +``` + +Record who approved deletion, exact target IDs, reason, timestamp, what minimal receipt remains, and whether a legal/security hold prevented deletion. Never delete all artifacts, caches, runs, or comments with an unresolved glob or broad loop. + +## 6. Protected-main and consumer canary receipt + +Copy this template into the authorized incident/traceability record and replace every placeholder with observed evidence. `N/A` requires a reason; blanks do not prove acceptance. + +```markdown +### Operational acceptance receipt — + +- Decision timestamp and time zone: +- Incident commander: +- Security/governance approver: +- Central repository: +- Integrated protected commit: +- Trusted workflow file and source SHA: @ +- Protected-main run ID / attempt / URL: / / +- Consumer repository and PR: # +- Consumer exact source head: +- Consumer live base ref and SHA at decision: @ +- Required ruleset/check inventory source: +- Positive scenario and expected result: / +- Positive observed result: +- Negative control and expected rejection: / +- Negative observed result: +- Credential/provider class used, never value: +- Rollback target and rehearsal/result: / +- Residual risks and owners: +- Retention/deletion policy applied: +- Reopen conditions: +- Final disposition: +``` + +Acceptance is scenario-specific. Reopen when a later protected-main/consumer run contradicts the receipt, the evidence identity was wrong, a new publication path appears, or policy/configuration drift changes the trust boundary. diff --git a/docs/automation/SECURITY.md b/docs/automation/SECURITY.md new file mode 100644 index 000000000..665780cc7 --- /dev/null +++ b/docs/automation/SECURITY.md @@ -0,0 +1,99 @@ +# Security architecture and control objectives + +Status: accepted baseline +Last reviewed: 2026-08-09 + +## 1. Security objectives + +1. Untrusted pull-request content never executes with privileged workflow authority. +2. Every privileged action is bound to a validated repository, actor, PR, exact head, live base, purpose, and expected operation. +3. Checks, statuses, model results, reviews, and merge authority cannot impersonate one another. +4. Credentials use least privilege, short lifetime, explicit purpose, and minimal distribution. +5. Evidence remains diagnostically useful without publishing credentials or unbounded attacker-controlled content. +6. Supply-chain inputs are immutable or integrity-verified and fail closed on origin, TLS, pin, signature, or schema mismatch. +7. Operational incidents close only after protected-main or real-consumer proof. +8. Business-required PII stays usable under purpose-bound access and audit controls; indiscriminate masking is not the default privacy mechanism. + +## 2. Identity and authorization + +- Repository rulesets and branch protection are the final policy enforcement layer. +- Formal GitHub review identities remain distinct from automated reviewer identities. +- Dispatch workers validate canonical payload fields against live GitHub state before any write or model execution. +- OIDC/App exchanges are preferred for cross-repository authority. Audience, issuer, repository, workflow source, and requested scope must be constrained. +- `GITHUB_TOKEN` permissions are read-only by default and expanded only for a bounded job. +- A writer lease authorizes mutation of one repository/ref/expected-head tuple, not general organization writes. +- Expected-head comparisons guard source, ref, comment-resolution, branch-update, and merge mutations against time-of-check/time-of-use races. + +## 3. Secret contract + +| Secret or authority | Allowed purpose | Prohibited use | +|---|---|---| +| `NVIDIA_NIM_API_KEY` | Actual approved model calls in OpenCode/Noema/Strix or autonomous development | Checkout, deterministic tests, source materialization, anonymous/free providers, merge authority | +| OIDC request token | Exchange for one reviewed App/provider purpose | Logging, artifact retention, general shell inheritance | +| GitHub App token | Bounded target-repository reads/writes named by the exchange policy | Unrelated repositories or jobs; human-review impersonation | +| `PR_REVIEW_MERGE_TOKEN` / compatible reviewed fallback | Existing cross-repository scheduler compatibility path | Scope expansion, self-approval, unrelated product operations | +| `github.token` | Same-repository capability under explicit job permissions | Assumed cross-repository authority | +| `COPILOT_GITHUB_TOKEN` | None in autonomous development | Any autonomous development or review execution | + +New reusable workflows must declare explicit secrets. `secrets: inherit` is prohibited unless a reviewed compatibility exception enumerates why every inherited secret is safe. The present `deploy-pages.yml` example is legacy debt and must not be copied into new contracts. + +## 4. Pull-request and workflow trust + +Metadata-only `pull_request_target` entrypoints may create stable required contexts but must not execute PR-controlled code, use PR-controlled workflow files, or expose secrets. Privileged review/fix work runs from protected default-branch workflow source through a validated default-branch dispatch. PR source is either inert input or runs in a low-privilege, credential-isolated sandbox. + +Workflow source identity is captured separately from target source identity. Mutable branch references are insufficient for release or cross-repository trust where an immutable SHA is practical. + +## 5. Evidence confidentiality and integrity + +All CI-facing publication paths—stdout, stderr, timeouts, exceptions, service logs, command metadata, comments, summaries, artifacts, and structured result markers—share a credential redaction boundary. Redaction occurs before tail selection, truncation, or JSON publication so a secret outside the final tail cannot influence unsafe selection and a secret cannot be split across fields. + +The redactor protects credentials, not all sensitive business meaning. PII controls are: + +- limit collection to evidence necessary for the review or incident purpose; +- restrict repository, workflow, artifact, and log access; +- use short retention and deletion for raw evidence; +- store bounded hashes/classifications/receipts where raw content is unnecessary; +- audit access and disclosure to named operator roles; and +- keep model-provider routing consistent with repository data classification and consent. + +This preserves legitimate investigation and review while avoiding a blanket masking policy that would make enterprise workflows unusable. + +## 6. Supply-chain controls + +- Pin third-party actions and downloaded tool releases to immutable revisions or verified digests. +- Verify archive origin, redirect policy, expected member set, checksum/signature, and executable boundary before use. +- Keep source materialization separate from privileged publication and signing. +- Treat artifacts from other runs as untrusted until run, attempt, name, digest, source, and schema identity are verified. +- Generate SBOM/provenance/attestation evidence without claiming a SLSA level that has not been independently demonstrated. +- Maintain hash-locked Python requirements and equivalent ecosystem locks. +- Reject GPL/AGPL or otherwise disallowed dependencies under organization license policy. + +## 7. Model and prompt security + +PR content, comments, source, documentation, test output, and tool output are untrusted instructions. Model runners must separate system policy from repository data, constrain tool and network access, validate structured output, bind findings to exact source receipts, and prevent tool-call markup or model prose from becoming executable commands. A model verdict cannot authorize a human-only action. + +Anonymous or free models receive no unrelated credential. Private or classified repositories require explicit trusted-base policy before external model processing. Provider failure or malformed output fails the affected gate closed. + +## 8. Egress and runtime isolation + +Runners use hardened, ephemeral environments where practical. Egress is restricted or audited by job purpose. Coverage and proof commands execute with scrubbed environments, bounded time, controlled network posture, isolated temporary copies, and no persisted checkout credential. A child process cannot alter the original workspace through the sandbox copy contract. + +Resource controls and credential redaction are independent. A correct redactor does not bound child memory, total stdout capture, or service-log file growth; those require explicit quotas and tests. + +## 9. Control-framework alignment + +Official version decisions and primary references are maintained in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). This architecture can contribute bounded engineering evidence toward: + +- final NIST SSDF 1.1 practices for protected development environments, provenance, verification, and vulnerability response, while SSDF 1.2 remains informative as an initial public draft; +- NIST SP 800-204D software-supply-chain controls in CI/CD; +- final NIST SP 800-92 log lifecycle and incident-use guidance, with SP 800-92 Rev. 1 treated as an informative initial public draft; +- ISO/IEC 27001:2022 risk-based management-system concerns and ISO/IEC 27002:2022 access, logging, monitoring, supplier, and incident-control guidance; +- ISO/IEC 42001:2023 governance of AI-assisted review and human accountability; +- the AICPA 2017 Trust Services Criteria with revised points of focus (2022) for security, availability, processing integrity, confidentiality, and privacy evidence; and +- conditional Korean Cloud Security Assurance Program (CSAP) readiness questions for an actual in-scope cloud service and public-sector use. + +These mappings are not certification or attestation claims. ISO/IEC 27002 is guidance and is not itself certifiable; SOC 2 requires a scoped service description and an independent attestation engagement; Korean CSAP requires assessment of the actual cloud service under the applicable current program and legal criteria. No ISO certificate, SOC 2 report, CSAP tier/eligibility, SLSA level, or regulatory compliance follows from repository source or tests alone. Any such claim requires defined scope, control ownership, operating evidence over the required period, authorized independent assessment, and remediation of gaps in [TRACEABILITY.md](TRACEABILITY.md). + +## 10. Security change gate + +Any change to events, permissions, secrets, dispatch schema, source materialization, sandbox output, model routing, review authority, merge logic, artifact provenance, or egress requires threat-model review, negative tests, rollback, exact-head security checks, independent approval, and protected-main acceptance. diff --git a/docs/automation/TEST_STRATEGY.md b/docs/automation/TEST_STRATEGY.md new file mode 100644 index 000000000..e805ad669 --- /dev/null +++ b/docs/automation/TEST_STRATEGY.md @@ -0,0 +1,111 @@ +# Test strategy — CWL automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 + +## 1. Quality policy + +Testing must prove authority and failure behavior, not merely execute happy paths. Production changes are test-first when feasible, public Python interfaces have beginner-readable docstrings, and owned production code must reach 100% statement and branch coverage. Coverage is necessary but does not replace realistic integration, security, concurrency, compatibility, or protected-main tests. + +Every hosted result is bound to the exact head that produced it. A new commit makes predecessor results historical. + +## 2. Test layers + +| Layer | Purpose | Examples | +|---|---|---| +| Pure unit | Parsing, normalization, classification, state transitions | evidence schemas, redaction tokens, retry classes, reviewer eligibility | +| Contract | Lock workflow events, permissions, names, inputs, secrets, dispatch payloads, docs | `tests/test_*_contract.py`, source-string and YAML structure assertions | +| Adversarial/security | Challenge trust, identity, parsing, credentials, and output boundaries | replay, stale head, prompt-shaped text, ANSI/Unicode, malformed JSON, symlinks | +| Integration | Exercise scripts against realistic Git/GitHub-shaped fixtures | scheduler decisions, materializers, sandbox wrappers, artifact receipts | +| Concurrency/race | Prove expected-head and idempotency behavior | moved head/base, duplicate mention, overlapping writer, rerun attempt | +| Performance/resource | Bound attacker-influenced processing | large redaction input, deep JSON, output and file quotas | +| Compatibility | Preserve thin consumers and runtime versions | Python versions, shell syntax, Node action runtime, result/dispatch schemas | +| Operational acceptance | Demonstrate integrated behavior | protected-main central run and real product-repository canary | + +## 3. Mandatory gate inventory + +A source pull request normally requires: + +- focused regression tests for the changed boundary; +- complete repository test suite; +- 100% production statement and branch coverage; +- 100% public docstring coverage; +- compile/syntax checks for supported languages and shells; +- secret scan, Semgrep, CodeQL, dependency/OSV, Scorecard, SBOM, and applicable Strix evidence; +- documentation contract and `git diff --check`; +- current-head automated review and zero valid unresolved findings; and +- qualifying independent non-author approval under live repository policy. + +Skipped-required, neutral-required, absent, queued, cancelled, stale-head, or synthetic-only evidence is non-passing. + +## 4. Revision and authority tests + +Tests must cover: + +- source head differs from synthetic merge SHA; +- PR snapshot base differs from current live base tip; +- head/base moves after evidence collection and before write; +- check success, status success, bot approval, comment prose, dismissed approval, author approval, and human current-head approval remain distinct; +- last-push approval invalidation and unresolved threads; +- expected-head branch update, comment resolution, auto-merge, and direct merge rejection; and +- protected-main acceptance uses the integrated commit, not the feature-branch head. + +## 5. Sandbox and redaction test matrix + +| Path | Success | Failure | Timeout | Setup/launch failure | Structured output | +|---|---:|---:|---:|---:|---:| +| `sandboxed_verify.py` stdout/stderr | Required | Required | Required | Required | Required | +| `sandboxed_web_e2e.py` E2E output | Required | Required | Required | Required | Required | +| Backend/frontend service logs | Required | Required | N/A | Required | Tail metadata | +| Command metadata | Required | Required | Required | Required | Result JSON | + +Credential forms include assignments, JSON keys/values, Authorization/Cookie, private keys, JWT/provider tokens, URL userinfo, separated CLI arguments, `curl -u`, `sshpass`, MySQL/Docker password forms, nested `sh -c`/`env`, ANSI/backspace/cursor/default-ignorable splits, multiline and escaped forms, explicit allowed values, and cross-stream/result reassembly. + +Preservation assertions include exact child argv and environment semantics, ordinary diagnostic text, stdout/stderr distinction, exit code, cleanup/keep-sandbox behavior, stable result keys, valid one-line JSON, and bounded near-linear processing. Unsafe preflight/setup paths must return the documented code without output that can collide with a secret. + +Redaction tests do not claim resource safety. Separate fixtures must bound total captured output, in-memory processing, and service-log file size. + +## 6. Retry and incident tests + +For each retrying operation, tests include one accepted transient class, exhaustion, backoff/attempt count, success after transient failure, and immediate rejection of integrity, authorization, TLS, ref, schema, and product/test failures. A retry must not erase the first failing boundary or replace failure with a success-shaped skip. + +Incident tests verify symptom-to-root-cause distinction, rollback behavior, reopen criteria, and protected-main/consumer proof. + +## 7. Documentation-as-code tests + +The current documentation gate verifies structural contracts: + +- every required document and ADR exists and is indexed; +- internal index links resolve; +- Mermaid fences are balanced and critical exact-head transitions are present; +- required workflow/script names referenced by architecture exist; +- data-model entities use two-or-more-word `snake_case` names and retain the + selected many-observations-to-one-revision relationships; +- explicit secret contracts and the prohibition on `COPILOT_GITHUB_TOKEN` are present; +- traceability contains every PRD/TRD identifier and the exact source, test, + receipt, owner/target, and status columns; and +- no timeless architecture depends on an unstable literal head SHA or run ID. + +These tests do not prove Mermaid rendering, every ERD semantic, live GitHub +truth, standards correctness, or that a named operational receipt currently +exists. Rendering/review, live API/ruleset inspection, focused source tests, and +protected-main acceptance remain separate gates. The test name and PR summary +must not overstate a structural assertion as runtime proof. + +## 8. Realistic consumer acceptance + +After a central workflow repair merges: + +1. select an affected low-risk product repository; +2. run the real thin consumer at a known exact source head; +3. prove the central trusted workflow revision and expected inputs/secrets; +4. exercise a positive scenario and the relevant negative control; +5. verify GitHub check/review/status identity and diagnostic usefulness; +6. rehearse or execute rollback; and +7. record an `operational_acceptance` receipt in traceability. + +Source-branch unit success alone cannot close a fleet incident. + +## 9. Test evidence hygiene + +Fixtures use synthetic credentials and non-personal data. Logs and artifacts are bounded and short-lived. Test secrets must still be redacted because scanners and humans cannot reliably distinguish a fixture from a live capability by appearance alone. diff --git a/docs/automation/THREAT_MODEL.md b/docs/automation/THREAT_MODEL.md new file mode 100644 index 000000000..a274aac6f --- /dev/null +++ b/docs/automation/THREAT_MODEL.md @@ -0,0 +1,189 @@ +# Threat model — CWL automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 +Method: repository-scoped asset/entry-point analysis with STRIDE categories and abuse-case validation + +## 1. Overview + +`ContextualWisdomLab/.github` is the organization automation control plane. Its runtime surfaces are GitHub Actions workflows, helpers under `scripts/ci/`, organization and repository rulesets, GitHub API objects, workflow artifacts, review/model providers, and thin consumers in product repositories. It reads attacker-influenced pull-request and provider data, produces review and security evidence, and can request privileged branch, review, status, or merge operations. + +The primary security objective is to keep untrusted product source and evidence useful without allowing either to acquire trusted workflow, credential, human-review, or merge authority. This model covers the repository-wide control plane rather than one pull request or current diff. Intended controls and current implementation are not interchangeable: the exact implementation and test status for each threat is recorded in §7, and open gaps remain non-authorizing. + +Product application runtimes and their domain data are outside this repository's primary runtime. They matter here only when a central workflow receives their source, metadata, logs, credentials, or mutation authority. + +## 2. Threat model, trust boundaries, and assumptions + +### 2.1 Assets + +- protected repository refs and source history; +- branch/ruleset and required-check integrity; +- formal human review identity and decision history; +- GitHub, App, OIDC, model-provider, cloud, and deployment credentials; +- exact-head test, review, security, provenance, and operational evidence; +- product source, private repository content, business PII, and incident logs; +- runner, artifact, cache, package, and release integrity; and +- availability of shared review and merge queues. + +### 2.2 Actors + +| Actor | Assumed capability | +|---|---| +| External contributor | Controls fork/PR source, filenames, metadata, test output, and some comments | +| Compromised dependency/provider | Controls package/action/model/tool response or hosted artifact | +| Repository writer | Can push branches and influence PR content; cannot be assumed to have independent-review authority | +| Automated reviewer/bot | Can publish configured checks, statuses, comments, or formal bot reviews | +| Independent human reviewer | Can submit qualifying review under repository policy | +| Organization operator | Configures rulesets, secrets, Apps, runners, and required workflows | +| Concurrent automation writer | Can mutate a branch unless leases and expected-head checks prevent collision | + +### 2.3 Trust boundaries + +| Boundary | Less-trusted side | More-trusted side | Security invariant | +|---|---|---|---| +| PR head to trusted workflow | Fork/source tree, filenames, metadata, test behavior | Protected default-branch workflow and write credential | A privileged event never checks out or executes PR-controlled code; source is inert data or runs without privileged credentials. | +| Dispatch caller to central worker | Actor claims and payload fields | Protected `repository_dispatch` worker | Validate actor, repository, operation, PR, source head, live base, visibility, and replay identity against live GitHub state before privilege or model use. | +| Artifact/run to evidence consumer | Logs, archives, result markers, run metadata | Gate normalizer and scheduler | Bind repository, workflow source, run ID/attempt, artifact name/digest, schema, and exact revision; never trust a success-shaped payload alone. | +| Provider/model to review gate | Model text, tool output, provider status | Normalized finding/review evidence | Treat output as untrusted data, validate its schema and source receipt, and never promote it to human or merge authority. | +| Sandbox to public evidence | Child stdout/stderr, commands, exceptions, service logs | CI log, summary, comment, artifact, result JSON | Canonicalize and redact the complete boundary before truncation/serialization while preserving bounded ordinary diagnosis. | +| Scheduler to target repository | Proposed mutation and credential | Protected ref, review, status, or merge API | Require least privilege, branch-local ownership, current policy, exact expected head, and a last-moment live-state refresh. | +| Operator configuration to runtime | Rulesets, secrets, Apps, allowlists, workflow toggles | Organization fleet | Configuration is privileged change, not trusted merely because it exists; audit scope, issuer, review, and protected-main acceptance. | + +### 2.4 Assumptions + +- GitHub correctly authenticates API actors and preserves commit, review, check, and run identities, while configuration drift and compromised credentials remain in scope. +- Protected default-branch workflow source is trusted only after required current-head checks and qualifying review; a mutable branch name alone is not an immutable identity. +- Every API read is a time-bounded observation. A writer re-fetches live head, base, policy, and ownership state immediately before mutation. +- Credential values are provisioned outside source control, can be revoked, and are not legitimate diagnostic evidence. Availability of a credential does not expand its documented authority. +- Product source, PR prose, comments, logs, artifacts, package metadata, and provider/model responses can be malicious even when their transport is authenticated. +- Automated identities cannot become qualifying human reviewers through wording, token choice, or a success-shaped result. +- GitHub remains the durable evidence system of record subject to its configured retention; no separate evidence database is assumed. +- A fully compromised organization owner or GitHub control plane can override repository controls. The repository can reduce, detect, and reconstruct that risk but cannot cryptographically prevent the platform owner from exercising platform authority. + +### 2.5 Input control + +| Control class | Inputs | Required treatment | +|---|---|---| +| Attacker-controlled | PR/fork source, paths, refs, metadata, issue comments, child output, service logs, untrusted artifacts, source archives, model/tool text | Parse as data; bound size/time; canonicalize; validate schemas and identities; isolate execution; redact before publication. | +| Developer-controlled | Central workflow/script changes, prompts, action pins, lockfiles, tests, documentation | Treat as proposed code until protected review and exact-head gates pass; use immutable dependency identities and consumer canaries. | +| Operator-controlled | Rulesets, required workflows, Apps, OIDC audiences, secrets, allowlists, reruns, disables, rollback decisions | Require named authority, least privilege, change receipt, rollback, and post-change audit; never infer approval from operator access alone. | +| Platform/provider-controlled | GitHub API responses, OIDC claims, runner images, registries, model/provider responses | Authenticate transport and issuer, bind freshness and integrity, validate content, classify outages, and retain independent deterministic/human authorities. | + +## 3. Attack surface, mitigations, and attacker stories + +### 3.1 Entry points + +PR events; dispatch payloads; workflow-call inputs; issue comments; scheduled sweeps; workflow artifacts; source archives; package registries; model responses; CLI arguments and environment allowlists; child stdout/stderr; service logs; GitHub API responses; review comments; status/check publication; and merge/fix APIs. + +### 3.2 Threat and control matrix + +| ID | STRIDE | Threat / realistic path | Primary controls | Residual risk / required evidence | +|---|---|---|---|---| +| TM-01 | Information disclosure | Child process prints token in stdout, timeout bytes, exception, service tail, command, or JSON result | Shared canonical redactor on every publication path; explicit sensitive values; no-output unsafe preflight | Unknown token formats and unbounded raw service files; adversarial regression and retention controls required | +| TM-02 | Elevation / disclosure | Broad `secrets: inherit` gives a reusable workflow unrelated credentials | Explicit secret contracts; job-level least privilege; migrate legacy deploy caller | Legacy caller remains inventory debt until replaced | +| TM-03 | Spoofing / elevation | Attacker sends a `repository_dispatch` payload naming another repo/head or replays a mention | Actor/repo allowlist, canonical payload hash, live PR comparison, exact-name replay ledger | Upstream identity-provider compromise; inspect issuer/audience and token issuance logs | +| TM-04 | Tampering | Stale check/review from predecessor head is reused after push | Exact commit/run binding, live refresh, last-push approval policy, expected-head mutation | GitHub policy configuration drift; audit rulesets continuously | +| TM-05 | Elevation | Automated model review is treated as qualifying independent human approval | Separate evidence classes; author/reviewer identity checks; GitHub aggregate review decision | Misconfigured ruleset or privileged token bypass; ruleset audit and negative merge tests | +| TM-06 | Elevation / tampering | `pull_request_target` checks out or executes malicious PR code with secrets/write token | Metadata-only bootstrap; privileged default-branch dispatch; low-privilege sandbox | Future workflow edit can reintroduce path; contract tests and CodeQL/Semgrep | +| TM-07 | Tampering / spoofing | Prompt injection makes reviewer call tools, reveal secrets, or emit success-shaped output | Treat source/comments as data; constrained tools/egress; structured output validation; source receipts | Model/tool vulnerabilities; independent deterministic gates and human review remain mandatory | +| TM-08 | Tampering | Compromised action/package/archive or mutable workflow ref changes executed code | Immutable pins, hashes/signatures, origin/redirect/member validation, trusted workflow SHA | Compromise of trusted publisher or signing root; provenance and rotation plan | +| TM-09 | Information disclosure | Anonymous/free provider receives private source or inherited provider keys | Trusted-base data-classification policy; provider-scoped environment; no credential on anonymous path | Incorrect classification; require accountable owner and negative private canary | +| TM-10 | Denial of service | Huge or adversarial log triggers quadratic redaction, memory growth, or service-file exhaustion | Linear parsing tests, timeouts, tail bounds after redaction | Total capture/file quotas are not fully solved by redaction; separate resource-limit work required | +| TM-11 | Tampering | Concurrent writer overwrites branch, resolves wrong thread, or merges moved head | Branch-local lease, last-moment refresh, expected-head API, no force-push | Actors outside lease protocol; GitHub protections and conflict detection | +| TM-12 | Repudiation | Merge or incident closure cannot be tied to exact evidence and operator | Run/attempt/revision receipts, formal review records, traceability, protected-main acceptance | Retention expiry; keep minimal durable receipt and hashes | +| TM-13 | Information disclosure | Per-item PII masking is removed and broad logs expose business content | Purpose-bound access, minimal evidence scope, short retention, audit, repository classification | Authorized-user misuse and mosaic inference; periodic access/retention review | +| TM-14 | Tampering | Retry converts integrity/auth/TLS/ref/schema failure into eventual green | Classified retry allowlist; permanent classes fail immediately | Misclassification; regression fixtures for every class and observable attempt ledger | +| TM-15 | Tampering / denial | Rollback restores vulnerable workflow or closes incident without consumer proof | Versioned rollback target, security review, reopen criteria, protected-main canary | Emergency rollback time pressure; require post-incident reconciliation | + +### 3.3 Attack paths requiring explicit tests + +#### 3.3.1 Credential evidence escape + +1. A contributor causes a test or service to print a credential using JSON, ANSI/default-ignorable splits, multiline headers, nested shell commands, URL userinfo, or cross-stream fragments. +2. A wrapper captures or tails the text. +3. A partial redactor misses the form or redacts after tail selection. +4. The credential appears in a public log, summary, comment, artifact, or result marker. + +Required counterevidence: complete-boundary redaction tests for completed, timeout, exception, command, service, and structured output; valid JSON and ordinary diagnostics preserved; fixed linear-time fixtures. + +#### 3.3.2 Confused-deputy dispatch + +1. A caller submits valid-looking repository, PR, head, base, and requested action fields. +2. A central worker trusts payload identity without live comparison. +3. The worker exchanges a privileged token and writes to the victim repository or branch. + +Required counterevidence: actor and target allowlists; immutable default-branch worker; canonical schema; live repo/PR/head/base/ref comparison; exact invocation ledger; expected-head write. + +#### 3.3.3 Reviewer spoofing + +1. A bot publishes `APPROVED`, a success status, or approval-like prose. +2. Scheduler logic counts it as the required independent review. +3. A bypass-capable credential merges despite GitHub review state. + +Required counterevidence: formal review source, exact commit, non-author/non-bot eligibility, aggregate GitHub review decision, ruleset, and negative merge-path fixtures. + +### 3.4 Out-of-scope attacker stories + +These stories do not become repository findings without an additional repository-controlled failure: + +- compromise of GitHub's global control plane, an external provider's root signing authority, or a maintainer workstation with already-authorized organization-owner access; +- a fully authorized organization owner intentionally removing protections and audit history, although ruleset drift detection and external audit remain defense in depth; +- vulnerabilities in a product application's runtime that no central workflow executes, publishes, mutates, or supplies with authority; +- disclosure of intentionally public source to a provider explicitly approved for that public repository, absent credential, policy, or consent failure; and +- availability loss caused only by a declared external outage when the control plane fails closed and continues unrelated safe work. + +Output-memory and service-file exhaustion are not out of scope, but they are explicitly unresolved by credential redaction alone and remain TM-10 work. Certification or regulatory compliance is also not inferred from this threat model. + +## 4. Severity calibration + +Severity depends on reachable authority, affected repositories, data classification, persistence, and whether GitHub protection or independent review still blocks impact. + +| Severity | Repository-context examples | +|---|---| +| Critical | Attacker-controlled PR code executes with a write/secret-bearing trusted workflow; a confused deputy writes or merges a protected ref in another repository; a live organization credential is published with practical fleet-wide write authority. | +| High | Current-head required evidence can be forged into a false green; private source/PII is sent to an unauthorized provider; a compromised action or archive executes in a privileged job; a writer bypasses expected-head protection and corrupts a shared branch. | +| Medium | One repository's review/merge queue is persistently denied without a privilege or confidentiality breach; bounded sensitive business context is exposed only to an unintended but already authenticated audience; missing receipts prevent incident reconstruction while protected mutation remains blocked. | +| Low | Documentation, non-authorizing telemetry, or diagnostics drift without changing a trust boundary; a bounded optional provider failure has an accurate non-passing result and safe continuation path. | + +A hypothetical injection without attacker control of the input, a stale record that is rejected before authorization, or a provider outage that remains visibly non-passing is not promoted merely because the vulnerability class could be severe elsewhere. + +## 5. Privacy position + +The system does not solve privacy by masking every name, email address, or business fact. That would erase the context required for investigation and review. Instead it limits who can access raw evidence, for what purpose, how long, and through which classified provider. Credential redaction remains mandatory because credentials are capability-bearing and never legitimate diagnostic content. + +## 6. Residual risks and owners + +| Risk | Owner | Closure evidence | +|---|---|---| +| Total stdout/stderr and service-log growth | Sandbox/output-limit owner | Bounded memory/file tests and protected-main hostile-output run | +| Legacy inherited deploy secrets | Deploy workflow owner | Explicit secret migration and real deployment canary | +| Reviewer availability | Organization governance owner | Eligible reviewer pool and counted exact-head approval receipt | +| Provider and GitHub outage | Platform operator | Queue-age telemetry, deferred work, and recovery run | +| Ruleset drift | Organization administrator | Scheduled ruleset audit and remediation receipt | +| Private-source model classification | Repository data owner | Trusted-base policy, negative control, provider-specific egress evidence | + +Threats are reopened whenever source, identity, permissions, provider, retention, or deployment topology changes. + +## 7. Exact source and test traceability + +Paths below are the current concrete control and regression locations. A listed path is not proof that every branch is implemented; the status column names material gaps. + +| Threat | Exact control source | Exact regression evidence | Current status | +|---|---|---|---| +| TM-01 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Protected integration and consumer proof tracked by PR #842. | +| TM-02 | `.github/workflows/deploy-pages.yml`; `docs/automation/SECURITY.md` | `tests/test_automation_documentation_contract.py` | Partial: documentation guard only; explicit-secret migration and a runtime negative contract remain open. | +| TM-03 | `.github/workflows/agent-mention-router.yml`; `.github/workflows/opencode-review-dispatch.yml`; `scripts/ci/agent_mention_router.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_agent_mention_complete_payload_binding.py`; `tests/test_agent_mention_idempotency.py`; `tests/test_agent_mention_workflow_contract.py`; `tests/test_opencode_security_boundaries.py` | Implemented for named paths; every new dispatch type needs equivalent binding. | +| TM-04 | `scripts/ci/pr_head_replay_guard.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_head_replay_guard.py`; `tests/test_pr_review_merge_scheduler.py` | Implemented with live-ruleset dependency. | +| TM-05 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/opencode_existing_approval_gate.py`; `scripts/ci/noema_review_gate.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_opencode_existing_approval_gate.py`; `tests/test_noema_review_gate.py` | Implemented with live reviewer/ruleset configuration dependency. | +| TM-06 | `.github/workflows/opencode-review.yml`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/security-scan.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_review_execution_contracts.py`; `tests/test_required_workflow_queue_contract.py` | Implemented for inventoried required entrypoints; workflow inventory must remain current. | +| TM-07 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/opencode_adversarial_receipts.py`; `scripts/ci/review_execution_contracts.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_review_execution_contracts.py` | Defense in depth; deterministic and human gates remain mandatory. | +| TM-08 | `scripts/ci/compile_opencode_review_lock.sh`; `scripts/ci/materialize_base_python_requirements.py`; `.github/workflows/trusted-uv-materializer-quality-ci.yml` | `tests/test_trusted_uv_download_contract.py`; `tests/test_uv_redirect_boundary.py`; `tests/test_strix_workflow_dependency_hashes.py` | Implemented for covered toolchains; new ecosystems require inventory. | +| TM-09 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_noema_review_gate.py`; `tests/test_opencode_security_boundaries.py` | Partial: routing is tested; repository data-owner consent/classification remains governance evidence. | +| TM-10 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Partial: algorithmic fixtures exist; total capture and service-file quotas remain open. | +| TM-11 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/pr_review_fix_scheduler.py`; `scripts/ci/pr_auto_rebase.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py`; `tests/test_pr_auto_rebase.py` | Partial: expected-head guards exist; no single persisted fleet writer-lease service. | +| TM-12 | `scripts/ci/opencode_dispatch_status.py`; `scripts/ci/agent_mention_router.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_agent_mention_receipt_authority.py`; `tests/test_agent_mention_artifact_ledger.py`; `tests/test_pr_head_replay_guard.py` | Partial: receipts are distributed and retention can expire them. | +| TM-13 | `docs/automation/SECURITY.md`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_automation_documentation_contract.py` | Partial: public/private routing and policy are checked; no complete access/retention audit gate. | +| TM-14 | `scripts/ci/run_opencode_review_model_pool.sh`; `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/strix_model_utils.sh` | `tests/test_opencode_model_pool_runner.py`; `tests/test_pr_review_merge_scheduler.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | Implemented for covered retry loops; each new retry path requires classification fixtures. | +| TM-15 | `docs/automation/RUNBOOK.md`; `docs/automation/INCIDENT_RUNBOOK.md`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_automation_documentation_contract.py` | Partial: guarded merge tests exist; fleet rollback rehearsal and consumer receipt remain operational evidence. | + +Cross-document requirement status remains in [TRACEABILITY.md](TRACEABILITY.md). Incident commands and evidence templates are in [RUNBOOK.md](RUNBOOK.md). diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md new file mode 100644 index 000000000..0cd8e438f --- /dev/null +++ b/docs/automation/TRACEABILITY.md @@ -0,0 +1,65 @@ +# Requirements and evidence traceability + +Status: living authoritative matrix +Last reviewed: 2026-08-09 + +Status values: **implemented**, **partial**, **proposed**, or **operational-proof-pending**. A row marked implemented still requires fresh exact-head evidence for each change. + +## 1. Product and technical requirements + +Receipt names describe the required evidence shape; they are not claims that a +particular transient run is current. A mutation must re-fetch the live receipt. + +| Requirement | TRD / ADR | Exact implementation source | Exact tests | Required gate / receipt authority | Owner / closure target | Status | +|---|---|---|---|---|---|---| +| `PRD-01` fresh live state | `TRD-EVT-01`, ADR-0007 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_agent_mention_sweep.py` | `scan-pr-queue` plus live PR/ruleset API snapshot | automation maintainer / every decision | implemented | +| `PRD-02` exact head + live base | `TRD-REV-01`, ADR-0002/0014 | `scripts/ci/pr_head_replay_guard.py`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_head_replay_guard.py`; `tests/test_opencode_review_context.py`; `tests/test_pr_review_fix_scheduler.py` | exact-head Check Runs and expected-head mutation response | workflow owners / before every write | partial: path inventory remains distributed | +| `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | implemented with live-ruleset dependency | +| `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; `.github/workflows/noema-review.yml` | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_noema_review_gate.py` | live ruleset requires two approvals, stale dismissal, last-pusher separation | repository admins / before merge | implemented; human capacity remains external | +| `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | partial: no shared persisted lease service | +| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; external hourly prompt contract | `tests/test_required_workflow_queue_contract.py`; external continuation-ledger receipt | 15-minute queue sweep plus two fresh whole-queue sweeps in the hourly run | automation operator / before run exit | partial: external orchestrator spans GitHub | +| `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py` | source-backed finding/RCA and protected recovery receipt | incident owner / before closure | partial: procedural and review control | +| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result marker | sandbox owners / `.github#842` integration + canary | operational-proof-pending | +| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/**`; root `ARCHITECTURE.md`; repository entrypoints | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link audit, `git diff --check`, reviewer approval | architecture maintainers / current documentation PR | proposed | +| `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/**`; `docs/automation/RUNBOOK.md` receipt template | workflow-specific contracts and real consumer canary | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | partial: receipts remain distributed | +| `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | required workflows; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; workflow contract suites | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | partial: implicit-v1 dispatch and inherited-secret legacy remain | +| `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | partial: new routes require audit | +| `PRD-13` PII alternative controls | `TRD-SEC-01`, `TRD-RET-01` | `docs/automation/SECURITY.md`; workflow access and artifact retention settings | privacy/access review; no complete automated gate | purpose/audience/retention/deletion/access receipt | data/service owner / before processing business PII | proposed/partial | +| `PRD-14` quality/readability | verification §12, ADR-0012 | `pyproject.toml`; pinned requirement/hash manifests; owned Python/workflow source | complete `tests/`; compile/syntax; coverage; interrogate | exact-head tests, 100% owned production statement/branch and public docstrings | change author / before PR readiness | implemented policy; fresh proof required | + +## 2. Documentation coverage + +| Artifact | Required content | Machine gate | Status | +|---|---|---|---| +| PRD | users, two modes, outcomes, degraded behavior, acceptance/non-goals | file/index/term contract | added in documentation PR | +| TRD | events, revisions, evidence authority, permissions, retries, leases, redaction | file/index/source-name contract | added in documentation PR | +| Architecture | contexts, components, planes, trust/failure boundaries | Mermaid and source-name contract | added in documentation PR | +| Data model / ERD | conceptual vs persisted and all evidence/governance entities | entity/naming/cardinality/link contract | added in documentation PR | +| UML | component, two sequences, state, authority, topology, retry, sandbox | diagram-section/fence contract | added in documentation PR | +| Security / threat model | required attack paths, privacy alternative, residual risk | term/link contract | added in documentation PR | +| Test strategy | realistic gate, security, performance, consumer proof | term/link contract | added in documentation PR | +| Operability / runbook | SLI/SLO, queue/provider failures, exact queries, rollback, retention, receipt, closure/reopen | term/link contract | added in documentation PR | +| ADR set | sixteen indexed decisions with alternatives, tests, rollback, supersession | index/file/section contract | added in documentation PR | + +## 3. Redaction incident lineage + +- `ContextualWisdomLab/.github#841` is the closed, unmerged Jules/Sentinel branch. It established the original disclosure concern but mixed an unrelated SSRF slice and omitted publication paths; it is historical evidence, not the integration vehicle. +- `ContextualWisdomLab/.github#842` is the clean current-main replacement for the sandbox evidence-redaction slice. It owns completed/timeout/exception output, service tails, command/result metadata, explicit allowed values, canonicalization, structural JSON handling, diagnostic preservation, and exhaustive hostile forms. +- Output-memory and service-file quotas remain a separate resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. +- Every head change on #842 invalidates predecessor test counts, coverage counts, reviews, and run IDs. The PR body and redaction doctoring must be updated from the final exact-head evidence before merge. + +## 4. Known gaps and next evidence + +| Gap | Risk | Next bounded evidence | +|---|---|---| +| No persisted cross-repository continuation/writer ledger | collision and queue-starvation reconstruction | decide whether GitHub Project/artifact state is sufficient before proposing persistence | +| SLI aggregation incomplete | buyer cannot quantify reliability | emit bounded run metrics keyed by repo/head/workflow/attempt | +| Sandbox total-output/service-file quota incomplete | memory/disk DoS | fail-first hostile-output quota tests and separate implementation PR | +| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | explicit secret interface plus real deployment positive/negative canary | +| Operational receipts distributed | incident closure hard to audit | dated protected-main/consumer receipt index | +| Project #1 requires GraphQL/`gh` project scope | agents without that capability cannot acquire visible project item | add supported connector or ensure native PR auto-add; never invent state | +| Master context current-state section is dated 2026-07-08 | stale operational narrative | separate timeless context from generated/daily live-state appendix | + +## 5. Standards traceability + +Research status, normative version choices, and APA 7 references are maintained in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). In particular, NIST SSDF 1.1 remains the final normative baseline while SSDF 1.2 is an initial public draft; SLSA 1.2 is the current SLSA specification at this review date. diff --git a/docs/automation/TRD.md b/docs/automation/TRD.md new file mode 100644 index 000000000..cf58af56d --- /dev/null +++ b/docs/automation/TRD.md @@ -0,0 +1,230 @@ +# Technical requirements — CWL automation control plane + +Status: accepted baseline +Last reviewed: 2026-08-09 + +## 1. System boundary + +The control plane consists of trusted workflows in `.github/workflows/`, their helpers in `scripts/ci/`, organization/repository rulesets, GitHub API state, and thin caller workflows. The hourly maintainer policy selects work but is not an evidence authority. Product source and tests remain in each product repository. + +Normative keywords **MUST**, **MUST NOT**, **SHOULD**, and **MAY** describe intended contracts. [TRACEABILITY.md](TRACEABILITY.md) identifies contracts that are not yet fully enforced. + +### Requirement identifiers + +| ID | Normative boundary | +|---|---| +| `TRD-EVT-01` | Trigger trust and live dispatch validation (§2) | +| `TRD-REV-01` | Exact source, live base, workflow, run, and merge identity (§3) | +| `TRD-AUTH-01` | Evidence-class and merge-authority separation (§4) | +| `TRD-RUN-01` | Work-conserving lifecycle and double exit sweep (§5) | +| `TRD-WRITE-01` | Branch-local writer lease and expected-head mutation (§6) | +| `TRD-SEC-01` | Least permission, explicit secret purpose, and PII controls (§7) | +| `TRD-RETRY-01` | Classified bounded retry and timeout behavior (§8) | +| `TRD-LOG-01` | Complete-boundary sandbox redaction and diagnostic preservation (§9) | +| `TRD-IF-01` | Versioned checks, dispatches, workflow calls, results, and receipts (§10) | +| `TRD-RET-01` | Purpose-limited evidence retention and deletion (§11) | + +## 2. Trigger and execution semantics + +| Trigger | Intended use | Trust requirement | +|---|---|---| +| `pull_request_target` | Metadata-only required contexts and trusted base-branch policy | MUST NOT check out or execute PR-controlled source with write credentials or secrets. | +| `pull_request` | Unprivileged exact-head source tests | Fork restrictions and read-only credentials MUST remain effective. | +| `repository_dispatch` | Default-branch privileged review, fix, retry, or agent invocation | Payload, actor, repository, PR number, head/base identity, and allowed operation MUST be validated against live state. | +| `workflow_run` | Re-evaluate after a named upstream workflow | Artifacts and reported SHAs MUST be treated as untrusted until rebound to live PR identity. | +| `workflow_call` | Stable thin consumer contract | Inputs and secrets MUST be explicit, minimal, versioned, and documented. | +| `schedule` | Queue sweep, full-tree security scan, inventory, or recovery | MUST use protected default-branch source and idempotent bounded work. | +| `issue_comment` | Authorized agent invocation in repositories where a default-branch router exists | Author association, exact comment identity, canonical payload, and replay ledger MUST be validated. | + +The event cadence of GitHub workflows is independent of the hourly commercial-maintenance cadence. A scheduled maintainer run MUST derive actions from live GitHub state; it MUST NOT infer acceptance from its recurrence or memory. + +## 3. Revision identity + +Each decision record MUST distinguish: + +- `source_revision`: the exact immutable PR head commit under review; +- `base_revision_snapshot`: the PR API's recorded base SHA for historical context; +- `live_base_revision`: the current protected base-ref tip resolved independently at decision time; +- `workflow_revision`: the immutable trusted workflow/script revision; +- `run_id` and `run_attempt`: workflow execution identity; and +- `merge_revision`: the integrated or merge-group revision when applicable. + +An exact-head result proves only the source revision it names. It does not prove compatibility with a later base tip. A synthetic merge result MUST be labeled as such and MUST NOT silently substitute for source-head evidence. Any source-head change invalidates predecessor checks, reviews, and approvals unless GitHub policy explicitly and authoritatively preserves a still-applicable approval. + +## 4. Evidence taxonomy and authority + +| Evidence class | Examples | May establish | May not establish | +|---|---|---|---| +| `check_evidence` | GitHub Check Run, required workflow job | Deterministic gate outcome for its named revision | Human approval or merge authority | +| `status_evidence` | Commit status context, CodeRabbit status | External service signal for its named commit | Formal review or branch-protection satisfaction unless configured as required | +| `review_evidence` | Formal `APPROVED`, `CHANGES_REQUESTED`, inline thread | Reviewer decision, author, commit, and thread state | Test success or scanner execution | +| `model_evidence` | OpenCode, Noema, Strix finding/output | Advisory analysis or a configured machine gate | Qualifying human identity | +| `workflow_evidence` | Run, job, attempt, artifact receipt | Execution provenance and result | Correct source binding without validation | +| `dependency_evidence` | Upstream PR/release/attestation | Dependency state | Current consumer compatibility without consumer proof | +| `operational_acceptance` | Protected-main or real-consumer run | Runtime closure for a stated scenario | Unrelated scenarios or future revisions | + +Merge authorization MUST be the intersection of actual repository policy, required current evidence, current mergeability, zero actionable unresolved threads, expected-head semantics, and qualifying reviews. No single evidence producer may redefine another authority class. + +## 5. Work-conserving run lifecycle + +1. Refresh repository, pull-request, issue, exact revision, gate, ruleset, and writer state. +2. Build separate lanes for mergeable PRs, defects, operations, issues, documentation, automation drift, and bounded product gaps. +3. Select the highest-value safe item. +4. For a defect, establish RED evidence before production repair when feasible. +5. Identify symptom, immediate cause, root cause, owner, distinct remedies, and feasibility. +6. Acquire or confirm a branch-local writer lease immediately before mutation. +7. Apply the smallest cohesive change; preserve unrelated user and writer changes. +8. Run focused and complete verification; bind hosted evidence to the new exact head. +9. Merge only under repository policy; then run protected-main or consumer acceptance when required. +10. Update authoritative documentation and immediately choose the next item. +11. Before stopping, perform a fresh whole-queue sweep. If it finds work, execute it and sweep again. + +Pending checks, reviewer latency, rate limits, or provider cooldowns are deferred states, not whole-run blockers. + +## 6. Writer lease and mutation contract + +A `writer_lease` is branch-local and identifies repository, branch/ref, expected head, actor, scope, acquisition time, and expiry/heartbeat. Immediately before a source, ref, or PR-state write, the writer MUST re-fetch the target head/base and relevant writer state. Movement outside the expected lineage aborts the write. + +Writers MUST NOT force-push shared history, manufacture one-shot write workflows, synthesize approval, weaken required gates, or race another write-capable actor. Read-only review and checks do not themselves constitute a writer conflict. A blocked branch MUST cause rotation to another non-conflicting lane when useful work exists. + +## 7. Permissions and secrets + +- Workflow and job `permissions` MUST default to read-only and be expanded only at the step/job that requires the capability. +- Cross-repository writes SHOULD use short-lived OIDC/App authority with audience and repository restrictions. +- Long-lived tokens are compatibility fallbacks, not the preferred architecture; their names and scopes MUST remain stable until a reviewed migration. +- Reusable workflows MUST declare explicit secret contracts. Blanket `secrets: inherit` MUST be removed where the callee needs fewer secrets. The current `deploy-pages.yml` inherited-secret caller pattern is a documented legacy exception and not a template for new workflows. +- `NVIDIA_NIM_API_KEY` MAY enter only model-execution jobs. It MUST NOT be supplied to deterministic tests, source materialization, checkout, artifact inspection, or anonymous/free model execution. +- `COPILOT_GITHUB_TOKEN` MUST NOT be used for autonomous development. +- Tokens and allowed environment values MUST be registered with the evidence redactor before any potentially colliding output is emitted. +- PII is not a generic secret token. Business-required PII MUST be protected by access, purpose, audience, retention, and audit controls rather than indiscriminate content masking. + +## 8. Failure classification, retry, and timeout + +| Class | Examples | Retry contract | +|---|---|---| +| Transient infrastructure | bounded DNS failure, connection reset, GitHub 5xx, runner acquisition failure | MAY retry with a small bounded count, backoff/jitter, total time budget, and preserved final evidence. | +| Provider capacity | rate limit, documented temporary capacity exhaustion | MAY defer or rotate provider; MUST NOT convert exhaustion to approval. | +| Integrity/authentication | checksum mismatch, invalid signature, OIDC audience mismatch, bad token | MUST fail immediately and closed; no speculative retry with broader authority. | +| Authorization/policy | 401/403, disallowed actor/repository, missing required approval | MUST fail/defer with exact prerequisite; MUST NOT invent authority. | +| Revision/ref/schema | head moved, ref missing, malformed payload, unsupported schema | MUST fail immediately, refresh state, and re-plan. | +| TLS/supply chain | certificate failure, immutable pin mismatch, unexpected redirect/origin | MUST fail immediately and closed. | +| Product/test defect | assertion, compile, lint, coverage, behavior failure | Requires RCA and source repair; rerun alone is not remediation. | + +Every retry loop MUST state attempt count, per-attempt timeout, total budget, retryable classification, and final failure output. Long model reviews MAY receive hours when accuracy requires it, but a running model job cannot reserve the maintenance invocation. + +## 9. Evidence output and sandbox redaction + +Evidence publication includes human stdout/stderr, timeout payloads, exceptions, service log tails, command metadata, job summaries, comments, artifacts, and structured result markers. Each publication path MUST use the same canonical redaction boundary before truncation or serialization. + +The redactor MUST: + +- detect supported credential assignments, authorization/cookie forms, private-key material, provider tokens, JWT-like values, URL userinfo, explicit allowed values, and separated/nested command options; +- canonicalize terminal/Unicode control evasions before detection; +- parse supported JSON structurally, preserve benign keys/types, and redact sensitive string values without corrupting valid JSON; +- preserve ordinary diagnostic text, stream identity, exit code, stable result keys, and valid one-line result JSON; +- prevent cross-stream, marker, and result-boundary reassembly of an allowed value; +- reject unsafe short, whitespace-only, non-printable/control-bearing, + marker-colliding, or wrapper-owned allowed values before execution without + emitting colliding evidence; printable single-line values may contain spaces; +- process attacker-influenced text in bounded linear or near-linear time; and +- redact a complete service log before selecting the bounded tail. + +If argument parsing or sandbox setup fails before a safe redaction context exists, the wrapper MUST emit no attacker-controlled or credential-bearing evidence and MUST return the documented setup-failure code. Output-memory and service-file quotas are separate requirements and are not implied by redaction alone. + +## 10. Stable interfaces + +Stable external interfaces include required check names, formal review semantics, reusable workflow input/secret names, repository-dispatch payload schemas, result-marker schemas, exit-code meanings, artifact receipt formats, and documented recovery commands. A breaking change requires a versioned bridge, consumer inventory, migration plan, rollback, and independently verified consumer acceptance. + +Internal helper layout may change without a consumer migration when those stable interfaces and authority boundaries are preserved. + +### 10.1 Required check and review names + +| Interface | Stable emitted name | Defining source | +|---|---|---| +| OpenCode bootstrap | `required-workflow-bootstrap` | `.github/workflows/opencode-review.yml` | +| Source-tree sentinel | `coverage-source-tree` | `.github/workflows/opencode-review.yml` | +| Coverage evidence | `coverage-evidence` | `.github/workflows/opencode-review.yml` | +| OpenCode review check | `opencode-review` | `.github/workflows/opencode-review.yml` | +| Noema review check | `noema-review` | `.github/workflows/noema-review.yml` | +| Strix security check | `strix` | `.github/workflows/strix.yml` | +| Merge scheduler check | `scan-pr-queue` | `.github/workflows/pr-review-merge-scheduler.yml` | + +The live ruleset inventory determines which names are required for a target. +Renaming an emitted name is a breaking consumer/ruleset change. A formal review +submission is not interchangeable with the same-named Check Run. + +### 10.2 Repository-dispatch schemas + +Current payloads are legacy implicit version 1: they do not carry a +`schema_version` field. Senders and receivers MUST therefore reject unknown or +missing required fields, validate live identity, and treat adding a new required +field or changing a type as breaking. The next breaking revision MUST add an +explicit version and a compatibility bridge. + +| Event type | Required fields and types | Optional fields | Receiver and rejection | +|---|---|---|---| +| `opencode-review` | `target_repository: owner/name`, `pr_number: positive integer`, `pr_base_ref: string`, `pr_base_sha: 40-hex`, `pr_head_ref: string`, `pr_head_sha: 40-hex` | none in the security identity | `opencode-review-dispatch.yml`; reject unauthorized actor/target, malformed identity, fork, closed PR, or any live mismatch. | +| `strix-scan` (PR scope) | same repository/PR/base/head identity as `opencode-review` | `strix_llm: allowlisted string` | `strix.yml`; reject incomplete/malformed metadata, unallowlisted model, or live mismatch. | +| `noema-review` | `target_repository: owner/name`, `pr_number: positive integer` | none | `noema-review.yml`; resolve the live PR/head and fail rather than trust caller-supplied review evidence. | +| `pr-review-autofix` | `target_repository`, `pr_number`, `pr_base_ref`, `pr_base_sha`, `pr_head_ref`, `pr_head_sha` with the types above | `resolve_conflict: boolean-like string`, default false | `pr-review-autofix.yml`; same-repository mutable head only; reject every base/head mismatch before checkout and push. | +| `merge-scheduler` / `pr-review-fix-scheduler` | `target_repository: owner/name` for cross-repository dispatch | bounded scheduler fields matching the `workflow_call` names/types below | corresponding scheduler workflow; invalid type/range/target fails before mutation. | +| `agent-mention-noema` / `agent-mention-opencode` | `target_repository`, `pr_number`, `pr_head_sha`, `pr_base_sha`, `base_branch`, `requested_agent`, `agent_invocation_key`, `requested_by`, `source_comment_id` | OpenCode-only fixed control fields: `trigger_reviews`, `review_dispatch_limit`, `enable_auto_merge`, `update_branches`, `merge_mode` | mention dispatch workflows and `agent_mention_router.py`; exact agent/key/comment/head claim and artifact receipt must agree. | + +### 10.3 Reusable workflow inputs and secret classes + +| Interface | Inputs | Secret contract | +|---|---|---| +| `pr-review-merge-scheduler.yml` | `dry_run:boolean`; string-encoded bounded `max_prs`, `pr_number`, `review_dispatch_limit`, `branch_update_limit`, `stale_opencode_minutes`; `trigger_reviews:boolean`; `enable_auto_merge:boolean`; `update_branches:boolean`; `merge_mode:{direct_or_auto,auto,direct,disabled}`; `project_flow`, `base_branch` strings | Caller secrets are not declared; jobs use the scoped workflow token and configured mutation/App fallbacks only where needed. | +| `pr-review-fix-scheduler.yml` | `dry_run:boolean`; bounded string `max_prs`, `max_dispatches`, `retry_hours`; `target_repository`, `autofix_workflow`, `autofix_repository`, `base_branch`, `canonical_ref` strings | Mutation fallback is limited to `PR_REVIEW_MERGE_TOKEN` or `OPENCODE_APPROVE_TOKEN`; target and head are revalidated. | +| `deploy-pages.yml` | required `project_name:string`, `build_dir:string`; optional `custom_domain:string` | Legacy caller uses `secrets: inherit`, but the callee consumes only `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`; explicit declarations are the required migration. | + +Credential classes remain purpose-specific: model-provider secrets (including +`NVIDIA_NIM_API_KEY`, Noema/provider keys), review-publication/App authority, +branch/merge mutation tokens, cloud deployment secrets, and the per-job GitHub +token MUST NOT be aliased merely because one route is unavailable. + +### 10.4 Sandbox result schemas and exit codes + +Both markers are exactly one ASCII line: a literal marker, one space, then a +valid JSON object with keys serialized deterministically. They are currently +implicit version 1; consumers MUST reject unknown missing required keys, and a +future incompatible schema MUST add `schema_version`. + +| Marker | Required JSON fields and types | +|---|---| +| `SANDBOXED_VERIFY_RESULT` | `allowed_env:string[]`, `command:string[]`, `cwd:string`, `elapsed_seconds:number`, `evidence_note:string`, `exit_code:integer`, `network:{default,required,not-required}`, `sandbox:string`, `sandboxed:true` | +| `SANDBOXED_WEB_E2E_RESULT` | `allowed_env:string[]`, `backend_cmd:string`, `backend_ready:boolean`, `cwd:string`, `e2e_cmd:string`, `elapsed_seconds:number`, `evidence_note:string`, `exit_code:integer`, `frontend_cmd:string`, `frontend_ready:boolean`, `network:{default,required,not-required}`, `sandbox:string`, `sandboxed:true` | + +| Exit | Meaning | +|---:|---| +| child code | Completed verification/E2E process; preserve its exact return code. | +| `2` | Safely redacted command-line usage error after an unambiguous preflight context exists. | +| `124` | Verification or E2E timeout. | +| `125` | Web backend/frontend readiness did not become true within the startup contract. | +| `126` | Unsafe allow-value preflight, sandbox/setup/launch/cleanup, or another wrapper failure. Preflight/setup paths without a safe context emit no attacker-controlled result. | + +The marker and exit value MUST agree. If the final marker would contain a +sensitive literal it is suppressed, and suppression cannot turn failure into a +success-shaped receipt. + +### 10.5 Scheduler and mention receipts + +`pr_review_merge_scheduler.py` emits one final JSON object with +`schema_version: "pr-review-merge-scheduler/v2"`, counts, run configuration, +and per-PR decisions. Each decision carries PR number, internal action, +contract decision, reason, notes, and optional typed guidance. Allowed +`merge_mode` values are `direct_or_auto`, `auto`, `direct`, and `disabled`. + +Agent invocations use the exact-name artifact prefix +`cwl-agent-invocation-` as the durable idempotency ledger. The key +binds repository, PR, source comment, requester, requested agent, complete +downstream controls, and head/base identity. A reaction or acknowledgement +comment is user feedback, not the authoritative receipt. + +## 11. Data retention and privacy + +Raw PR source, logs, and artifacts MUST use the shortest retention compatible with review and incident needs. Evidence stores SHOULD retain hashes, classifications, decisions, and bounded diagnostic excerpts rather than unlimited raw content. Access to unmasked business PII MUST be purpose-bound and auditable. Credentials MUST never be retained as evidence. + +## 12. Verification + +The minimum verification classes are defined in [TEST_STRATEGY.md](TEST_STRATEGY.md). Requirement-to-implementation status is maintained in [TRACEABILITY.md](TRACEABILITY.md); documentation presence is not proof of implementation. diff --git a/docs/automation/UML.md b/docs/automation/UML.md new file mode 100644 index 000000000..1d52655d5 --- /dev/null +++ b/docs/automation/UML.md @@ -0,0 +1,215 @@ +# UML and behavior diagrams + +Status: accepted baseline +Last reviewed: 2026-08-09 + +These diagrams are normative at the boundary level. Detailed step names remain in workflow source and are checked through [TRACEABILITY.md](TRACEABILITY.md). + +## 1. Bounded-context component view + +```mermaid +flowchart TB + Governance["GitHub governance context"] + Review["Review and evidence context"] + Execution["Sandbox execution context"] + Scheduling["Maintenance and merge context"] + Product["Product repository context"] + + Product --> Governance + Governance --> Review + Review --> Execution + Review --> Scheduling + Scheduling --> Governance + Governance --> Product +``` + +## 2. PR-maintenance sequence + +```mermaid +sequenceDiagram + participant A as Maintainer automation + participant G as GitHub + participant C as Central workflows + participant R as Review providers + participant H as Human reviewer + + A->>G: Fetch PRs, exact heads, live bases, gates, writers + G-->>A: Current state snapshot + A->>C: Run smallest safe test-first repair + C->>R: Request current-head machine evidence + R-->>G: Check, status, or formal bot review + H-->>G: Qualifying current-head review + A->>G: Re-fetch and merge with expected head + G-->>A: Protected merge or precise rejection + A->>G: Request protected-main acceptance +``` + +The final merge request occurs only if the machine, human, ruleset, thread, mergeability, and freshness evidence all authorize it. Any wait returns the automation to another ledger lane. + +## 3. Product-development sequence + +```mermaid +sequenceDiagram + participant A as Maintainer automation + participant G as GitHub queue + participant D as Deterministic gates + participant M as Optional model path + participant H as Human reviewer + + A->>G: Prove no higher-priority safe PR or issue action + A->>A: Select one bounded buyer/control-plane gap + A->>D: Add failing contract then minimum implementation + D-->>A: Tests, coverage, security, docs evidence + opt Model-backed analysis is material + A->>M: Invoke with scoped NVIDIA credential + M-->>A: Validated advisory or gate evidence + end + A->>G: Open or update exact-head PR + H-->>G: Independent review + A->>G: Return to whole queue +``` + +## 4. Evidence gate state machine + +```mermaid +stateDiagram-v2 + [*] --> Observed + Observed --> Incomplete: required evidence absent or pending + Observed --> Failed: current evidence fails + Observed --> Stale: head or live base moves + Incomplete --> Observed: evidence event + Failed --> Repairing: root cause and feasible remedy + Repairing --> Observed: new exact head + Stale --> Observed: refresh identity + Observed --> GateClean: all current gates satisfied + GateClean --> Stale: head or live base moves + GateClean --> Integrated: expected-head protected merge + Integrated --> Accepted: protected-main or consumer proof + Accepted --> [*] +``` + +`GateClean` is not incident closure. `Accepted` is scenario-specific and can be reopened by contradictory live evidence. + +## 5. Reviewer and merge authority flow + +```mermaid +flowchart TB + Machine["Checks, statuses, model evidence"] + Formal["Formal review submissions"] + Policy["Ruleset and required-gate policy"] + Scheduler["Expected-head merge scheduler"] + Merge["GitHub protected merge"] + + Machine --> Scheduler + Formal --> Scheduler + Policy --> Scheduler + Scheduler -->|all authorities agree| Merge + Scheduler -->|anything absent, stale, or failed| Policy +``` + +Machine evidence cannot enter the `Formal` authority class. A qualifying human approval cannot replace a failed deterministic check. + +## 6. Deployment and control-plane topology + +```mermaid +flowchart TB + Required["Organization required workflows"] + Central["Protected .github default branch"] + Runner["Ephemeral GitHub-hosted runner"] + Target["Target product repository"] + Provider["OIDC/App and review providers"] + + Required --> Central + Central --> Runner + Target --> Required + Runner --> Provider + Runner --> Target + Provider --> Runner +``` + +The protected central source is trusted code; the target PR source is untrusted input. Short-lived provider or App authority is scoped to the job that needs it. + +## 7. Retry and failure classification + +```mermaid +flowchart TB + Failure["Observed failure"] + Classify{"Evidence-backed class?"} + Transient["Bounded transient retry"] + Permanent["Fail closed and repair or defer"] + Continue["Record evidence and continue queue"] + + Failure --> Classify + Classify -->|DNS, reset, 5xx, capacity| Transient + Classify -->|auth, integrity, TLS, ref, schema, test| Permanent + Transient -->|budget remains| Failure + Transient -->|exhausted| Continue + Permanent --> Continue +``` + +## 8. Sandbox evidence publication sequence + +```mermaid +sequenceDiagram + participant W as Verify/web wrapper + participant P as Child process or service + participant X as Shared redactor + participant E as CI evidence sink + + W->>P: Execute unchanged command in bounded sandbox + P-->>W: stdout, stderr, timeout, exception, or service log + W->>X: Complete output plus explicit sensitive values + X-->>W: Canonicalized and redacted evidence + W->>E: Preserve stream context, exit code, and valid result JSON + Note over W,E: Redact before tail selection or serialization +``` + +If safe redaction cannot be established before parsing or setup output, the wrapper returns its setup-failure code without publishing attacker-controlled evidence. + +## 9. Mention routing and idempotency sequence + +```mermaid +sequenceDiagram + participant G as GitHub event + participant R as Mention router + participant L as Artifact ledger + participant A as Approved agent + + G->>R: Comment event or authenticated sweep + R->>G: Re-fetch actor, comment, PR, and exact head + R->>L: Claim canonical invocation key + alt completed or active claim exists + L-->>R: Duplicate; no side effect + else claim acquired + R->>A: Canonical allowlisted request + A-->>R: Bounded result receipt + R->>L: Complete claim with outcome + end +``` + +Visible mention text never becomes executable input. Comment edits, agent +changes, and head changes produce a new identity or invalidate the old request; +redelivery of the same identity is observable but side-effect free. + +## 10. Merge-mode and external-head state + +```mermaid +stateDiagram-v2 + [*] --> Inspecting + Inspecting --> Blocked: evidence, thread, policy, or mergeability fails + Inspecting --> ExternalWait: fork or external head + Inspecting --> DirectAttempt: clean and direct or direct_or_auto + Inspecting --> AutoQueued: auto mode and policy eligible + DirectAttempt --> Integrated: expected-head merge succeeds + DirectAttempt --> AutoQueued: direct_or_auto eligible fallback + DirectAttempt --> Blocked: non-fallback error or head moved + AutoQueued --> Inspecting: head, base, review, or check event + ExternalWait --> Inspecting: maintainer repair or new live evidence + Blocked --> Inspecting: new live evidence + Integrated --> [*] +``` + +`direct_or_auto` is an ordered policy, not two simultaneous writers. External +heads remain reviewable, but central automation neither direct-merges nor queues +auto-merge for them; it records the maintainer prerequisite and continues other +work. diff --git a/docs/automation/adr/0001-branch-writer-leases-and-read-only-audit.md b/docs/automation/adr/0001-branch-writer-leases-and-read-only-audit.md new file mode 100644 index 000000000..fe8838231 --- /dev/null +++ b/docs/automation/adr/0001-branch-writer-leases-and-read-only-audit.md @@ -0,0 +1,57 @@ +# ADR-0001: Branch-local writer leases and a separate read-only fleet audit + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL repository maintainers + +## Context + +Multiple agents, scheduled maintainers, Jules, dependency bots, and humans may inspect or change the same repository. Treating any activity as a repository-wide lock wastes capacity; ignoring writers risks lost work, invalid tests, wrong-thread resolution, and non-fast-forward or destructive recovery. + +## Decision drivers + +- Preserve every actor's work and source history. +- Allow non-conflicting branches and read-only audits to proceed concurrently. +- Bind a write to the exact branch head inspected immediately beforehand. +- Make collision recovery observable and non-destructive. +- Avoid private-memory-only ownership claims. + +## Alternatives considered + +1. **No lease; rely on Git conflicts.** Rejected because API writes, comments, thread resolution, and branch mutations can race without a file conflict. +2. **One repository-wide writer lock.** Rejected because a waiting PR would idle unrelated branches, docs, issues, and operations. +3. **Branch-local writer lease plus independent read-only audit.** Selected because it limits exclusivity to the mutable target while preserving fleet visibility and throughput. + +## Decision + +Every source/ref/PR-state writer owns one repository/ref/expected-head tuple for a bounded interval. Immediately before mutation it re-fetches the target head/base and relevant writer state. If source-affecting state moved or another write-capable actor owns the branch, it stops writing that branch and rotates to disjoint work. + +Read-only fleet auditors may inspect all repositories and produce evidence/handoffs but never mutate source, refs, PR state, reviews, or Project status. Review/check execution alone is not a source-writer conflict. + +## Consequences + +Positive: collisions are localized, unrelated work continues, and stale local state cannot authorize a write. Negative: the lease is partly procedural until a durable shared ledger exists; last-moment API refreshes add latency and complexity. + +## Failure and recovery + +On movement or collision, preserve both trees, capture exact remote/local heads, stop writes, and prepare a manual or separately reviewed reconciliation. Never force-push, wholesale-select `ours/theirs`, reset another actor's work, or deploy a one-shot repair workflow. If the lease holder disappears, expire the lease only after live branch and actor evidence shows no active writer. + +## Security and governance impact + +The lease narrows mutation authority and reduces confused-deputy and TOCTOU risk. It does not grant review or merge approval and cannot override branch protection. + +## Tests and acceptance + +- moved-head/base and duplicate-dispatch tests; +- expected-head branch update/merge/autofix rejection; +- mention idempotency and exact artifact-ledger tests; +- a live collision exercise showing no source loss; and +- queue rotation while the blocked branch remains untouched. + +## Migration and rollback + +Document writer identity and expected head in Project/issue/automation state where supported. Existing branch writers adopt the last-moment refresh before mutation. Rollback removes only the shared ledger implementation; expected-head and non-destructive behavior remain mandatory. + +## Supersession conditions + +Supersede when GitHub provides an authoritative native branch-write lease with identity, expiry, and expected-head enforcement, or when a reviewed central lease service provides stronger guarantees without becoming a single availability bottleneck. diff --git a/docs/automation/adr/0002-exact-source-and-live-base-binding.md b/docs/automation/adr/0002-exact-source-and-live-base-binding.md new file mode 100644 index 000000000..05de18082 --- /dev/null +++ b/docs/automation/adr/0002-exact-source-and-live-base-binding.md @@ -0,0 +1,57 @@ +# ADR-0002: Bind evidence to exact source head and independently resolved live base + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL repository maintainers + +## Context + +GitHub exposes a PR head, a base SHA captured in PR metadata, synthetic merge commits, live branch refs, checks, statuses, reviews, workflow runs, and artifacts. These identities can diverge. Earlier operational incidents showed that a green predecessor, synthetic merge, or stale base snapshot can be misrepresented as current source evidence. + +## Decision drivers + +- Prove exactly what source was tested and reviewed. +- Detect compatibility drift when the protected base advances. +- Preserve useful synthetic-merge evidence without mislabeling it. +- Make all write and merge operations race-safe. + +## Alternatives considered + +1. **Trust PR API `base.sha` and `head.sha` once per run.** Rejected because the live base and head may move after observation. +2. **Use only GitHub's synthetic merge SHA.** Rejected because it obscures the source commit and may not exist or match required source-head review. +3. **Track source head, PR base snapshot, independently resolved live base, workflow source, run/attempt, and merge revision separately.** Selected. + +## Decision + +All checks, reviews, statuses, artifacts, findings, and mutations name the immutable `source_revision`. Decision time independently resolves the protected base ref into `live_base_revision` and keeps the PR metadata base as historical `base_revision_snapshot`. Trusted workflow source SHA and Actions run/attempt are separate identities. + +Any source-head movement invalidates predecessor-head evidence. Any live-base movement triggers compatibility re-evaluation under repository policy. Synthetic merge and merge-group evidence remain labeled and cannot silently replace source-head proof. Mutations use expected-head semantics. + +## Consequences + +Positive: evidence is auditable and stale success cannot authorize a merge. Negative: more API calls and evidence fields are required; a moving base can regenerate costly checks. + +## Failure and recovery + +On mismatch, abort the mutation, mark evidence stale, refresh the PR/head/base/workflow state, and rerun only the gates required for the new identity. Do not transplant review text or run URLs into a current-head claim. + +## Security and governance impact + +This decision prevents stale-evidence spoofing, review replay, and TOCTOU writes. It supports ruleset and last-push review semantics but does not weaken GitHub's native interpretation. + +## Tests and acceptance + +- source head versus synthetic merge fixtures; +- live base advancing after PR snapshot; +- workflow SHA and run-attempt artifact identity; +- stale formal review/status/check rejection; +- expected-head update/merge failure; and +- protected-main acceptance naming the integrated SHA. + +## Migration and rollback + +Add identity fields to payloads and results compatibly, update consumers, then make validation mandatory. Rollback may disable an optional consumer but must not return to a single ambiguous `sha` field. + +## Supersession conditions + +Supersede only if GitHub exposes one cryptographically bound, policy-authoritative evidence object that simultaneously and unambiguously identifies source, current base, workflow source, run attempt, review, and merge context. diff --git a/docs/automation/adr/0003-classified-bounded-retries.md b/docs/automation/adr/0003-classified-bounded-retries.md new file mode 100644 index 000000000..7a02c4f2c --- /dev/null +++ b/docs/automation/adr/0003-classified-bounded-retries.md @@ -0,0 +1,57 @@ +# ADR-0003: Retry only classified transient failures within a bounded budget + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL reliability and security maintainers + +## Context + +GitHub, runners, DNS, package sources, and model providers fail transiently, but integrity, authentication, authorization, TLS, ref, schema, and product-test failures require remediation. Blind retries waste hours, obscure the first boundary, amplify provider load, and can convert unsafe or deterministic failures into misleading green outcomes. + +## Decision drivers + +- Recover automatically from genuine transient infrastructure faults. +- Fail quickly and closed on security, policy, identity, and source defects. +- Bound time, attempts, queue pressure, and cost. +- Preserve a concrete final diagnostic and original failure class. + +## Alternatives considered + +1. **Never retry.** Rejected because common network/provider transients needlessly block evidence. +2. **Retry every non-zero result.** Rejected because it hides defects and attacks, and can cause storms. +3. **Positive allowlist of transient classes with bounded attempts/backoff/total budget.** Selected. + +## Decision + +Each retrying operation defines accepted transient evidence, attempt count, per-attempt timeout, total budget, backoff/jitter, idempotency key, and final failure record. DNS/reset/GitHub 5xx/provider-capacity failures may qualify. Checksum/signature/pin, TLS, 401/403, OIDC, disallowed actor, malformed payload/schema, missing/moved ref, and test failures do not retry as infrastructure. + +A provider retry or fallback never changes the required output schema or converts exhaustion into approval. The maintainer defers long-running/retrying work and continues other lanes. + +## Consequences + +Positive: fewer flaky failures without weakening fail-closed behavior. Negative: classifiers require maintenance and can be wrong; tests need realistic provider/transport fixtures. + +## Failure and recovery + +If classification is uncertain, fail closed and expose bounded evidence. If retry logic causes load or masks a defect, disable only that retry class, preserve the base operation, and open an incident. Exhaustion records every attempt class without secrets. + +## Security and governance impact + +The decision prevents retries from bypassing integrity, auth, ref, or policy boundaries and reduces denial-of-service amplification. Retry settings are reviewed source, not runtime attacker input. + +## Tests and acceptance + +- one positive transient class per operation; +- success after transient failure; +- exact exhaustion count and total budget; +- immediate permanent-class rejection; +- idempotent duplicate delivery; and +- final diagnostic retains first failing boundary and current revision. + +## Migration and rollback + +Instrument existing loops, enumerate current retry predicates, replace broad predicates with explicit classes, and add attempt receipts. Rollback returns to fail-closed single-attempt behavior, never broad retry. + +## Supersession conditions + +Supersede when an upstream platform provides equivalent typed, authenticated failure classes and bounded idempotent retry semantics that are verified end to end. diff --git a/docs/automation/adr/0004-explicit-secret-contracts.md b/docs/automation/adr/0004-explicit-secret-contracts.md new file mode 100644 index 000000000..e41ca3a55 --- /dev/null +++ b/docs/automation/adr/0004-explicit-secret-contracts.md @@ -0,0 +1,57 @@ +# ADR-0004: Use explicit minimal secret contracts for reusable workflows + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL security and workflow maintainers + +## Context + +Reusable workflows can receive explicitly named secrets or inherit a caller's full secret set. Inheritance is convenient but violates purpose limitation, complicates review, increases disclosure impact, and may supply model/deploy/write credentials to deterministic jobs that never need them. + +## Decision drivers + +- Least privilege and clear data-flow review. +- Stable credential identities during reviewed migrations. +- Short-lived OIDC/App authority where possible. +- Prevent secrets from reaching free/anonymous providers or untrusted code. +- Preserve compatible deployments while legacy callers migrate. + +## Alternatives considered + +1. **Use `secrets: inherit` everywhere.** Rejected as over-broad and unauditable. +2. **Use one organization super-token.** Rejected due blast radius and authority conflation. +3. **Explicit named secrets plus scoped OIDC/App exchange; narrow documented legacy exceptions.** Selected. + +## Decision + +Every reusable workflow declares the minimum named secrets and documents purpose, required/optional status, consumer job, provider, and failure behavior. Deterministic jobs receive no model secret. `NVIDIA_NIM_API_KEY` is used only for actual approved model calls; `COPILOT_GITHUB_TOKEN` is prohibited for autonomous development. Anonymous/free provider execution receives no unrelated credential. + +Cross-repository privileges prefer short-lived OIDC/App exchange constrained by trusted workflow source and target. Existing credential names/scopes remain unchanged until a separately reviewed migration. The current inherited-secret deploy pattern is a legacy exception to remove, not precedent. + +## Consequences + +Positive: smaller blast radius, readable contracts, provider isolation, and easier audit. Negative: caller updates and compatibility bridges are required; OIDC/App availability becomes an explicit dependency. + +## Failure and recovery + +Missing or failed exchange causes only the privileged/model operation to fail closed. Do not fall back to a broader undocumented token. Continue deterministic/read-only work. Rotate/revoke any exposed credential and inspect every job that could have inherited it. + +## Security and governance impact + +This implements least privilege, separation of duties, purpose limitation, and auditable credential flow. Credential availability never grants review or merge authority beyond GitHub policy. + +## Tests and acceptance + +- workflow contracts enumerate allowed secrets and job permissions; +- deterministic and anonymous paths assert model/write secrets absent; +- OIDC audience/target/actor negative tests; +- redaction covers all secret transport and error paths; +- real positive/negative consumer canary after migration. + +## Migration and rollback + +Inventory inherited callers, add explicit callee declarations, update one low-risk caller, run positive and missing-secret controls, then migrate the fleet. Rollback restores the prior caller only if it does not re-expose a known credential; otherwise disable the optional deployment/review path. + +## Supersession conditions + +Supersede when GitHub enforces callee-declared per-secret purpose and automatic least-privilege token attenuation across reusable workflows with equivalent audit evidence. diff --git a/docs/automation/adr/0005-independent-review-authority.md b/docs/automation/adr/0005-independent-review-authority.md new file mode 100644 index 000000000..dce50e662 --- /dev/null +++ b/docs/automation/adr/0005-independent-review-authority.md @@ -0,0 +1,56 @@ +# ADR-0005: Preserve counted independent human review and stale-head semantics + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL governance maintainers + +## Context + +OpenCode, Noema, Strix, CodeRabbit, checks, statuses, and comments provide valuable automated evidence. GitHub branch policy separately requires formal reviews and may require a non-author approval, approval after the last push, thread resolution, and code-owner review. Conflating those authorities can let automation merge ahead of governance, especially if a routine token later gains bypass capability. + +## Decision drivers + +- Preserve human accountability and repository policy. +- Prevent author-self, bot, comment, status, dismissed, or predecessor reviews from satisfying human approval. +- Keep useful automated analysis without pretending it has another identity. +- Make approval capacity a governance prerequisite, not a reason to weaken source. + +## Alternatives considered + +1. **Count any `APPROVED`-looking output.** Rejected because provenance and authority differ. +2. **Let one automated reviewer replace human review.** Rejected because it violates repository policy and separation of duties. +3. **Require GitHub's aggregate policy plus an eligible formal exact-head human approval and separate automated gates.** Selected. + +## Decision + +Merge scheduling requires the live GitHub review decision and, where configured, at least one formal current-head `APPROVED` review from an eligible non-author human identity. Automated identities, PR author identities, dismissed reviews, comments/reactions, statuses, check conclusions, model text, synthetic approvals, and predecessor-head submissions do not qualify. + +Automated OpenCode/Noema/Strix/CodeRabbit evidence remains independently required where policy specifies. Last-push and unresolved-thread requirements remain GitHub-authoritative. No agent may self-approve, impersonate, use an alternate author credential, or reduce gates to unblock itself. + +## Consequences + +Positive: auditable separation of duties and defense against reviewer spoofing. Negative: merge latency can depend on eligible reviewer capacity; automated clean evidence cannot finish a protected merge alone. + +## Failure and recovery + +If qualifying approval is absent, mark only that PR as external-governance wait, retain expected-head-safe merge intent when supported, and rotate to other work. If a review is stale or dismissed, request a new review on the current head; never rewrite source solely to retrigger a provider unless the source repair has independent value. + +## Security and governance impact + +The decision reduces privilege escalation and repudiation risk. Branch protection and rulesets remain final; source logic is a defense-in-depth precondition, not a bypass. + +## Tests and acceptance + +- author, bot, OpenCode-only, comment, status, dismissed, stale, and wrong-commit negative fixtures; +- eligible non-author current-head positive fixture; +- live aggregate `reviewDecision` and last-push/thread requirements; +- routine token with hypothetical bypass still refuses unauthorized merge; and +- current ruleset audit. + +## Migration and rollback + +Update positive scheduler fixtures to include real policy-authorized state. Existing automated reviewer identities and credential names remain stable. Rollback may disable automated merge while preserving human review; it may not restore automated approval substitution. + +## Supersession conditions + +Supersede only if organization governance formally changes the independent-review requirement through an audited ruleset decision and an equivalent separation-of-duties control is documented. diff --git a/docs/automation/adr/0006-protected-main-operational-acceptance.md b/docs/automation/adr/0006-protected-main-operational-acceptance.md new file mode 100644 index 000000000..8425d5391 --- /dev/null +++ b/docs/automation/adr/0006-protected-main-operational-acceptance.md @@ -0,0 +1,57 @@ +# ADR-0006: Operational incidents close on protected-main or real-consumer evidence + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL reliability and product maintainers + +## Context + +Feature-branch tests prove source behavior in a development context. Central workflows, required-workflow sourcing, rulesets, dispatch permissions, secrets, provider integration, and product consumers may behave differently after merge. Several incidents recurred because source integration was treated as operational closure. + +## Decision drivers + +- Verify the exact integrated source and deployed GitHub configuration. +- Catch base-branch sourcing, credential, event, and consumer-contract failures. +- Keep closure scenario-specific and reopenable. +- Produce evidence buyers and operators can audit. + +## Alternatives considered + +1. **Close on local tests.** Rejected because it misses hosted integration. +2. **Close when the PR merges.** Rejected because merge does not prove runtime activation. +3. **Require protected-main central proof and, for consumer-facing changes, a real low-risk consumer positive/negative canary.** Selected. + +## Decision + +Every operational incident names an acceptance scenario before repair. After protected merge, run the scenario against the integrated commit and capture repository, workflow source, source/base identity, run/attempt, outcome, and rollback result. Central reusable/required workflow changes additionally run a real product-repository consumer and relevant negative control. + +An `operational_acceptance` record closes only the named incident/scenario. Contradictory live evidence reopens it. + +## Consequences + +Positive: fewer paper fixes and stronger acquisition/audit evidence. Negative: closure takes longer and depends on safe canary availability and provider/platform health. + +## Failure and recovery + +If protected-main or consumer proof fails, reopen the incident, classify whether source, configuration, permissions, provider, or consumer caused the failure, and continue the normal test-first loop. Do not roll back unrelated gates or declare the consumer out of scope after the fact. + +## Security and governance impact + +Operational proof confirms real authority, secret, and trust-boundary behavior without bypass. Canaries use low-risk classified data and least privilege. + +## Tests and acceptance + +- source PR exact-head gate; +- protected-main workflow source receipt; +- real consumer positive path; +- relevant negative control (missing policy/secret, stale head, unsafe content, or provider failure); +- rollback or deterministic rehearsal; and +- dated traceability entry. + +## Migration and rollback + +Add acceptance criteria to incident issues/PRs, then index existing high-risk central repairs as evidence becomes available. If no safe real canary exists, keep the incident open or feature inactive rather than inventing evidence. + +## Supersession conditions + +Supersede when continuous verified deployment automatically binds protected source, configuration, consumer, negative control, and rollback evidence with equivalent auditability. diff --git a/docs/automation/adr/0007-work-conserving-maintenance.md b/docs/automation/adr/0007-work-conserving-maintenance.md new file mode 100644 index 000000000..a28553d66 --- /dev/null +++ b/docs/automation/adr/0007-work-conserving-maintenance.md @@ -0,0 +1,57 @@ +# ADR-0007: Maintenance is work-conserving; reporting is not completion + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL product and automation maintainers + +## Context + +Long model reviews, human approval, queued checks, provider cooldowns, and one blocked PR repeatedly caused finite maintenance invocations to stop after reporting status. An hourly recurrence then became an excuse to defer work despite safe tasks in other branches, docs, issues, operations, or product gaps. + +## Decision drivers + +- Maximize validated repository progress within each finite invocation. +- Avoid polling and repetitive blocker narration. +- Keep dependencies and writer safety without serializing the whole fleet. +- Make documentation and operational proof first-class work. + +## Alternatives considered + +1. **Stop after one named request or blocker.** Rejected because it strands executable work. +2. **Poll until the active PR completes.** Rejected because it wastes budget and can starve the queue. +3. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, and require a double exit sweep.** Selected. + +## Decision + +Each run maintains live lanes for mergeable PRs, defects/reviews/checks, protected-main acceptance, issues, documentation/automation drift, and one bounded buyer-visible gap. Pending external states enter a deferred set keyed by repository, PR, head, and run/review identity. After every substantive action the automation refreshes affected state and immediately selects another safe item. + +A merge, dispatch, completed document, reviewer request, or external-governance wait never completes the invocation by itself. Before termination, a whole-queue sweep includes docs versus implementation. If it finds work, execute it and sweep again. Only real execution-budget exhaustion or a second fresh sweep proving every lane non-actionable permits termination. + +## Consequences + +Positive: higher throughput, less queue starvation, and durable progress while providers wait. Negative: the automation needs careful scope/lease management and may leave several deferred items that require a shared ledger. + +## Failure and recovery + +If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If budget expires mid-item, leave exact continuation evidence; do not claim product completion. + +## Security and governance impact + +Work conservation never permits bypassing approval, checks, writer leases, or permissions. It changes scheduling, not authority. Routine output is suppressed so status narration cannot substitute for work. + +## Tests and acceptance + +- queued/pending item does not block selection of another lane; +- duplicate dispatch is deferred, not polled; +- branch-local lease blocks only one branch; +- docs drift appears as executable debt; +- double exit sweep finds newly available work; and +- scheduled-run output is empty except defined notification events. + +## Migration and rollback + +Update maintainer prompts and scheduler policies with the continuation ledger and exit sweeps. Remove duplicative inactive loops. Rollback may reduce concurrency but must preserve no-report-as-completion and rotation from external waits. + +## Supersession conditions + +Supersede if a durable queue engine provides provably fair, dependency-aware, lease-safe work conservation and equivalent termination proofs across repositories. diff --git a/docs/automation/adr/0008-central-control-plane-and-thin-consumers.md b/docs/automation/adr/0008-central-control-plane-and-thin-consumers.md new file mode 100644 index 000000000..d8205c317 --- /dev/null +++ b/docs/automation/adr/0008-central-control-plane-and-thin-consumers.md @@ -0,0 +1,58 @@ +# ADR-0008: Central control-plane ownership with thin product consumers + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL architecture maintainers + +## Context + +Copying OpenCode, Strix, Noema, merge, and security workflows into every repository creates policy drift, duplicate fixes, inconsistent credentials/check names, and an acquisition liability. Moving product-specific build/release/deployment into the central repository would create the opposite failure: tight coupling and loss of independent product operation. + +## Decision drivers + +- One reviewed source for organization-wide governance and trust boundaries. +- Independently deployable and usable products. +- Stable, explicit, versioned interfaces. +- Product-owned domain quality and release behavior. +- Safe gradual migration from thick local copies. + +## Alternatives considered + +1. **Full workflow copy in every repository.** Rejected due drift and repair multiplication. +2. **Centralize all product CI/CD and data.** Rejected due coupling, privilege concentration, and product autonomy loss. +3. **Central shared policy/trusted execution plus thin versioned callers; product-specific gates stay local.** Selected. + +## Decision + +`ContextualWisdomLab/.github` owns required/reusable governance workflows, trusted dispatchers, evidence schemas/normalization, shared security policy, and merge/fix schedulers. Product repositories own product code/data, domain tests, platform matrices, release/deploy/migrations, and thin calls or ruleset enrollment. + +Central interfaces declare stable events, inputs, secret contracts, result/check names, revision semantics, and failure behavior. Internal central implementation may evolve behind those interfaces. Products remain functional when optional central model services are unavailable, while required governance gates remain truthfully non-passing. + +## Consequences + +Positive: high-leverage fixes and uniform policy with modular products. Negative: central defects have fleet blast radius; compatibility and canary discipline are mandatory. + +## Failure and recovery + +On a central regression, stop/disable only the affected optional route or revert to a reviewed compatible central version. Product-specific build/release paths continue. Do not restore thick copies as an untracked permanent workaround. + +## Security and governance impact + +Central trust concentrates responsibility, so immutable source, minimal permissions, consumer allowlists, explicit secrets, and protected-main canaries are required. Product repositories cannot weaken central required gates through local copies. + +## Tests and acceptance + +- central workflow contract and immutable-source tests; +- thin consumer input/secret compatibility; +- product-local gate independence; +- one positive and negative real consumer canary; +- inventory rejects unexpected thick duplicates; and +- rollback retains product operability. + +## Migration and rollback + +Inventory local copies, identify product-specific logic, move only shared policy centrally, replace each repository copy with a thin caller/ruleset enrollment, verify exact consumers, then remove the duplicate. Rollback pins/reverts the thin contract or central version without recreating divergent policy. + +## Supersession conditions + +Supersede if GitHub provides organization-native policy modules with equivalent versioning, testing, secret attenuation, trusted source identity, and product-specific extension points. diff --git a/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md new file mode 100644 index 000000000..52e764915 --- /dev/null +++ b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md @@ -0,0 +1,69 @@ +# ADR-0009: Redact the complete sandbox evidence boundary while preserving diagnosis + +Status: Accepted; protected integration and consumer proof pending in `ContextualWisdomLab/.github#842` +Date: 2026-08-09 +Decision owners: CWL security and CI maintainers + +## Context + +`sandboxed_verify.py` and `sandboxed_web_e2e.py` execute proof commands and publish completed output, timeout payloads, exceptions, backend/frontend service tails, command metadata, and structured result JSON. Applying redaction to only some print calls leaves credentials reachable through JSON, terminal/Unicode evasion, nested commands, explicit allowed environment values, or stream/result boundaries. Over-broad pattern matching can instead erase benign keys and the diagnostics needed to fix CI. + +`ContextualWisdomLab/.github#841` identified the disclosure class but mixed an unrelated SSRF slice and missed publication paths. It was closed unmerged. #842 is the clean replacement and preserves #841 as historical evidence. + +## Decision drivers + +- Protect credentials across every CI/publication path. +- Preserve ordinary failure/timeout text, exit codes, commands after safe value substitution, and valid stable result JSON. +- Keep executed argv/environment/network/cleanup semantics unchanged. +- Resist JSON, ANSI, Unicode, multiline, nested-shell, and boundary-reassembly bypasses. +- Keep attacker-influenced processing bounded and near-linear. +- Avoid blanket PII masking and avoid corrupting benign diagnostic metadata. + +## Alternatives considered + +1. **Rely on GitHub's automatic secret masking.** Rejected because unknown, transformed, split, and child-generated values are not comprehensively covered. +2. **Redact only stdout/stderr at final print.** Rejected because timeouts, exceptions, service files/tails, commands, JSON, and pre-parser errors bypass it. +3. **Remove all logs or redact every `token`-like key.** Rejected because it destroys diagnosis and hides benign metrics such as token counts. +4. **One shared canonical redactor applied to complete content before truncation/serialization, with explicit allowed values and fail-closed preflight.** Selected. + +## Decision + +All human and structured evidence flows through `redact_sensitive_log.py`. It canonicalizes terminal/default-ignorable evasions, structurally handles supported JSON, recognizes credential semantics and exact explicit values, redacts complete service content before tail selection, sanitizes command/result metadata, separates publication boundaries, and preserves non-sensitive text/types/schema. + +Explicit allowed environment values may be used only when they are long enough, single-line, printable, non-whitespace-only, non-control-bearing, non-marker-colliding, and not owned by the wrapper. Safe single-line values containing spaces remain eligible when exact-literal redaction and boundary tests pass. Unsafe values fail before execution without publishing colliding evidence. Setup/launch/parser errors are emitted only after a safe redaction context exists; otherwise the wrapper exits with the setup-failure code and no attacker-controlled output. + +Credential redaction is not a total output quota and not a blanket PII masker. Resource limits and business-data access/retention remain separate controls. + +## Consequences + +Positive: a consistent, testable boundary closes asymmetric leaks while retaining useful diagnosis. Negative: redaction logic is security-critical and complex; unknown formats and raw on-disk service files remain residual risks; unsafe allowed values become a deliberate compatibility failure. + +## Failure and recovery + +If the redactor raises or cannot establish a safe context, fail closed before evidence publication, clean temporary state unless explicitly and safely retained, and return the documented setup code. If over-redaction breaks diagnosis, add a benign preservation fixture and narrow semantic classification without weakening exact secrets. If a live credential appeared, restrict evidence and rotate/revoke before source repair. + +## Security and governance impact + +This reduces credential disclosure and log-injection risk while preserving audit evidence. It does not grant permission to send private source/PII to an external model. Every new evidence sink must register with the boundary and threat model. + +## Tests and acceptance + +- completed/failure/timeout/exception output for both wrappers; +- backend/frontend full-log-before-tail behavior; +- JSON types/keys, multiline/escaped content, headers, cookies, private keys, JWT/provider tokens, URL userinfo; +- ANSI/backspace/cursor/default-ignorable and cross-stream/result reconstruction; +- separated and nested `sh -c`, `env`, `curl`, `sshpass`, MySQL, and Docker credential arguments; +- parser and temporary-workspace failures; +- single-line space-bearing allowed values plus rejection of short, whitespace-only, multiline/control, marker-colliding, and wrapper-owned values; +- stable exit code/result keys/valid JSON and ordinary diagnostic preservation; +- large hostile input and many explicit values within fixed performance budgets; +- 100% statement/branch/docstring evidence; and +- protected-main plus real consumer positive/negative proof. + +## Migration and rollback + +Integrate #842 through normal protection, then run a synthetic credential fixture from protected main and an affected consumer. Update the PR body, doctoring, and CHANGELOG with only final exact-head numbers. If regression requires rollback, disable the affected evidence publication or revert to a reviewed fail-closed no-output path; do not restore a known disclosure path. + +## Supersession conditions + +Supersede when the sandbox execution API provides typed, capability-safe evidence objects with built-in complete-boundary credential protection, bounded storage, diagnostic preservation, and equivalent adversarial proof. diff --git a/docs/automation/adr/0010-agent-mention-routing-and-idempotency-ledger.md b/docs/automation/adr/0010-agent-mention-routing-and-idempotency-ledger.md new file mode 100644 index 000000000..e7821926f --- /dev/null +++ b/docs/automation/adr/0010-agent-mention-routing-and-idempotency-ledger.md @@ -0,0 +1,86 @@ +# ADR-0010: Authenticated agent-mention routing with an idempotency ledger + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL automation maintainers + +## Context + +Review comments can explicitly request an approved agent, but redelivery, +edited comments, overlapping sweeps, forged bot names, and head changes can +otherwise cause duplicate or stale invocations. A comment is untrusted input; +neither visible text nor a prior automation memory proves authority. + +## Decision drivers + +- Exactly-once observable handling under at-least-once delivery. +- Explicit actor, repository, PR, comment, agent, and revision identity. +- No execution merely because a string resembles an agent mention. +- Recoverable evidence without a new operational database. + +## Alternatives considered + +1. **Dispatch every matching string.** Rejected because replay and spoofing are + uncontrolled. +2. **Keep process-local memory.** Rejected because workers are disposable and + concurrent. +3. **Validate live GitHub identity and persist an immutable claim/receipt in a + bounded artifact ledger.** Selected. + +## Decision + +The router accepts only supported comment events or authenticated sweeps, +normalizes an exact allowlisted agent name, re-fetches the live PR/comment and +current head, and constructs an invocation key from repository, PR, comment +identity/version, requested agent, and expected head. Before dispatch it claims +that key in the GitHub-backed artifact ledger. A completed, active claim makes a +redelivery a no-op; an expired incomplete claim may be recovered with a new +attempt that links its predecessor. + +Editing or deleting a comment, changing the PR head, or changing the requested +agent creates a different identity or invalidates the old request. The routed +worker receives canonical fields, never an executable string assembled from the +comment. + +## Consequences + +Duplicate model cost and repeated comments are bounded, and every invocation is +traceable. Artifact retention and concurrent claim races require explicit +handling; exactly-once side effects are achieved through idempotency rather than +assuming exactly-once event delivery. + +## Failure and recovery + +If live identity, the ledger, or dispatch is unavailable, publish no agent side +effect and retain a classified retryable receipt. Recovery re-fetches live state +and reclaims only an expired incomplete key. It never marks a failed dispatch as +completed. + +## Security and governance impact + +The design rejects forged identities, stale-head requests, arbitrary command +injection, and replay amplification. The router token may read comments and +write the bounded dispatch/ledger surfaces only; the invoked reviewer does not +inherit merge or branch-write authority. + +## Tests and acceptance + +- exact-name and near-match negative tests; +- duplicate, redelivery, edit, deletion, and head-change tests; +- concurrent claim winner/loser test; +- expired incomplete-claim recovery test; +- canonical payload and least-permission assertions; and +- protected-main receipt showing one observable invocation for one key. + +## Migration and rollback + +Deploy the ledger in observe-only mode, compare would-dispatch identities, +enable claims before side effects, then enable dispatch. Rollback disables new +dispatch while retaining ledger artifacts for diagnosis; it does not fall back +to unguarded string matching. + +## Supersession conditions + +Supersede if GitHub provides a native authenticated, idempotent agent-request +primitive with equivalent revision binding, replay protection, retention, and +audit evidence. diff --git a/docs/automation/adr/0011-provider-routing-and-credential-isolation.md b/docs/automation/adr/0011-provider-routing-and-credential-isolation.md new file mode 100644 index 000000000..a1eed6855 --- /dev/null +++ b/docs/automation/adr/0011-provider-routing-and-credential-isolation.md @@ -0,0 +1,83 @@ +# ADR-0011: Provider routing with per-purpose credential isolation + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL security and automation maintainers + +## Context + +Model-backed review can use NVIDIA NIM and a bounded GitHub Models fallback +pool. Provider availability and quality vary, while credentials differ in +scope, billing, rate limits, and disclosure risk. A missing optional provider +must not disable deterministic gates or cause another token to be silently +repurposed. + +## Decision drivers + +- Deterministic operation without optional model credentials. +- Explicit purpose and consumer for every secret. +- Bounded fallback with consistent review policy and evidence schema. +- Provider outages never synthesize approval. + +## Alternatives considered + +1. **One shared token for every provider and job.** Rejected due confused + authority and blast radius. +2. **A single provider with no fallback.** Rejected for avoidable availability + loss. +3. **Ordered candidates with provider-specific credentials, budgets, and + normalized results.** Selected. + +## Decision + +NVIDIA NIM is the configured first provider for OpenCode review, followed by an +explicit allowlisted GitHub Models candidate pool. Each provider reads only its +documented secret in the job that consumes it. Candidate selection preserves +the same prompt policy, read-only tool boundary, expected-head identity, +reasoning requirements, output normalization, and total retry budget. + +No credential aliasing is implicit: `NVIDIA_NIM_API_KEY`, GitHub model tokens, +App tokens, merge tokens, and `COPILOT_GITHUB_TOKEN` are distinct contracts. +Missing optional credentials skip only their provider route. Exhausting all +routes produces unavailable/failed evidence and cannot become an approval. + +## Consequences + +Review availability improves without conflating identities. Configuration and +cost monitoring become more complex, and provider-specific behavior needs +compatibility tests. + +## Failure and recovery + +Authentication, permission, integrity, TLS, schema, and policy failures fail +closed for that route and are not retried as transient transport failures. +Capacity, reset, timeout, and eligible 5xx failures may consume a bounded route +budget. Recovery rotates to another eligible provider or awaits operator repair +while deterministic work continues. + +## Security and governance impact + +Secret scopes, logs, artifacts, and child environments are purpose-limited. +Provider text remains untrusted advisory evidence until normalized and cannot +grant formal review, mutation, or merge authority by itself. + +## Tests and acceptance + +- missing-secret deterministic-path tests; +- provider order, budget, and fallback tests; +- authentication versus transient classification tests; +- secret non-propagation and log-redaction tests; +- normalized schema/head-binding parity across providers; and +- real protected-main primary and fallback canaries without secret output. + +## Migration and rollback + +Introduce a provider behind the candidate allowlist and canary it with a +dedicated secret. Rollback removes that candidate and secret mapping while +leaving deterministic gates and other providers unchanged. + +## Supersession conditions + +Supersede when provider routing, identity, cost, quality, or residency needs +require a dedicated gateway with equal or stronger credential attenuation and +evidence provenance. diff --git a/docs/automation/adr/0012-hash-pinned-toolchains-and-exact-base-materialization.md b/docs/automation/adr/0012-hash-pinned-toolchains-and-exact-base-materialization.md new file mode 100644 index 000000000..cc89bac3f --- /dev/null +++ b/docs/automation/adr/0012-hash-pinned-toolchains-and-exact-base-materialization.md @@ -0,0 +1,78 @@ +# ADR-0012: Hash-pinned toolchains and exact-base materialization + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL supply-chain maintainers + +## Context + +Central workflows execute across many repositories. Floating actions, packages, +containers, or default-branch source can change between evidence collection and +reproduction. A PR snapshot base SHA can also lag the live protected branch. + +## Decision drivers + +- Reproducible trusted tooling and dependency resolution. +- Independent source-head and live-base identity. +- Reviewable upgrades with bounded fleet rollout. +- Resistance to dependency substitution and tag movement. + +## Alternatives considered + +1. **Use latest versions and mutable tags.** Rejected because evidence cannot be + reproduced or attributed. +2. **Vendor every tool indefinitely.** Rejected due stale vulnerabilities and + maintenance burden. +3. **Pin immutable action/container revisions and hashed dependency sets; resolve + live base explicitly.** Selected. + +## Decision + +Trusted workflow actions and containers use immutable revisions or verified +digests. CI dependencies use reviewed lock/hash manifests and fail on an +unmatched artifact. The executing central workflow commit, target source head, +PR snapshot base, and independently fetched live protected-branch tip are +recorded separately. Mergeability, diff, and operational acceptance use the +live base appropriate to the decision, never an assumed alias. + +Dependency and toolchain updates are explicit PRs with provenance, security, +compatibility, and consumer-canary evidence. Generated lock/hash material is +reviewed with its human-readable source declaration. + +## Consequences + +Runs are more reproducible and supply-chain drift is visible. Upgrades require +regular maintenance, and unavailable pinned artifacts can block a route until a +reviewed replacement is integrated. + +## Failure and recovery + +Digest/hash/ref mismatch is an integrity failure: do not retry from an +alternative unpinned source. A transient registry transport failure may retry +within budget. Recovery restores the verified artifact or lands a reviewed pin +update and regenerates evidence. + +## Security and governance impact + +The decision reduces mutable-reference and dependency-confusion attacks. +Credentials used to fetch private artifacts remain scoped and must not enter +untrusted build steps or published evidence. + +## Tests and acceptance + +- immutable action/container reference checks; +- `--require-hashes` or equivalent dependency installation; +- generated-lock consistency and negative mismatch tests; +- source-head, snapshot-base, live-base, and workflow-source assertions; and +- a protected-main consumer canary for each material toolchain update. + +## Migration and rollback + +Inventory mutable references, pin the highest-risk trusted paths first, add +contract tests, then migrate remaining dependencies. Rollback returns to the +previous reviewed immutable pin, never to an unbounded latest tag. + +## Supersession conditions + +Supersede if a verified hermetic build/attestation platform provides stronger +immutable resolution, transparent provenance, and equivalent consumer support. diff --git a/docs/automation/adr/0013-autofix-and-merge-authority-separation.md b/docs/automation/adr/0013-autofix-and-merge-authority-separation.md new file mode 100644 index 000000000..d9873a280 --- /dev/null +++ b/docs/automation/adr/0013-autofix-and-merge-authority-separation.md @@ -0,0 +1,84 @@ +# ADR-0013: Separate autofix authority from merge authority + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL governance maintainers + +## Context + +Autofix may modify an eligible same-repository PR head, while the merge +scheduler may integrate a policy-clean exact head. Combining them lets a repair +actor approve or merge its own unreviewed output and makes credential scope and +incident attribution ambiguous. + +## Decision drivers + +- Every generated change receives fresh independent evidence. +- Least-privilege, purpose-specific mutation credentials. +- Expected-head race protection and branch-local writer exclusion. +- Clear recovery when conflict or validation fails. + +## Alternatives considered + +1. **One omnipotent repair-and-merge workflow.** Rejected due self-authorization + and blast radius. +2. **Disable all automated repair.** Rejected because bounded source-actionable + fixes can safely reduce maintenance toil. +3. **Distinct fix and merge paths with a mandatory new-head gate cycle.** + Selected. + +## Decision + +The autofix scheduler may dispatch the trusted autofix worker only for an +allowlisted, source-actionable, current-head finding on a mutable +same-repository branch. It acquires a branch writer lease, checks expected head +before checkout and push, produces the minimum diff, runs focused gates, and +publishes a new commit without approval or merge authority. + +The merge scheduler cannot reuse predecessor-head evidence. It re-fetches the +new head and requires all deterministic, security, formal-review, thread, +ruleset, mergeability, and last-pusher conditions before an expected-head merge. +Model exhaustion, process blockers, failed peer checks, external heads, and +unresolved conflicts are not autofix invitations. + +## Consequences + +Generated changes cannot bypass review, and failures have an identifiable +owner. Repair-to-merge latency increases because the complete gate cycle runs +again. + +## Failure and recovery + +If the head moves, validation fails, the diff exceeds scope, or the lease is +lost, the worker publishes no push. Conflict repair remains on the PR branch +and must still pass a new-head cycle. A merge failure triggers a live evidence +refresh, never an autofix retry by implication. + +## Security and governance impact + +Fix credentials write only eligible PR branches; merge credentials invoke only +the guarded GitHub merge primitive. Neither path can publish a qualifying human +review. Logs and receipts name the actor class without exposing tokens. + +## Tests and acceptance + +- eligibility and non-actionable-blocker tests; +- expected-head checks before checkout, commit, push, and merge; +- writer-lease contention tests; +- generated-head invalidation of all predecessor evidence; +- credential/actor separation assertions; and +- protected-main proof that a real generated head was independently rechecked + and reviewed before merge. + +## Migration and rollback + +Remove merge permissions from autofix jobs, remove repair permissions from the +merge path, add the new-head gate assertion, then canary one safe repository. +Rollback disables autofix dispatch while leaving manual repair and the guarded +merge scheduler available. + +## Supersession conditions + +Supersede only if GitHub supplies a native proposed-change object that is +cryptographically bound to mandatory independent review and cannot be merged by +its proposing authority. diff --git a/docs/automation/adr/0014-trusted-metadata-event-and-default-branch-dispatch.md b/docs/automation/adr/0014-trusted-metadata-event-and-default-branch-dispatch.md new file mode 100644 index 000000000..67a91f258 --- /dev/null +++ b/docs/automation/adr/0014-trusted-metadata-event-and-default-branch-dispatch.md @@ -0,0 +1,81 @@ +# ADR-0014: Metadata-only privileged events with default-branch dispatch + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL security and workflow maintainers + +## Context + +`pull_request_target` supplies a trusted base-repository context and can reach +secrets or write permissions, while PR filenames, refs, source, artifacts, and +comments are attacker-controlled. Review and autofix still need bounded PR data +and, in some paths, privileged publication or mutation. + +## Decision drivers + +- Untrusted PR code never executes with privileged event authority. +- Stable required contexts still materialize for fork and same-repository PRs. +- Privileged workers execute immutable protected-default-branch source. +- Every dispatch is rebound to live repository/PR/revision identity. + +## Alternatives considered + +1. **Checkout and run the PR from `pull_request_target`.** Rejected due direct + secret and repository compromise risk. +2. **Use only unprivileged `pull_request`.** Rejected because trusted + publication/organization coordination and stable fork handling are limited. +3. **Metadata-only privileged entrypoint plus authenticated default-branch + dispatch and bounded PR-data materialization.** Selected. + +## Decision + +Privileged PR events may read and validate metadata and materialize stable check +contexts, but MUST NOT check out or execute PR-controlled code, actions, +containers, package hooks, or shell text. Work requiring trusted publication or +mutation is sent through an allowlisted `repository_dispatch` payload to a +workflow loaded from the protected default branch. + +The receiver validates triggering actor and sender, target allowlist, event +schema, open PR state, base/head repository, refs and SHAs, mutability, and live +GitHub identity before reading PR source as inert data or running it inside an +explicit low-privilege sandbox. Any mismatch fails before a secret/write step. + +## Consequences + +The primary secret-execution boundary is reviewable and reusable across +repositories. Workflows become more complex and need payload compatibility, +idempotency, and two-stage observability. + +## Failure and recovery + +Missing actor configuration, malformed payload, unavailable live metadata, or +identity movement fails closed. Recovery creates a fresh dispatch from current +live state; it never edits the old payload or broadens the privileged event. + +## Security and governance impact + +PR content remains untrusted across filenames, archives, symlinks, artifacts, +logs, prompts, and commands. OIDC/App and mutation credentials exist only in the +specific protected-source job that needs them and are not inherited by PR +execution. + +## Tests and acceptance + +- workflow contract rejects PR checkout/execution in privileged entrypoints; +- actor/target/schema/base/head mismatch negative tests; +- fork/external-head and symlink/archive adversarial tests; +- least-permission and secret-environment assertions; and +- protected-main positive/negative dispatch canary with exact-head receipt. + +## Migration and rollback + +Move executable privileged steps behind a default-branch dispatcher, add live +binding tests, then remove them from the event workflow. Rollback disables the +dispatch or reverts its protected implementation; it never restores privileged +PR-head execution. + +## Supersession conditions + +Supersede if GitHub provides a native primitive that combines immutable trusted +workflow source, attenuated identity, exact PR revision binding, fork support, +and equivalent audit evidence without `pull_request_target` dispatch. diff --git a/docs/automation/adr/0015-direct-or-auto-merge-state-machine.md b/docs/automation/adr/0015-direct-or-auto-merge-state-machine.md new file mode 100644 index 000000000..1f2c755e5 --- /dev/null +++ b/docs/automation/adr/0015-direct-or-auto-merge-state-machine.md @@ -0,0 +1,79 @@ +# ADR-0015: Guarded `direct_or_auto` merge state machine + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL governance maintainers + +## Context + +Repositories differ in native auto-merge availability and immediate merge +policy. A scheduler must not leave an approved clean PR idle when direct merge +is allowed, nor bypass policy when direct merge fails. Historical documentation +described only `auto`, `direct`, and `disabled`, while the implementation uses +`direct_or_auto` as the default compatibility mode. + +## Decision drivers + +- One explicit, race-safe merge policy per scheduler run. +- Productive fallback without converting policy/integrity errors into success. +- External heads and non-clean states remain outside automated merge. +- Every merge is bound to the expected current head. + +## Alternatives considered + +1. **Auto-merge only.** Rejected because repositories without native support + remain idle. +2. **Direct merge only.** Rejected because queueing native policy is sometimes + the correct integration path. +3. **Ordered `direct_or_auto` with classified fallback.** Selected. + +## Decision + +The supported modes are `disabled`, `auto`, `direct`, and `direct_or_auto`. +`direct_or_auto` first attempts a guarded expected-head direct merge only for a +clean, same-repository, policy-authorized head. It falls back to native +auto-merge only for an explicitly classified “direct unavailable but auto +eligible” response. Head movement, failed/absent evidence, unresolved threads, +external heads, conflicts, authorization, and integrity errors do not fallback. + +One run performs these steps serially under the branch writer contract. It +re-fetches live state around each mutation. A queued auto-merge is a wait state, +not integration or operational acceptance. + +## Consequences + +Eligible PRs progress across heterogeneous repository settings with a concrete +logged outcome. Error classification and tests are more complex, and the mode +must remain visible in result receipts. + +## Failure and recovery + +A rejected direct attempt records the exact class. Eligible compatibility +failure may queue auto-merge; every other failure stops mutation and returns to +live inspection. Recovery occurs on a new event/sweep with fresh evidence. + +## Security and governance impact + +Neither path weakens rulesets, approval counts, last-pusher separation, thread +resolution, current-head checks, or expected-head semantics. External/fork +heads require a maintainer merge after evidence remains current. + +## Tests and acceptance + +- each mode and unsupported value; +- direct success and the narrow eligible fallback; +- head movement, external head, conflict, policy, authorization, and failed-gate + negative paths; +- no simultaneous direct/auto writer; and +- scheduler v2 receipt plus protected merge/queued-state evidence. + +## Migration and rollback + +Expose the mode in workflow inputs and result schema, canary +`direct_or_auto`, then use it as the default. Rollback selects `auto`, `direct`, +or `disabled` explicitly without changing evidence gates. + +## Supersession conditions + +Supersede if GitHub offers one universal expected-head merge primitive with +native queuing, classified errors, and equivalent ruleset enforcement. diff --git a/docs/automation/adr/0016-fail-closed-security-gate-composition.md b/docs/automation/adr/0016-fail-closed-security-gate-composition.md new file mode 100644 index 000000000..2a210cba9 --- /dev/null +++ b/docs/automation/adr/0016-fail-closed-security-gate-composition.md @@ -0,0 +1,82 @@ +# ADR-0016: Fail-closed composition of independent security and review gates + +Status: Accepted +Date: 2026-08-09 +Decision owners: CWL security and governance maintainers + +## Context + +The control plane receives checks, commit statuses, formal reviews, model +outputs, scanner findings, threads, ruleset parameters, and workflow receipts. +Treating any one signal as aggregate approval, or treating skipped/pending tool +execution as success, allows coverage gaps and confused authority. + +## Decision drivers + +- Independent detection layers retain their issuer and meaning. +- Required absence, skip, cancellation, staleness, or ambiguity is non-passing. +- Provider/tool failure is distinguished from a source finding. +- Live repository policy remains the final configured gate inventory. + +## Alternatives considered + +1. **One aggregate bot status.** Rejected because it hides missing and stale + authorities. +2. **Fail open on unavailable model/scanner.** Rejected because tool outage can + authorize unreviewed code. +3. **Intersect live required authorities and fail closed per configured gate.** + Selected. + +## Decision + +Merge eligibility is the intersection of live ruleset/branch policy, exact-head +deterministic checks, configured security gates, qualifying formal approvals, +last-pusher separation, current thread state, mergeability, expected-head +identity, and writer authority. Check, Status, Review, model, scanner, workflow, +and operational evidence remain distinct records. + +A required gate that is absent, pending, queued, skipped, neutral, cancelled, +timed out, stale, malformed, or unavailable is non-passing. An optional advisory +route may be skipped only when policy explicitly labels it optional; that skip +cannot satisfy another required class. Tool failures produce operational +diagnosis, not invented source findings or approval. + +## Consequences + +Coverage gaps are visible and cannot silently authorize merge. Provider outages +can delay affected PRs, so work-conserving rotation, bounded retries, and clear +operator receipts are required. + +## Failure and recovery + +Classify the failing authority, preserve its exact receipt, and retry only +eligible transient failures. Restore the gate or land a separately reviewed +policy change; never edit a result to success. Any new head reruns the entire +applicable intersection. + +## Security and governance impact + +The model resists status/review impersonation, stale evidence reuse, +fail-open outages, and scanner suppression. Ruleset changes require live audit, +independent review, rollback, and consumer proof. + +## Tests and acceptance + +- every evidence class remains non-interchangeable; +- absent/pending/skipped/neutral/cancelled/stale/malformed negative fixtures; +- provider failure cannot synthesize approval or a source finding; +- live ruleset inventory and two-approval/last-pusher/thread assertions; and +- protected-main canary demonstrating all configured authorities and one + negative control. + +## Migration and rollback + +Inventory current gates and issuer identities, separate aggregate signals, +enable fail-closed behavior with diagnostic summaries, then require it in the +ruleset. Rollback reverts a broken implementation while retaining the previous +reviewed gate inventory; it does not disable gates ad hoc. + +## Supersession conditions + +Supersede when a policy engine can cryptographically attest the same independent +authorities, exact revision, live policy version, and fail-closed semantics. diff --git a/docs/automation/adr/README.md b/docs/automation/adr/README.md new file mode 100644 index 000000000..a7d5a1b2d --- /dev/null +++ b/docs/automation/adr/README.md @@ -0,0 +1,28 @@ +# Automation architecture decision records + +Last reviewed: 2026-08-09 + +ADRs record durable decisions, not transient run status. A changed exact head, run ID, or provider outage belongs in traceability/incident evidence. Superseded ADRs remain in the index with their replacement. + +| ADR | Decision | Status | +|---|---|---| +| [ADR-0001](0001-branch-writer-leases-and-read-only-audit.md) | Branch-local writer leases and separate read-only fleet audit | Accepted | +| [ADR-0002](0002-exact-source-and-live-base-binding.md) | Exact source-head and independently resolved live-base binding | Accepted | +| [ADR-0003](0003-classified-bounded-retries.md) | Classified bounded retries with fail-closed permanent failures | Accepted | +| [ADR-0004](0004-explicit-secret-contracts.md) | Explicit minimal reusable-workflow secret contracts | Accepted | +| [ADR-0005](0005-independent-review-authority.md) | Counted independent review and stale-head semantics | Accepted | +| [ADR-0006](0006-protected-main-operational-acceptance.md) | Protected-main/consumer evidence closes operational incidents | Accepted | +| [ADR-0007](0007-work-conserving-maintenance.md) | Work-conserving automation; reporting is not completion | Accepted | +| [ADR-0008](0008-central-control-plane-and-thin-consumers.md) | Central control-plane ownership and thin product consumers | Accepted | +| [ADR-0009](0009-sandbox-evidence-redaction-boundary.md) | One complete sandbox evidence-redaction boundary with diagnostic preservation | Accepted; integration proof pending in `.github#842` | +| [ADR-0010](0010-agent-mention-routing-and-idempotency-ledger.md) | Authenticated agent-mention routing with an idempotency ledger | Accepted | +| [ADR-0011](0011-provider-routing-and-credential-isolation.md) | Ordered provider routing with per-purpose credential isolation | Accepted | +| [ADR-0012](0012-hash-pinned-toolchains-and-exact-base-materialization.md) | Hash-pinned toolchains and independently resolved live-base identity | Accepted | +| [ADR-0013](0013-autofix-and-merge-authority-separation.md) | Separate autofix branch mutation from protected merge authority | Accepted | +| [ADR-0014](0014-trusted-metadata-event-and-default-branch-dispatch.md) | Keep privileged PR events metadata-only and execute validated work from protected source | Accepted | +| [ADR-0015](0015-direct-or-auto-merge-state-machine.md) | Ordered guarded direct merge with narrow native auto-merge fallback | Accepted | +| [ADR-0016](0016-fail-closed-security-gate-composition.md) | Compose independent live security/review authorities fail closed | Accepted | + +## ADR quality contract + +Every ADR includes context, drivers, alternatives, decision, consequences, failure/recovery, security/governance impact, tests/acceptance, migration/rollback, and supersession conditions. A decision that changes any of those sections requires an ADR update or replacement plus traceability and operational evidence. diff --git a/docs/doctoring/automation-control-plane-standards.md b/docs/doctoring/automation-control-plane-standards.md new file mode 100644 index 000000000..08e1d0827 --- /dev/null +++ b/docs/doctoring/automation-control-plane-standards.md @@ -0,0 +1,91 @@ +# Doctoring — automation control-plane standards baseline + +Research date: 2026-08-09 +Scope: architecture, secure development, CI/CD supply chain, GitHub trust, AI-assisted review, observability, and testing + +## Version decisions + +- NIST SP 800-218 version 1.1 is the current **final** Secure Software Development Framework. The proposed version 1.2 was published as an initial public draft on 2025-12-17, so this repository treats it as informative until final. +- NIST SP 800-92 (2006) remains the final log-management publication. SP 800-92 Rev. 1 was published as an initial public draft on 2023-10-11 and remains informative; it does not silently supersede the final publication. +- SLSA version 1.2 is the current SLSA specification. The architecture uses its provenance and verification concepts without claiming a SLSA level that has not been demonstrated. +- ISO/IEC/IEEE 42010:2022 is the architecture-description baseline; the documentation separates stakeholders, viewpoints, models, and concerns. +- ISO/IEC 27001:2022 and ISO/IEC 42001:2023 are management-system alignment references. ISO/IEC 27002:2022 supplies information-security control guidance, including the logging, monitoring, access, and incident concerns relevant here; ISO states that 27002 itself is guidance and is not certifiable. +- The software-testing baseline names concrete published parts rather than an unspecified series: ISO/IEC/IEEE 29119-1:2022 for general concepts and ISO/IEC/IEEE 29119-2:2021 for test processes. Other 29119 parts are not implied unless separately cited and mapped. +- SOC 2 alignment refers specifically to the AICPA 2017 Trust Services Criteria with revised points of focus (2022), not to a self-awarded certification or report. +- CSAP means the Korean Cloud Security Assurance Program (클라우드서비스 보안인증). It is considered only for an in-scope cloud service and applicable Korean public-sector use; this repository is not a CSAP assessment or certificate. +- GitHub's current secure-use guidance warns against executing untrusted content under `pull_request_target` or `workflow_run` privilege. The central metadata-bootstrap/default-branch-dispatch split follows that boundary. +- OpenTelemetry Specification 1.59.0 and its stable log data model are the current observability references observed on the research date. + +## Applied implications + +| Source | Applied control | +|---|---| +| NIST SSDF 1.1 | protected development environment, provenance, verification, vulnerability response, continuous improvement | +| NIST SSDF 1.2 initial public draft | informative gap review only; no draft practice is reported as a final requirement | +| NIST SP 800-204D | stage-separated CI/CD supply-chain controls, integrity verification, policy enforcement, evidence | +| NIST SP 800-92 final / Rev. 1 draft | log generation, transmission, storage, access, disposal, incident use, and retention planning; the draft is informative only | +| SLSA 1.2 | immutable provenance identity, artifact/source verification, no unsupported level claim | +| ISO/IEC/IEEE 42010:2022 | viewpoint-based architecture, explicit stakeholders/concerns, model consistency | +| ISO/IEC 27001:2022 | risk, least privilege, supplier/change/logging/incident governance | +| ISO/IEC 27002:2022 | guidance for information-security controls; no 27002 certification claim | +| ISO/IEC 42001:2023 | human accountability, AI provider governance, validated model output, continual improvement | +| GitHub secure-use and OIDC guidance | trusted workflow source, untrusted input handling, minimal token permissions, short-lived federation | +| OpenTelemetry | correlation by time and execution identity across logs, metrics, and traces | +| ISO/IEC/IEEE 29119-1:2022 and 29119-2:2021 | common testing concepts and governed test processes across lifecycle models | +| AICPA 2017 Trust Services Criteria (revised points of focus, 2022) | evidence design for security, availability, processing integrity, confidentiality, and privacy; not a SOC 2 report | +| Korean CSAP program and applicable notice | conditional readiness questions for an in-scope cloud service; no certification, tier, or public-sector eligibility claim | + +## Alignment and certification limits + +- NIST publications and SLSA concepts are engineering guidance here. Their citation does not establish regulatory compliance, a SLSA level, or independent assurance. +- ISO/IEC 27002 provides control guidance and cannot itself be certified. ISO/IEC 27001 or ISO/IEC 42001 certification would require a defined management-system scope, implemented controls, evidence over time, and an authorized independent certification process. +- ISO/IEC/IEEE 29119 alignment means that the test strategy uses compatible concepts and process concerns. No conformance assessment against either named part has been performed. +- SOC 2 is an attestation engagement using AICPA criteria. Repository tests and traceability can contribute evidence, but they are not a SOC 2 report and do not establish control design or operating effectiveness for a scoped service period. +- Korean CSAP evaluates an actual cloud service against applicable legal and program criteria. Eligibility, service type/tier, data location, isolation, operational controls, and the current governing notice must be resolved for the real deployment with KISA or an authorized assessor. Source-repository controls alone cannot establish CSAP readiness or certification. +- Where a cited draft and final publication coexist, the final publication is normative for this baseline and the draft is informative until its official status changes. + +## APA 7 references + +American Institute of Certified Public Accountants, Assurance Services Executive Committee. (2023). *2017 Trust Services Criteria for security, availability, processing integrity, confidentiality, and privacy (with revised points of focus—2022).* https://www.aicpa-cima.com/resources/download/2017-trust-services-criteria-with-revised-points-of-focus-2022 + +Booth, H., Ogata, M., Kent, K., Souppaya, M., & Dodson, D. (2025). *Secure software development framework (SSDF) version 1.2: Recommendations for mitigating the risk of software vulnerabilities* (Initial Public Draft, NIST Special Publication 800-218 Rev. 1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-218r1.ipd + +Chandramouli, R., Kautz, F., & Torres Arias, S. (2024). *Strategies for the integration of software supply chain security in DevSecOps CI/CD pipelines* (NIST Special Publication 800-204D). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-204D + +GitHub. (n.d.). *About protected branches*. Retrieved August 9, 2026, from https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches + +GitHub. (n.d.). *OpenID Connect*. Retrieved August 9, 2026, from https://docs.github.com/en/actions/concepts/security/openid-connect + +GitHub. (n.d.). *Reuse workflows*. Retrieved August 9, 2026, from https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows + +GitHub. (n.d.). *Secure use reference*. Retrieved August 9, 2026, from https://docs.github.com/en/actions/reference/security/secure-use + +International Organization for Standardization. (2022b). *Information security, cybersecurity and privacy protection—Information security management systems—Requirements* (ISO/IEC 27001:2022). https://www.iso.org/standard/27001 + +International Organization for Standardization. (2022a). *Information security, cybersecurity and privacy protection—Information security controls* (ISO/IEC 27002:2022). https://www.iso.org/standard/75652.html + +International Organization for Standardization. (2023). *Information technology—Artificial intelligence—Management system* (ISO/IEC 42001:2023). https://www.iso.org/standard/42001 + +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2021). *Software and systems engineering—Software testing—Part 2: Test processes* (ISO/IEC/IEEE 29119-2:2021). https://www.iso.org/standard/79428.html + +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2022a). *Software and systems engineering—Software testing—Part 1: General concepts* (ISO/IEC/IEEE 29119-1:2022). https://www.iso.org/standard/81291.html + +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2022b). *Software, systems and enterprise—Architecture description* (ISO/IEC/IEEE 42010:2022). https://www.iso.org/standard/74393.html + +Kent, K., & Souppaya, M. (2006). *Guide to computer security log management* (NIST Special Publication 800-92). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-92 + +Korea Internet & Security Agency. (n.d.). *클라우드서비스 보안인증(CSAP).* Retrieved August 9, 2026, from https://www.kisa.or.kr/1050603 + +과학기술정보통신부. (2023). *클라우드컴퓨팅서비스 보안인증에 관한 고시* (과학기술정보통신부고시 제2023-4호). 국가법령정보센터. https://law.go.kr/LSW/admRulInfoP.do?admRulSeq=2100000218804 + +OpenTelemetry Authors. (2026). *OpenTelemetry specification 1.59.0*. https://opentelemetry.io/docs/specs/otel/ + +Open Source Security Foundation. (2026). *SLSA specification, version 1.2*. https://slsa.dev/spec/v1.2/ + +Scarfone, K., & Souppaya, M. (2023). *Cybersecurity log management planning guide* (Initial Public Draft, NIST Special Publication 800-92 Rev. 1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-92r1.ipd + +Souppaya, M., Scarfone, K., & Dodson, D. (2022). *Secure software development framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities* (NIST Special Publication 800-218). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-218 + +## Review cadence + +Re-check final/draft status and current versions at least quarterly and whenever a boundary-changing PR cites these sources. Replace a source only through an ADR/traceability update; do not silently change the normative baseline. diff --git a/docs/org-required-workflow-rollout.md b/docs/org-required-workflow-rollout.md index 9c42ab063..c1c88cdb6 100644 --- a/docs/org-required-workflow-rollout.md +++ b/docs/org-required-workflow-rollout.md @@ -2,7 +2,14 @@ Updated: 2026-07-23 06:35 KST -## Decision +> **Historical rollout ledger, not a live ruleset receipt.** This document +> preserves dated rollout observations, exact heads, and migrations. The live +> organization state must be re-fetched before mutation, and the durable current +> contract is indexed in [`docs/automation/README.md`](automation/README.md). +> In particular, later entries supersede the early `~ALL` narrative with the +> explicit `.github`, `IRT-bibliography-set`, and `noema` exclusions. + +## Decision at the recorded rollout snapshot Use an organization repository ruleset instead of copying workflow files into each repository. diff --git a/docs/scorecard-governance.md b/docs/scorecard-governance.md index 80f6cc130..6e4db65f3 100644 --- a/docs/scorecard-governance.md +++ b/docs/scorecard-governance.md @@ -16,15 +16,18 @@ or the inherited organization ruleset must require all of the following: - stale approvals to be dismissed after a push; - current-head OpenCode review evidence from the central required workflow; - code owner review coverage through CODEOWNERS-owned workflow and CI paths, - with the organization required-workflow ruleset carrying the enforceable - single-maintainer approval gate; + with the live organization ruleset carrying the enforceable exactly-two- + approval gate and last-pusher separation; - review thread resolution before merge; - last-pusher approval protection; - force-push and branch deletion protection. The organization required-workflow ruleset remains the distribution layer for -other repositories. The branch protection rule is kept as the repository-local -signal that OpenSSF Scorecard can evaluate directly. +included repositories. The central `.github` repository, `noema`, and private +`IRT-bibliography-set` are deliberate exclusions verified by +`scripts/ci/audit_central_required_workflows.py`; they require repository-local +protection appropriate to their role. The branch protection rule is kept as the +repository-local signal that OpenSSF Scorecard can evaluate directly. ## MaintainedID diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py new file mode 100644 index 000000000..0e97983f5 --- /dev/null +++ b/tests/test_automation_documentation_contract.py @@ -0,0 +1,272 @@ +"""Contract tests for the authoritative automation documentation graph.""" + +from __future__ import annotations + +import re +from pathlib import Path + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[1] +AUTOMATION_DOCS = REPOSITORY_ROOT / "docs" / "automation" + +REQUIRED_DOCUMENTS = ( + "README.md", + "PRD.md", + "TRD.md", + "ARCHITECTURE.md", + "DATA_MODEL.md", + "ERD.md", + "UML.md", + "SECURITY.md", + "THREAT_MODEL.md", + "TEST_STRATEGY.md", + "OPERABILITY.md", + "INCIDENT_RUNBOOK.md", + "RUNBOOK.md", + "TRACEABILITY.md", +) + +REQUIRED_ADRS = ( + "0001-branch-writer-leases-and-read-only-audit.md", + "0002-exact-source-and-live-base-binding.md", + "0003-classified-bounded-retries.md", + "0004-explicit-secret-contracts.md", + "0005-independent-review-authority.md", + "0006-protected-main-operational-acceptance.md", + "0007-work-conserving-maintenance.md", + "0008-central-control-plane-and-thin-consumers.md", + "0009-sandbox-evidence-redaction-boundary.md", + "0010-agent-mention-routing-and-idempotency-ledger.md", + "0011-provider-routing-and-credential-isolation.md", + "0012-hash-pinned-toolchains-and-exact-base-materialization.md", + "0013-autofix-and-merge-authority-separation.md", + "0014-trusted-metadata-event-and-default-branch-dispatch.md", + "0015-direct-or-auto-merge-state-machine.md", + "0016-fail-closed-security-gate-composition.md", +) + +ADR_SECTIONS = ( + "## Context", + "## Decision drivers", + "## Alternatives considered", + "## Decision", + "## Consequences", + "## Failure and recovery", + "## Security and governance impact", + "## Tests and acceptance", + "## Migration and rollback", + "## Supersession conditions", +) + +DATA_ENTITIES = ( + "automation_run", + "repository_target", + "pull_request_snapshot", + "source_revision", + "base_revision", + "merge_revision", + "check_evidence", + "status_evidence", + "review_evidence", + "model_evidence", + "workflow_evidence", + "dependency_evidence", + "incident_hypothesis", + "handoff_record", + "operational_acceptance", + "secret_requirement", + "writer_lease", + "organization_target", + "orchestration_run", + "ruleset_snapshot", + "dispatch_envelope", + "invocation_claim", + "scheduler_decision", + "review_thread", + "security_finding", + "sandbox_evidence", + "sbom_snapshot", +) + + +def read_document(relative_path: str) -> str: + """Return one automation document as UTF-8 text.""" + + return (AUTOMATION_DOCS / relative_path).read_text(encoding="utf-8") + + +def test_authoritative_document_set_exists_and_is_indexed() -> None: + """Every required document exists, has a heading, and appears in the index.""" + + index = read_document("README.md") + for relative_path in REQUIRED_DOCUMENTS: + document_path = AUTOMATION_DOCS / relative_path + assert document_path.is_file(), relative_path + content = document_path.read_text(encoding="utf-8") + assert content.startswith("# "), relative_path + if relative_path != "README.md": + assert f"({relative_path})" in index, relative_path + + assert (REPOSITORY_ROOT / "ARCHITECTURE.md").is_file() + for entrypoint in ("README.md", "AGENTS.md", "CLAUDE.md", "ARCHITECTURE.md"): + assert "docs/automation/README.md" in ( + REPOSITORY_ROOT / entrypoint + ).read_text(encoding="utf-8") + + +def test_adrs_are_indexed_and_use_the_complete_decision_template() -> None: + """The ADR index and every decision retain the required analysis sections.""" + + adr_index = read_document("adr/README.md") + for relative_path in REQUIRED_ADRS: + assert f"({relative_path})" in adr_index + content = read_document(f"adr/{relative_path}") + assert content.startswith("# ADR-") + assert "Status:" in content and "Date:" in content + for section in ADR_SECTIONS: + assert section in content, f"{relative_path}: {section}" + + +def test_markdown_links_and_mermaid_fences_are_well_formed() -> None: + """Local links resolve and every Markdown/Mermaid fence is balanced.""" + + link_pattern = re.compile(r"\[[^\]]+\]\(([^)]+)\)") + for document_path in AUTOMATION_DOCS.rglob("*.md"): + content = document_path.read_text(encoding="utf-8") + assert content.count("```") % 2 == 0, document_path + for target in link_pattern.findall(content): + if "://" in target or target.startswith(("#", "mailto:")): + continue + relative_target = target.split("#", 1)[0] + assert (document_path.parent / relative_target).resolve().exists(), ( + document_path, + target, + ) + + +def test_architecture_references_real_control_plane_sources() -> None: + """Architecture terms remain tied to current workflow and script filenames.""" + + architecture = read_document("ARCHITECTURE.md") + read_document("TRD.md") + required_sources = ( + ".github/workflows/opencode-review.yml", + ".github/workflows/opencode-review-dispatch.yml", + ".github/workflows/noema-review.yml", + ".github/workflows/strix.yml", + "scripts/ci/pr_review_merge_scheduler.py", + "scripts/ci/pr_review_fix_scheduler.py", + "scripts/ci/redact_sensitive_log.py", + "scripts/ci/sandboxed_verify.py", + "scripts/ci/sandboxed_web_e2e.py", + ) + for relative_path in required_sources: + assert (REPOSITORY_ROOT / relative_path).exists(), relative_path + assert Path(relative_path).name in architecture, relative_path + + +def test_conceptual_erd_uses_explicit_two_word_snake_case_entities() -> None: + """The evidence model preserves authority entities and naming conventions.""" + + data_model = read_document("DATA_MODEL.md") + for entity_name in DATA_ENTITIES: + assert re.fullmatch(r"[a-z]+(?:_[a-z]+)+", entity_name) + assert entity_name in data_model + assert "conceptual and logical" in data_model + assert "not a claim" in data_model + assert "pull_request_snapshot }o--|| source_revision" in data_model + assert "automation_run }o--o| source_revision" in data_model + assert "merge_revision ||--o{ operational_acceptance" in data_model + assert "source_revision ||--o{ operational_acceptance" not in data_model + assert "orchestration_run ||--o{ automation_run" in data_model + assert "string source_kind" in data_model + + +def test_exact_head_and_stable_interface_contracts_are_explicit() -> None: + """Critical invalidation, dispatch, marker, and receipt shapes stay named.""" + + uml = read_document("UML.md") + trd = read_document("TRD.md") + assert "GateClean --> Stale" in uml + assert "direct_or_auto" in uml + for marker in ("SANDBOXED_VERIFY_RESULT", "SANDBOXED_WEB_E2E_RESULT"): + assert marker in trd + for field in ("allowed_env", "elapsed_seconds", "evidence_note", "exit_code"): + assert field in trd + assert 'pr-review-merge-scheduler/v2' in trd + assert "cwl-agent-invocation-" in trd + + +def test_traceability_has_exact_identifiers_and_accountability_columns() -> None: + """The matrix carries stable IDs and exact implementation accountability.""" + + traceability = read_document("TRACEABILITY.md") + for number in range(1, 15): + assert f"`PRD-{number:02d}`" in traceability + for requirement_id in ( + "TRD-EVT-01", + "TRD-REV-01", + "TRD-AUTH-01", + "TRD-RUN-01", + "TRD-WRITE-01", + "TRD-SEC-01", + "TRD-RETRY-01", + "TRD-LOG-01", + "TRD-IF-01", + "TRD-RET-01", + ): + assert requirement_id in traceability + for heading in ( + "Exact implementation source", + "Exact tests", + "Required gate / receipt authority", + "Owner / closure target", + "Status", + ): + assert heading in traceability + + +def test_security_and_operations_contracts_keep_authorities_separate() -> None: + """High-risk credential, evidence, privacy, and closure rules stay explicit.""" + + corpus = "\n".join( + read_document(relative_path) + for relative_path in ( + "PRD.md", + "TRD.md", + "SECURITY.md", + "THREAT_MODEL.md", + "OPERABILITY.md", + "INCIDENT_RUNBOOK.md", + "TRACEABILITY.md", + ) + ) + for required_term in ( + "NVIDIA_NIM_API_KEY", + "COPILOT_GITHUB_TOKEN", + "secrets: inherit", + "pull_request_target", + "repository_dispatch", + "expected-head", + "qualifying independent", + "protected-main", + "PII", + ): + assert required_term in corpus + + +def test_timeless_architecture_does_not_embed_transient_run_identity() -> None: + """Stable design documents avoid historical exact heads and run identifiers.""" + + stable_corpus = "\n".join( + read_document(relative_path) + for relative_path in ( + "PRD.md", + "TRD.md", + "ARCHITECTURE.md", + "DATA_MODEL.md", + "UML.md", + "SECURITY.md", + ) + ) + assert re.search(r"\b[0-9a-f]{40}\b", stable_corpus) is None + assert re.search(r"\brun(?:_id)?[ =`]\d{8,}\b", stable_corpus, re.IGNORECASE) is None diff --git a/tests/test_pr_governance_audit_contract.py b/tests/test_pr_governance_audit_contract.py index fe4cbdec7..cb133ad82 100644 --- a/tests/test_pr_governance_audit_contract.py +++ b/tests/test_pr_governance_audit_contract.py @@ -1,6 +1,18 @@ from pathlib import Path +def test_governance_audit_is_explicitly_historical_and_stale(): + """A dated fleet fixture must never masquerade as live mutation authority.""" + + audit = Path("PR_GOVERNANCE_AUDIT.md").read_text(encoding="utf-8") + + assert "historical snapshot" in audit.lower() + assert "not current authority" in audit + assert "re-fetch" in audit.lower() + assert "Freshness policy" in audit + assert "docs/automation/README.md" in audit + + def test_html4tree_public_fork_queue_requires_central_review_gate(): """Guard the html4tree onboarding gap documented by the live audit.""" audit = Path("PR_GOVERNANCE_AUDIT.md").read_text(encoding="utf-8") From d647acf9cfd704ec188a3b7fadaea0795d4b3779 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:02:32 +0900 Subject: [PATCH 02/69] docs(automation): reconcile whole-conversation documentation scope --- docs/automation/DOCUMENTATION_AUDIT.md | 95 ++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/automation/DOCUMENTATION_AUDIT.md diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md new file mode 100644 index 000000000..857491e4a --- /dev/null +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -0,0 +1,95 @@ +# Whole-conversation documentation fitness audit + +Status: active_pr +Last reviewed: 2026-08-09 +Scope: `ContextualWisdomLab/.github` automation control plane and its contracts with thin repository consumers + +## Purpose + +This audit answers whether the durable PRD, TRD, Architecture, UML, ERD/data model, security, testing, operations, ADR, and traceability set is sufficient for the durable decisions represented across the current CWL project conversation and planning material. + +Conversation history, prompts, PR bodies, attached planning files, broad master-context documents, and model output are **candidate evidence**, not architectural authority. A decision enters the canonical graph only after it is revalidated against protected-main implementation, current GitHub state, or an explicit accepted-architecture decision. Product-specific detail remains owned by the product repository rather than being copied into the organization control plane. + +## Controlled maturity vocabulary + +Every durable claim in this documentation graph MUST map to one of these states: + +- `implemented_on_protected_main` — observable in the current protected default-branch source and covered by current contract/evidence expectations. +- `active_pr` — implemented or documented only on an open pull request; not shipped truth. +- `accepted_architecture` — approved intended contract whose implementation may be incomplete. +- `planned` — bounded future work with no implementation claim. +- `research_only` — evidence or design exploration that has not crossed an architecture/release gate. +- `superseded` — preserved historical evidence replaced by a newer accepted decision or integration path. +- `out_of_scope` — intentionally owned elsewhere or excluded from this control plane. + +`accepted baseline`, `implemented`, `partial`, `proposed`, and `operational-proof-pending` in older documents are descriptive aliases only and MUST be interpreted through the mapping above before a runtime claim is made. + +## Whole-conversation ownership reconciliation + +| Durable conversation/planning family | Central control-plane relevance | Canonical disposition | +|---|---|---| +| PR review → repair → current-head checks → merge → next work loops | Direct | `accepted_architecture`; PRD/TRD/ADR-0007 define work conservation and authority. | +| Hourly autonomous continuation and no report-as-completion | Direct | `accepted_architecture`; every prompt edit, documentation assessment/update, inventory, status comment, review request, CI dispatch, Draft/Ready transition, auto-merge enablement, commit, merge, document update, or one buyer slice is an intermediate event. | +| Exact source-head versus PR-base snapshot versus independently resolved live-base tip | Direct | `accepted_architecture`; TRD revision identity and ADR-0002. | +| Reviewer/check/status/model/merge/release authority separation | Direct | `accepted_architecture`; TRD evidence taxonomy and ADR-0005/0016. | +| Central `.github` ownership with thin modular leaf consumers | Direct | `accepted_architecture`; Architecture and ADR-0008. | +| NVIDIA NIM development-agent policy and prohibition on `COPILOT_GITHUB_TOKEN` | Direct | `accepted_architecture`; PRD/TRD/Security/ADR-0004/0011. | +| CSAP/SOC 2 evidence-readiness, PII purpose/access/retention/audit controls | Direct | `accepted_architecture`; Security/Threat Model/Operability/standards doctoring; no certification claim. | +| TEPP / multilingual structural topic measurement / psychometrics compute design | Shared quality/research principles only | `out_of_scope` here; detailed PRD/TRD/model equations belong to the TEPP/psychometrics repository. Central docs retain only reusable CI/review/coverage interfaces. | +| fast-mlsirm psychometric recovery/GPU contracts | Shared quality evidence | `out_of_scope` product detail; central control plane owns reusable evidence transport and merge gates only. | +| OriginWeave agentic browser runtime | Shared automation/agent integration boundary only | `out_of_scope` product architecture; OriginWeave owns browser/runtime design. | +| EmbedRelay embedding interoperability | Shared release/security/test policy only | `out_of_scope` product architecture; EmbedRelay owns adapter/vector design. | +| MHTML ETL Gateway | Shared CI/security/release policy only | `out_of_scope` product architecture; ETL repository owns parser/data/API design. | +| LifeOS | Shared CI/security/release policy only | `out_of_scope` product architecture; LifeOS owns product/data/auth design. | +| BandScope / Inkspan / pg-erd-cloud / naruon / AppGuardrail and other leaf products | Thin caller, review, security, release and evidence interfaces | `out_of_scope` product semantics; central docs specify only the reusable contract and consumer acceptance requirements. | +| Organization ecosystem composition through naruon/contextual-orchestrator/connectors | Interface-level concern | `accepted_architecture` only for central interface ownership, credentials, provenance and thin-consumer rules; business composition is leaf-owned. | + +## Documentation family fitness + +| Artifact family | Fitness on this PR | Required residual action | +|---|---|---| +| PRD | ADEQUATE | Keep the two operating modes, degraded behavior and measurable acceptance aligned with live implementation. | +| TRD | ADEQUATE | Preserve explicit event, identity, authority, secret, retry, timeout, lease, result and compatibility contracts. | +| Architecture | ADEQUATE | Keep central/leaf bounded contexts, data/control planes, trust and failure domains current. | +| UML | ADEQUATE | Machine-check all required component, sequence, state, authority, retry, deployment, lease and continuation flows. | +| ERD / data model | PARTIAL | Add first-class aliases/entities for remediation alternatives, continuation handoff, documentation artifacts and traceability records; remain conceptual until a persistence ADR is accepted. | +| Security / threat model | ADEQUATE | Keep credential, confused-deputy, untrusted-source, prompt-injection, stale-evidence, PII and supply-chain cases current. | +| Test strategy | ADEQUATE | Add documentation-fitness regression coverage for this audit, maturity mapping and standards freshness. | +| Operability / incident runbook | ADEQUATE | Preserve protected-main/consumer closure and queue/provider/runner/DNS recovery evidence. | +| ADR set | ADEQUATE | Extend work-conservation wording with the explicit intermediate-event/no-soft-timeout invariant. | +| Traceability | PARTIAL | Add explicit documentation-artifact and continuation/remediation mappings and current requirements/product-quality standards. | +| Standards doctoring | PARTIAL | Keep SLSA 1.2 and final-vs-draft SSDF handling; add ISO/IEC/IEEE 29148:2018 and ISO/IEC 25010:2023. | + +## No-soft-timeout continuation invariant + +The hourly recurrence is a continuation mechanism after genuine practical execution/tool-budget exhaustion. It is not a voluntary wall-clock timeout. + +The following events have zero terminal credit while any safe lane remains: prompt update, documentation assessment or update, inventory, RCA without remediation, status/report comment, review request, workflow dispatch/rerun, queued or running CI/model evidence, Draft/Ready transition, auto-merge enablement, commit, merge, document completion, protected-main acceptance of one scenario, or completion of one buyer-visible slice. + +Before any voluntary termination the automation performs a fresh whole-queue sweep. If it finds an executable mutation, test, thread resolution, documentation repair, operational acceptance, issue action, merge, or bounded product/control-plane action, it executes that work and sweeps again. Only actual practical run/tool-budget exhaustion or a **second** fresh sweep proving all lanes non-actionable permits termination. + +## Evidence-to-canonicalization flow + +```mermaid +flowchart LR + A[Conversation / planning / PR evidence] --> B[Revalidate against live GitHub and protected main] + B --> C{Durable central decision?} + C -->|yes| D[Classify maturity state] + D --> E[PRD / TRD / Architecture / ADR / UML / ERD] + E --> F[Tests and traceability] + F --> G[Protected merge] + G --> H[Protected-main or consumer acceptance] + H --> I[implemented_on_protected_main] + C -->|leaf-owned| J[Explicit out_of_scope handoff to owning repository] + C -->|obsolete| K[superseded historical evidence] +``` + +## Exit condition for this documentation repair + +This audit is sufficient only when: + +1. it is indexed by `docs/automation/README.md`; +2. machine tests validate the controlled maturity vocabulary, whole-conversation ownership boundary, double-exit/no-soft-timeout rule, conceptual-vs-persisted boundary, standards freshness and local links; +3. the residual ERD/traceability/standards gaps above are repaired on the same canonical documentation line; +4. exact-head documentation/security/supply-chain checks and current review are clean; and +5. after protected merge, the canonical index is read from protected `main` before any `implemented_on_protected_main` claim is made. From b4f825326b16055bf7a37025b27d4182b69f3425 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:03:14 +0900 Subject: [PATCH 03/69] docs(automation): index whole-conversation fitness audit --- docs/automation/README.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/automation/README.md b/docs/automation/README.md index dbd60fd10..51a743b78 100644 --- a/docs/automation/README.md +++ b/docs/automation/README.md @@ -20,16 +20,20 @@ This directory is the durable specification for the ContextualWisdomLab automati and executable [RUNBOOK.md](RUNBOOK.md) — service objectives, observability, diagnosis, ownership, recovery, retention, rollback, and acceptance receipts. 9. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. -10. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. +10. [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md) — whole-conversation fitness, central-versus-leaf ownership, maturity states, and no-soft-timeout continuation contract. +11. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. -## Authority and change control +## Maturity and authority discipline + +A document being an accepted baseline does not mean every described behavior is already shipped. Durable claims map to the controlled maturity vocabulary in [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md): `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, or `out_of_scope`. The sources of truth are ordered as follows: 1. live GitHub repository, pull-request, ruleset, check, review, and workflow evidence for current state; 2. protected default-branch workflow and script source for implemented behavior; 3. this documentation set for intended cross-component contracts; -4. dated incident receipts and pull-request bodies for historical evidence only. +4. dated incident receipts and pull-request bodies for historical evidence only; and +5. conversation, prompts, planning packs, and model output as candidate evidence only until revalidated and canonicalized. Historical SHAs, run IDs, check counts, and performance numbers belong in dated evidence or pull-request records, not timeless architecture. A head change makes predecessor-head checks and reviews historical. A base-branch change requires independently resolving the new live base tip and regenerating evidence. @@ -45,4 +49,6 @@ Changes to triggers, permissions, secret names, evidence identity, retry classes The central repository owns shared policy, trusted workflow entrypoints, evidence normalization, review dispatch, and merge/fix scheduling. Product repositories own their source, product-specific tests, release gates, deployments, data, and thin calls into central contracts. Central automation must not silently become an application runtime, a product database, an approval impersonator, or a repository-specific business workflow. -The hourly commercial-maintenance automation is an orchestration policy outside GitHub Actions. GitHub Actions remains the event-driven execution and evidence plane. The hourly policy must use live repository evidence and must not treat its private memory as authority. +The whole-conversation audit therefore does **not** copy product PRDs such as TEPP/psychometrics, OriginWeave, EmbedRelay, MHTML ETL, LifeOS, BandScope, Inkspan, pg-erd-cloud, naruon, or AppGuardrail into this repository. It records only their shared central interface and explicitly hands product semantics back to the owning repository. + +The hourly commercial-maintenance automation is an orchestration policy outside GitHub Actions. GitHub Actions remains the event-driven execution and evidence plane. The hourly policy must use live repository evidence and must not treat its private memory as authority. The hourly recurrence is continuation after genuine practical execution/tool-budget exhaustion, not a voluntary soft wall-clock timeout; meta actions never replace the required fresh exit sweeps. \ No newline at end of file From 81fc49207ae69f053a61d923c4ec5eeb525df98f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:03:57 +0900 Subject: [PATCH 04/69] docs(standards): add requirements and product-quality baselines --- .../automation-control-plane-standards.md | 20 +++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/docs/doctoring/automation-control-plane-standards.md b/docs/doctoring/automation-control-plane-standards.md index 08e1d0827..604e6e508 100644 --- a/docs/doctoring/automation-control-plane-standards.md +++ b/docs/doctoring/automation-control-plane-standards.md @@ -1,7 +1,7 @@ # Doctoring — automation control-plane standards baseline Research date: 2026-08-09 -Scope: architecture, secure development, CI/CD supply chain, GitHub trust, AI-assisted review, observability, and testing +Scope: architecture, requirements engineering, product quality, secure development, CI/CD supply chain, GitHub trust, AI-assisted review, observability, and testing ## Version decisions @@ -9,6 +9,8 @@ Scope: architecture, secure development, CI/CD supply chain, GitHub trust, AI-as - NIST SP 800-92 (2006) remains the final log-management publication. SP 800-92 Rev. 1 was published as an initial public draft on 2023-10-11 and remains informative; it does not silently supersede the final publication. - SLSA version 1.2 is the current SLSA specification. The architecture uses its provenance and verification concepts without claiming a SLSA level that has not been demonstrated. - ISO/IEC/IEEE 42010:2022 is the architecture-description baseline; the documentation separates stakeholders, viewpoints, models, and concerns. +- ISO/IEC/IEEE 29148:2018 is the current published requirements-engineering baseline and was confirmed in 2024. The Edition 3 Draft International Standard under development in 2026 is informative only until published; draft requirements MUST NOT be reported as the current final baseline. +- ISO/IEC 25010:2023 is the current product-quality-model baseline used to structure relevant correctness, reliability, security, maintainability, compatibility, usability/interaction, performance-efficiency, flexibility, and safety quality concerns without claiming formal conformance. - ISO/IEC 27001:2022 and ISO/IEC 42001:2023 are management-system alignment references. ISO/IEC 27002:2022 supplies information-security control guidance, including the logging, monitoring, access, and incident concerns relevant here; ISO states that 27002 itself is guidance and is not certifiable. - The software-testing baseline names concrete published parts rather than an unspecified series: ISO/IEC/IEEE 29119-1:2022 for general concepts and ISO/IEC/IEEE 29119-2:2021 for test processes. Other 29119 parts are not implied unless separately cited and mapped. - SOC 2 alignment refers specifically to the AICPA 2017 Trust Services Criteria with revised points of focus (2022), not to a self-awarded certification or report. @@ -26,6 +28,9 @@ Scope: architecture, secure development, CI/CD supply chain, GitHub trust, AI-as | NIST SP 800-92 final / Rev. 1 draft | log generation, transmission, storage, access, disposal, incident use, and retention planning; the draft is informative only | | SLSA 1.2 | immutable provenance identity, artifact/source verification, no unsupported level claim | | ISO/IEC/IEEE 42010:2022 | viewpoint-based architecture, explicit stakeholders/concerns, model consistency | +| ISO/IEC/IEEE 29148:2018 | explicit stakeholder/system/software requirements, requirement attributes, traceability, validation, and controlled change | +| ISO/IEC/IEEE 29148 Edition 3 DIS | informative future-gap review only until publication; never substitute draft text for the current final edition | +| ISO/IEC 25010:2023 | product-quality characteristics drive PRD/NFR/test/operability quality coverage and buyer-visible quality gaps | | ISO/IEC 27001:2022 | risk, least privilege, supplier/change/logging/incident governance | | ISO/IEC 27002:2022 | guidance for information-security controls; no 27002 certification claim | | ISO/IEC 42001:2023 | human accountability, AI provider governance, validated model output, continual improvement | @@ -38,6 +43,7 @@ Scope: architecture, secure development, CI/CD supply chain, GitHub trust, AI-as ## Alignment and certification limits - NIST publications and SLSA concepts are engineering guidance here. Their citation does not establish regulatory compliance, a SLSA level, or independent assurance. +- ISO/IEC/IEEE 42010, ISO/IEC/IEEE 29148, and ISO/IEC 25010 provide architecture, requirements, and quality-model baselines. Repository documentation may be aligned to their concepts, but no formal conformance assessment is claimed. - ISO/IEC 27002 provides control guidance and cannot itself be certified. ISO/IEC 27001 or ISO/IEC 42001 certification would require a defined management-system scope, implemented controls, evidence over time, and an authorized independent certification process. - ISO/IEC/IEEE 29119 alignment means that the test strategy uses compatible concepts and process concerns. No conformance assessment against either named part has been performed. - SOC 2 is an attestation engagement using AICPA criteria. Repository tests and traceability can contribute evidence, but they are not a SOC 2 report and do not establish control design or operating effectiveness for a scoped service period. @@ -60,11 +66,15 @@ GitHub. (n.d.). *Reuse workflows*. Retrieved August 9, 2026, from https://docs.g GitHub. (n.d.). *Secure use reference*. Retrieved August 9, 2026, from https://docs.github.com/en/actions/reference/security/secure-use +International Organization for Standardization. (2022a). *Information security, cybersecurity and privacy protection—Information security controls* (ISO/IEC 27002:2022). https://www.iso.org/standard/75652.html + International Organization for Standardization. (2022b). *Information security, cybersecurity and privacy protection—Information security management systems—Requirements* (ISO/IEC 27001:2022). https://www.iso.org/standard/27001 -International Organization for Standardization. (2022a). *Information security, cybersecurity and privacy protection—Information security controls* (ISO/IEC 27002:2022). https://www.iso.org/standard/75652.html +International Organization for Standardization. (2023a). *Systems and software engineering—Systems and software Quality Requirements and Evaluation (SQuaRE)—Product quality model* (ISO/IEC 25010:2023). https://www.iso.org/standard/78176.html -International Organization for Standardization. (2023). *Information technology—Artificial intelligence—Management system* (ISO/IEC 42001:2023). https://www.iso.org/standard/42001 +International Organization for Standardization. (2023b). *Information technology—Artificial intelligence—Management system* (ISO/IEC 42001:2023). https://www.iso.org/standard/42001 + +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2018). *Systems and software engineering—Life cycle processes—Requirements engineering* (ISO/IEC/IEEE 29148:2018). https://www.iso.org/standard/72089.html International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2021). *Software and systems engineering—Software testing—Part 2: Test processes* (ISO/IEC/IEEE 29119-2:2021). https://www.iso.org/standard/79428.html @@ -72,6 +82,8 @@ International Organization for Standardization, International Electrotechnical C International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2022b). *Software, systems and enterprise—Architecture description* (ISO/IEC/IEEE 42010:2022). https://www.iso.org/standard/74393.html +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2026). *Systems and software engineering—Life cycle processes—Requirements engineering* (ISO/IEC/IEEE DIS 29148, Edition 3, draft). https://www.iso.org/standard/89856.html + Kent, K., & Souppaya, M. (2006). *Guide to computer security log management* (NIST Special Publication 800-92). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-92 Korea Internet & Security Agency. (n.d.). *클라우드서비스 보안인증(CSAP).* Retrieved August 9, 2026, from https://www.kisa.or.kr/1050603 @@ -88,4 +100,4 @@ Souppaya, M., Scarfone, K., & Dodson, D. (2022). *Secure software development fr ## Review cadence -Re-check final/draft status and current versions at least quarterly and whenever a boundary-changing PR cites these sources. Replace a source only through an ADR/traceability update; do not silently change the normative baseline. +Re-check final/draft status and current versions at least quarterly and whenever a boundary-changing PR cites these sources. Replace a source only through an ADR/traceability update; do not silently change the normative baseline. \ No newline at end of file From 6c160ea36eb252a415688a2ae78a9f1b1e51b63e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:04:42 +0900 Subject: [PATCH 05/69] docs(standards): correct requirements draft source --- docs/doctoring/automation-control-plane-standards.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/doctoring/automation-control-plane-standards.md b/docs/doctoring/automation-control-plane-standards.md index 604e6e508..c17466533 100644 --- a/docs/doctoring/automation-control-plane-standards.md +++ b/docs/doctoring/automation-control-plane-standards.md @@ -82,7 +82,7 @@ International Organization for Standardization, International Electrotechnical C International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2022b). *Software, systems and enterprise—Architecture description* (ISO/IEC/IEEE 42010:2022). https://www.iso.org/standard/74393.html -International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2026). *Systems and software engineering—Life cycle processes—Requirements engineering* (ISO/IEC/IEEE DIS 29148, Edition 3, draft). https://www.iso.org/standard/89856.html +International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2026). *Systems and software engineering—Life cycle processes—Requirements engineering* (ISO/IEC/IEEE DIS 29148, Edition 3, draft). https://www.iso.org/standard/94091.html Kent, K., & Souppaya, M. (2006). *Guide to computer security log management* (NIST Special Publication 800-92). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-92 From 0f37e9af7cfd7d3552f7dfd9204ecbe93b7cf8b5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:05:20 +0900 Subject: [PATCH 06/69] docs(adr): prohibit soft-timeout and meta-action completion --- .../adr/0007-work-conserving-maintenance.md | 26 ++++++++++++------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/docs/automation/adr/0007-work-conserving-maintenance.md b/docs/automation/adr/0007-work-conserving-maintenance.md index a28553d66..438e0e80b 100644 --- a/docs/automation/adr/0007-work-conserving-maintenance.md +++ b/docs/automation/adr/0007-work-conserving-maintenance.md @@ -6,7 +6,7 @@ Decision owners: CWL product and automation maintainers ## Context -Long model reviews, human approval, queued checks, provider cooldowns, and one blocked PR repeatedly caused finite maintenance invocations to stop after reporting status. An hourly recurrence then became an excuse to defer work despite safe tasks in other branches, docs, issues, operations, or product gaps. +Long model reviews, human approval, queued checks, provider cooldowns, and one blocked PR repeatedly caused finite maintenance invocations to stop after reporting status. An hourly recurrence then became an excuse to defer work despite safe tasks in other branches, docs, issues, operations, or product gaps. A second failure mode appeared when prompt edits, documentation audits, review requests, CI dispatches, Draft/Ready transitions, auto-merge enablement, or one successful commit/merge were treated as convenient endpoints even though safe work remained. ## Decision drivers @@ -14,30 +14,34 @@ Long model reviews, human approval, queued checks, provider cooldowns, and one b - Avoid polling and repetitive blocker narration. - Keep dependencies and writer safety without serializing the whole fleet. - Make documentation and operational proof first-class work. +- Prevent elapsed wall-clock time or meta/control-plane activity from becoming an implicit voluntary timeout. ## Alternatives considered 1. **Stop after one named request or blocker.** Rejected because it strands executable work. 2. **Poll until the active PR completes.** Rejected because it wastes budget and can starve the queue. -3. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, and require a double exit sweep.** Selected. +3. **Treat the hourly cadence as a soft wall-clock budget and resume later.** Rejected because recurrence is a continuation mechanism, not evidence that the current finite invocation is work-complete. +4. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, and require a double exit sweep.** Selected. ## Decision Each run maintains live lanes for mergeable PRs, defects/reviews/checks, protected-main acceptance, issues, documentation/automation drift, and one bounded buyer-visible gap. Pending external states enter a deferred set keyed by repository, PR, head, and run/review identity. After every substantive action the automation refreshes affected state and immediately selects another safe item. -A merge, dispatch, completed document, reviewer request, or external-governance wait never completes the invocation by itself. Before termination, a whole-queue sweep includes docs versus implementation. If it finds work, execute it and sweep again. Only real execution-budget exhaustion or a second fresh sweep proving every lane non-actionable permits termination. +The hourly recurrence is **continuation after genuine practical execution/tool-budget exhaustion**, never a voluntary soft timeout. A prompt update, documentation assessment/update, inventory, RCA without remediation, status or blocker comment, review request, workflow dispatch/rerun, queued/running CI or model review, Draft/Ready transition, auto-merge enablement, commit, merge, completed document, protected-main proof of one scenario, external-governance wait, or completion of one buyer-visible slice is an intermediate event with zero terminal credit while another safe lane exists. + +Before termination, perform a **fresh whole-queue sweep** that includes PRs, issues, protected-main acceptance, docs versus implementation, quality/security/release debt, active writers, and buyer/control-plane gaps. If it finds work, execute the highest-value safe item and sweep again. Only actual practical execution/tool-budget exhaustion or a **second fresh sweep** proving every lane non-actionable permits termination. ## Consequences -Positive: higher throughput, less queue starvation, and durable progress while providers wait. Negative: the automation needs careful scope/lease management and may leave several deferred items that require a shared ledger. +Positive: higher throughput, less queue starvation, fewer report-only runs, and durable progress while providers or humans wait. Negative: the automation needs careful scope/lease management, exact defer identities, and a continuation ledger capable of distinguishing locally blocked work from globally empty work. ## Failure and recovery -If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If budget expires mid-item, leave exact continuation evidence; do not claim product completion. +If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If a tool/run budget is genuinely exhausted mid-item, leave exact continuation evidence and resume on the next recurrence; do not claim product completion. If a meta action accidentally becomes the last action, the next run treats that as unfinished continuation rather than completed work. ## Security and governance impact -Work conservation never permits bypassing approval, checks, writer leases, or permissions. It changes scheduling, not authority. Routine output is suppressed so status narration cannot substitute for work. +Work conservation never permits bypassing approval, checks, writer leases, permissions, privacy controls, or fail-closed evidence boundaries. It changes scheduling, not authority. Routine output is suppressed so status narration cannot substitute for work. The no-soft-timeout rule does not authorize unbounded single-job execution; component/job timeouts remain bounded according to their own technical contracts. ## Tests and acceptance @@ -45,13 +49,17 @@ Work conservation never permits bypassing approval, checks, writer leases, or pe - duplicate dispatch is deferred, not polled; - branch-local lease blocks only one branch; - docs drift appears as executable debt; -- double exit sweep finds newly available work; and +- prompt/document/status/review/dispatch/Draft/Ready/auto-merge actions cannot satisfy the run exit condition by themselves; +- one successful commit or merge cannot satisfy the exit condition while another safe lane exists; +- elapsed time below a real platform/tool-budget boundary cannot substitute for an exit sweep; +- the first exit sweep finding work forces another action; +- only practical budget exhaustion or the second fresh all-lanes-non-actionable sweep permits termination; and - scheduled-run output is empty except defined notification events. ## Migration and rollback -Update maintainer prompts and scheduler policies with the continuation ledger and exit sweeps. Remove duplicative inactive loops. Rollback may reduce concurrency but must preserve no-report-as-completion and rotation from external waits. +Update maintainer prompts and scheduler policies with the continuation ledger and double exit sweeps. Remove duplicative inactive loops and prompt wording that encourages status-first termination. Rollback may reduce concurrency but must preserve no-report-as-completion, no-soft-timeout semantics, branch rotation, and the second-sweep termination proof. ## Supersession conditions -Supersede if a durable queue engine provides provably fair, dependency-aware, lease-safe work conservation and equivalent termination proofs across repositories. +Supersede if a durable queue engine provides provably fair, dependency-aware, lease-safe work conservation and equivalent termination proofs across repositories, including explicit handling of meta actions, external waits, and practical run-budget exhaustion. \ No newline at end of file From 96097dbdb8fc05f206aab0a3c31206cef7cd8a08 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:05:55 +0900 Subject: [PATCH 07/69] docs(erd): model remediation and continuation evidence --- docs/automation/ERD.md | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/docs/automation/ERD.md b/docs/automation/ERD.md index 878b25cba..01c7724fb 100644 --- a/docs/automation/ERD.md +++ b/docs/automation/ERD.md @@ -17,7 +17,40 @@ and future-persistence decision gate are normative in | Identity and runs | `organization_target`, `orchestration_run`, `repository_target`, `automation_run`, `pull_request_snapshot` | A fleet invocation contains repository-scoped child runs; many observations/runs may bind the same immutable source/base revision. | | Evidence | `check_evidence`, `status_evidence`, `review_evidence`, `model_evidence`, `workflow_evidence`, `dependency_evidence` | Issuer classes remain separate and every PR record binds an exact source revision. | | Decisions and dispatch | `ruleset_snapshot`, `dispatch_envelope`, `invocation_claim`, `scheduler_decision`, `writer_lease` | Dated live-state observations and idempotency/expected-head identity precede side effects. | -| Security and operation | `merge_revision`, `review_thread`, `security_finding`, `sandbox_evidence`, `sbom_snapshot`, `incident_hypothesis`, `operational_acceptance`, `handoff_record` | Operational acceptance attaches to the protected integrated revision, never directly to the PR source; bounded evidence/digests are retained. | +| RCA and remediation | `incident_hypothesis`, `remediation_candidate`, `scheduler_decision` | Symptoms, root-cause hypotheses, alternative remedies, feasibility evidence, selected action and rejection reasons remain distinguishable. | +| Continuation | `continuation_handoff`, `handoff_record`, `automation_run` | Practical run-budget exhaustion may hand exact deferred identities and next executable lanes to a later recurrence; prompt/docs/status activity is never completion evidence. | +| Documentation governance | `documentation_artifact`, `traceability_record` | Durable requirements/decisions/diagrams carry maturity state and map to exact implementation/tests/evidence; conversation/planning material remains candidate evidence until revalidated. | +| Security and operation | `merge_revision`, `review_thread`, `security_finding`, `sandbox_evidence`, `sbom_snapshot`, `operational_acceptance`, `secret_requirement` | Operational acceptance attaches to the protected integrated revision, never directly to the PR source; bounded evidence/digests are retained. | + +## Logical control-plane relationship view + +```mermaid +erDiagram + automation_run ||--o{ pull_request_snapshot : observes + pull_request_snapshot }o--|| source_revision : binds + pull_request_snapshot }o--|| pr_base_snapshot : records + pull_request_snapshot }o--|| live_base_revision : resolves + source_revision ||--o{ check_evidence : has + source_revision ||--o{ status_evidence : has + source_revision ||--o{ review_evidence : has + source_revision ||--o{ model_evidence : has + source_revision ||--o{ workflow_evidence : has + source_revision ||--o{ dependency_evidence : has + pull_request_snapshot ||--o{ incident_hypothesis : diagnoses + incident_hypothesis ||--o{ remediation_candidate : evaluates + remediation_candidate }o--o| scheduler_decision : selected_by + automation_run ||--o{ writer_lease : guards + automation_run ||--o{ continuation_handoff : emits + continuation_handoff ||--o{ handoff_record : carries + documentation_artifact ||--o{ traceability_record : contributes + traceability_record }o--o| source_revision : implementation + traceability_record }o--o| check_evidence : verification + source_revision |o--o| merge_revision : integrates_as + merge_revision ||--o{ operational_acceptance : accepted_by + automation_run ||--o{ secret_requirement : materializes +``` + +`model_evidence` is the current DATA_MODEL alias for model/reviewer judgment evidence. `pr_base_snapshot` and `live_base_revision` make the distinction between event/PR snapshot base identity and an independently resolved current protected-base tip explicit. These are conceptual names; implementations may serialize equivalent fields inside existing GitHub/Actions receipts without creating tables. ## Physical-model decision @@ -26,4 +59,4 @@ proposal to materialize these logical entities must first add an ADR covering tenancy, access control, purpose and retention, deletion, encryption, schema migration, reconciliation with GitHub, backup/restore, and rollback. Until then, the diagrams describe relationships that implementations and evidence receipts -must preserve, not tables that operators should provision. +must preserve, not tables that operators should provision. \ No newline at end of file From e6870d684943225d3138c58772074d8a0bda76ed Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:07:10 +0900 Subject: [PATCH 08/69] test(docs): enforce conversation and standards freshness --- .../test_automation_documentation_contract.py | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 0e97983f5..64d3be613 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -24,6 +24,7 @@ "INCIDENT_RUNBOOK.md", "RUNBOOK.md", "TRACEABILITY.md", + "DOCUMENTATION_AUDIT.md", ) REQUIRED_ADRS = ( @@ -88,6 +89,16 @@ "sbom_snapshot", ) +CONTROLLED_MATURITY_STATES = ( + "implemented_on_protected_main", + "active_pr", + "accepted_architecture", + "planned", + "research_only", + "superseded", + "out_of_scope", +) + def read_document(relative_path: str) -> str: """Return one automation document as UTF-8 text.""" @@ -181,6 +192,87 @@ def test_conceptual_erd_uses_explicit_two_word_snake_case_entities() -> None: assert "string source_kind" in data_model +def test_whole_conversation_audit_has_controlled_maturity_and_ownership() -> None: + """Conversation-derived claims stay status-bound and product semantics stay leaf-owned.""" + + audit = read_document("DOCUMENTATION_AUDIT.md") + index = read_document("README.md") + for state in CONTROLLED_MATURITY_STATES: + assert f"`{state}`" in audit + for leaf_product in ( + "TEPP", + "OriginWeave", + "EmbedRelay", + "MHTML ETL", + "LifeOS", + "BandScope", + "Inkspan", + "pg-erd-cloud", + "naruon", + "AppGuardrail", + ): + assert leaf_product in audit + assert "candidate evidence" in audit + assert "product-specific detail remains owned by the product repository" in audit + assert "DOCUMENTATION_AUDIT.md" in index + + +def test_work_conservation_forbids_meta_action_and_soft_timeout_completion() -> None: + """Meta/control events cannot replace execution or the double exit proof.""" + + adr = read_document("adr/0007-work-conserving-maintenance.md") + audit = read_document("DOCUMENTATION_AUDIT.md") + corpus = adr + "\n" + audit + for phrase in ( + "prompt update", + "documentation assessment", + "Draft/Ready", + "auto-merge", + "soft timeout", + "second fresh sweep", + "execution/tool-budget exhaustion", + ): + assert phrase.casefold() in corpus.casefold() + assert "intermediate event" in corpus + + +def test_remediation_continuation_and_documentation_entities_are_explicit() -> None: + """RCA alternatives, handoff, documentation and traceability stay first-class logical concepts.""" + + erd = read_document("ERD.md") + for entity_name in ( + "remediation_candidate", + "continuation_handoff", + "documentation_artifact", + "traceability_record", + ): + assert re.fullmatch(r"[a-z]+(?:_[a-z]+)+", entity_name) + assert entity_name in erd + assert "no physical database is implied" in erd + assert "must first add an ADR" in erd + + +def test_standards_baseline_keeps_final_and_draft_versions_distinct() -> None: + """Normative standards stay current and draft replacements cannot masquerade as final.""" + + standards = ( + REPOSITORY_ROOT / "docs" / "doctoring" / "automation-control-plane-standards.md" + ).read_text(encoding="utf-8") + for final_standard in ( + "SLSA version 1.2", + "NIST SP 800-218 version 1.1", + "ISO/IEC/IEEE 42010:2022", + "ISO/IEC/IEEE 29148:2018", + "ISO/IEC 25010:2023", + ): + assert final_standard in standards + assert "SLSA version 1.1" not in standards + assert "Edition 3 Draft International Standard" in standards + assert "informative only until published" in standards + assert "Initial Public Draft" in standards + assert "informative until final" in standards + + def test_exact_head_and_stable_interface_contracts_are_explicit() -> None: """Critical invalidation, dispatch, marker, and receipt shapes stay named.""" From 4b02edbc0e4836fb4e3798beac2f3bc173f1eb9f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:08:53 +0900 Subject: [PATCH 09/69] docs(data-model): model remediation and continuation traceability --- docs/automation/DATA_MODEL.md | 91 ++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 18 deletions(-) diff --git a/docs/automation/DATA_MODEL.md b/docs/automation/DATA_MODEL.md index 3bb3d7f0e..fcbda1a96 100644 --- a/docs/automation/DATA_MODEL.md +++ b/docs/automation/DATA_MODEL.md @@ -70,6 +70,8 @@ erDiagram `pr_snapshot_base_sha` and `live_base_commit_sha` are deliberately separate. The former explains what GitHub recorded with the PR snapshot; the latter is resolved from the current protected ref at decision time. +For explanatory UML/ERD views the two roles may also be labelled `pr_base_snapshot` and `live_base_revision`; those labels are aliases of the two separate fields carried by `base_revision`, not permission to collapse them into one SHA. + ## 3. Evidence ERD ```mermaid @@ -136,16 +138,27 @@ check source; a Commit Status remains `status_evidence`. Neither is silently converted into `review_evidence`, and normalized model output remains `model_evidence` even when a workflow uses it as one configured gate input. -## 4. Operations and governance ERD +`model_evidence` is the conceptual equivalent of a `model_judgment` when a +reviewer/model verdict is discussed in prose. The stored authority class remains +model evidence unless GitHub independently records a formal review through a +reviewer identity whose eligibility is validated separately. + +## 4. Operations, RCA, continuation, and documentation ERD ```mermaid erDiagram automation_run ||--o{ incident_hypothesis : investigates - automation_run ||--o{ handoff_record : defers + incident_hypothesis ||--o{ remediation_candidate : evaluates + remediation_candidate }o--o| scheduler_decision : selected_by + automation_run ||--o{ continuation_handoff : emits + continuation_handoff ||--o{ handoff_record : carries repository_target ||--o{ writer_lease : protects merge_revision ||--o{ operational_acceptance : validates automation_run ||--o{ secret_requirement : evaluates incident_hypothesis ||--o{ operational_acceptance : closes_with + documentation_artifact ||--o{ traceability_record : contributes + traceability_record }o--o| source_revision : implementation + traceability_record }o--o| check_evidence : verification incident_hypothesis { string incident_hypothesis_uid PK @@ -154,6 +167,20 @@ erDiagram string owner_boundary string hypothesis_state } + remediation_candidate { + string remediation_candidate_uid PK + string remedy_kind + string feasibility_state + string rejection_reason + bool selected_flag + } + continuation_handoff { + string continuation_handoff_uid PK + string continuation_state + string deferred_identity_digest + string next_action + datetime recorded_at + } handoff_record { string handoff_record_uid PK string blocked_action @@ -183,8 +210,35 @@ erDiagram datetime acquired_at datetime expires_at } + documentation_artifact { + string documentation_artifact_uid PK + string document_path UK + string maturity_state + string artifact_digest + datetime reviewed_at + } + traceability_record { + string traceability_record_uid PK + string requirement_id + string decision_id + string implementation_locator + string evidence_locator + string traceability_state + } ``` +A `remediation_candidate` is one materially distinct possible root-cause-changing +remedy and retains feasibility/rejection evidence. Merely naming a blocker is +not a candidate remedy. A `continuation_handoff` exists only when a finite run +must transfer exact deferred identities and next executable work because a real +practical run/tool budget ended; it is never generated merely because a prompt, +document, review request, check dispatch, commit, or merge completed. + +A `handoff_record` is the concrete external-prerequisite/continuation item carried +by the handoff. `documentation_artifact` and `traceability_record` make document +fitness and requirement/decision-to-implementation evidence first-class without +claiming a database. + ## 5. Governance and dispatch ERD ```mermaid @@ -304,14 +358,18 @@ be collapsed into one pass/fail row. | `check_evidence` | Check Run result with source kind and revision | GitHub Check Runs API | | `status_evidence` | Commit status context, creator, state, and revision | Commit Status API | | `review_evidence` | Formal review submission with author, state, and commit | Pull Request Review API/GraphQL | -| `model_evidence` | Normalized provider/model output with digest and revision | Bounded review/security artifact or job output | +| `model_evidence` | Normalized provider/model output with digest and revision; prose alias `model_judgment` | Bounded review/security artifact or job output | | `workflow_evidence` | Workflow/run/job/attempt and artifact provenance | Actions APIs and bounded artifacts | | `dependency_evidence` | Cross-PR, package, workflow, or release prerequisite | GitHub/package/attestation evidence | | `incident_hypothesis` | Falsifiable RCA statement and owner boundary | Incident notes, PR body, or run artifact | -| `handoff_record` | Precise external prerequisite plus autonomous continuation | Maintainer ledger, issue, or PR comment when needed | +| `remediation_candidate` | One materially distinct remedy with feasibility, rejection, and selection evidence | RCA/PR review notes or bounded maintainer result; not persisted as a table | +| `continuation_handoff` | Finite-run transfer of exact deferred identities and next executable lanes after genuine budget exhaustion | External automation continuation ledger or bounded handoff artifact | +| `handoff_record` | Precise external prerequisite plus autonomous continuation item carried by a handoff | Maintainer ledger, issue, or PR comment when needed | | `operational_acceptance` | Protected-main or real-consumer scenario proof | Workflow run, check, artifact receipt, and dated traceability row | | `secret_requirement` | Explicit secret-to-purpose-to-job contract | `workflow_call` declaration, job environment, and security docs | | `writer_lease` | Branch-scoped exclusive mutation intent | Live Project/issue assignment, branch/head observation, or automation ledger | +| `documentation_artifact` | Canonical document path, digest, review time, and controlled maturity state | Repository Markdown plus Git blob/commit identity | +| `traceability_record` | Requirement/decision to implementation, test/evidence, maturity and closure mapping | `TRACEABILITY.md`, ADR links, tests, workflow/check receipts | | `organization_target` | Organization scope and observation time for fleet governance | GitHub organization API identity | | `orchestration_run` | One fleet/hourly invocation grouping repository-scoped child runs | External continuation ledger or top-level Actions sweep receipt | | `ruleset_snapshot` | Dated ruleset parameters, targets, exclusions, and digest | GitHub ruleset API response plus audit artifact | @@ -337,20 +395,17 @@ be collapsed into one pass/fail row. `source_revision`; a source-branch run is insufficient. 5. A `secret_requirement` names one purpose and consumer job; wildcard purpose is invalid. 6. A writer lease covers one repository/ref/expected-head tuple, never the whole organization by implication. -7. Handoff records always name a continuation action unless every work lane is freshly proven non-actionable. -8. Credential values and unrestricted raw logs are never data-model attributes. -9. A ruleset snapshot always has an observation time and digest; stale snapshots - cannot authorize a current mutation. -10. One invocation idempotency key has at most one completed claim; retries link - rather than overwrite their predecessor. -11. Checks, statuses, reviews, model results, scheduler decisions, security - findings, threads, sandbox results, and SBOMs - retain their distinct issuer and authority class. -12. An organization-wide invocation is one `orchestration_run` with - repository-scoped `automation_run` children. Atomic evidence and mutations - remain bound to one `repository_target`; a fleet parent does not imply a - cross-repository writer lease. +7. A `remediation_candidate` records a materially distinct feasible/rejected remedy; stale evidence, invented authority, gate weakening, or another writer race cannot be selected as a valid remedy. +8. A `continuation_handoff` is valid only for genuine practical run/tool-budget exhaustion and must carry exact deferred identities plus next executable work. Prompt updates, documentation work, inventory, review requests, CI dispatch, Draft/Ready changes, auto-merge enablement, commits, merges, or one completed slice cannot create run-complete evidence. +9. Handoff records always name a continuation action unless every work lane is freshly proven non-actionable. +10. `documentation_artifact.maturity_state` maps to the controlled vocabulary in `DOCUMENTATION_AUDIT.md`; `active_pr` or `accepted_architecture` must never be emitted as `implemented_on_protected_main` without protected-main evidence. +11. A `traceability_record` preserves distinct requirement/decision, implementation, verification, and operational-closure locators; absence of one cannot be inferred from another authority class. +12. Credential values and unrestricted raw logs are never data-model attributes. +13. A ruleset snapshot always has an observation time and digest; stale snapshots cannot authorize a current mutation. +14. One invocation idempotency key has at most one completed claim; retries link rather than overwrite their predecessor. +15. Checks, statuses, reviews, model results, scheduler decisions, security findings, threads, sandbox results, and SBOMs retain their distinct issuer and authority class. +16. An organization-wide invocation is one `orchestration_run` with repository-scoped `automation_run` children. Atomic evidence and mutations remain bound to one `repository_target`; a fleet parent does not imply a cross-repository writer lease. ## 9. Future persistence decision gate -Materialization is justified only if GitHub-native evidence cannot meet query latency, cross-repository history, retention, or audit requirements. Before implementation, measure those gaps and decide tenancy, access control, purpose limitation, legal retention, encryption, deletion, schema migration, disaster recovery, and reconciliation with GitHub as source of truth. +Materialization is justified only if GitHub-native evidence cannot meet query latency, cross-repository history, retention, or audit requirements. Before implementation, measure those gaps and decide tenancy, access control, purpose limitation, legal retention, encryption, deletion, schema migration, disaster recovery, and reconciliation with GitHub as source of truth. \ No newline at end of file From 7cf7ea12f8f0a7797cfe3d5e2f57892df0c524a1 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:10:57 +0900 Subject: [PATCH 10/69] test(docs): bind new evidence entities into data model --- tests/test_automation_documentation_contract.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 64d3be613..dfd9e723f 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -73,10 +73,14 @@ "workflow_evidence", "dependency_evidence", "incident_hypothesis", + "remediation_candidate", + "continuation_handoff", "handoff_record", "operational_acceptance", "secret_requirement", "writer_lease", + "documentation_artifact", + "traceability_record", "organization_target", "orchestration_run", "ruleset_snapshot", @@ -240,6 +244,7 @@ def test_remediation_continuation_and_documentation_entities_are_explicit() -> N """RCA alternatives, handoff, documentation and traceability stay first-class logical concepts.""" erd = read_document("ERD.md") + data_model = read_document("DATA_MODEL.md") for entity_name in ( "remediation_candidate", "continuation_handoff", @@ -248,6 +253,7 @@ def test_remediation_continuation_and_documentation_entities_are_explicit() -> N ): assert re.fullmatch(r"[a-z]+(?:_[a-z]+)+", entity_name) assert entity_name in erd + assert entity_name in data_model assert "no physical database is implied" in erd assert "must first add an ADR" in erd From 9aca5fb08e1f35ca042c052eb995670c9df9abe8 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:12:18 +0900 Subject: [PATCH 11/69] docs(traceability): align maturity and live redaction lineage --- docs/automation/TRACEABILITY.md | 106 +++++++++++++++++++------------- 1 file changed, 64 insertions(+), 42 deletions(-) diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md index 0cd8e438f..8fea40ae8 100644 --- a/docs/automation/TRACEABILITY.md +++ b/docs/automation/TRACEABILITY.md @@ -3,63 +3,85 @@ Status: living authoritative matrix Last reviewed: 2026-08-09 -Status values: **implemented**, **partial**, **proposed**, or **operational-proof-pending**. A row marked implemented still requires fresh exact-head evidence for each change. +Canonical maturity states are defined in [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md): `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, and `out_of_scope`. Descriptive qualifiers may follow one of these states, but must not replace the canonical state. A row marked `implemented_on_protected_main` still requires fresh exact-head evidence for each later change; a row marked `active_pr` is never shipped truth. ## 1. Product and technical requirements -Receipt names describe the required evidence shape; they are not claims that a -particular transient run is current. A mutation must re-fetch the live receipt. +Receipt names describe the required evidence shape; they are not claims that a particular transient run is current. A mutation must re-fetch the live receipt. | Requirement | TRD / ADR | Exact implementation source | Exact tests | Required gate / receipt authority | Owner / closure target | Status | |---|---|---|---|---|---|---| -| `PRD-01` fresh live state | `TRD-EVT-01`, ADR-0007 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_agent_mention_sweep.py` | `scan-pr-queue` plus live PR/ruleset API snapshot | automation maintainer / every decision | implemented | -| `PRD-02` exact head + live base | `TRD-REV-01`, ADR-0002/0014 | `scripts/ci/pr_head_replay_guard.py`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_head_replay_guard.py`; `tests/test_opencode_review_context.py`; `tests/test_pr_review_fix_scheduler.py` | exact-head Check Runs and expected-head mutation response | workflow owners / before every write | partial: path inventory remains distributed | -| `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | implemented with live-ruleset dependency | -| `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; `.github/workflows/noema-review.yml` | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_noema_review_gate.py` | live ruleset requires two approvals, stale dismissal, last-pusher separation | repository admins / before merge | implemented; human capacity remains external | -| `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | partial: no shared persisted lease service | -| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; external hourly prompt contract | `tests/test_required_workflow_queue_contract.py`; external continuation-ledger receipt | 15-minute queue sweep plus two fresh whole-queue sweeps in the hourly run | automation operator / before run exit | partial: external orchestrator spans GitHub | -| `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py` | source-backed finding/RCA and protected recovery receipt | incident owner / before closure | partial: procedural and review control | -| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result marker | sandbox owners / `.github#842` integration + canary | operational-proof-pending | -| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/**`; root `ARCHITECTURE.md`; repository entrypoints | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link audit, `git diff --check`, reviewer approval | architecture maintainers / current documentation PR | proposed | -| `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/**`; `docs/automation/RUNBOOK.md` receipt template | workflow-specific contracts and real consumer canary | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | partial: receipts remain distributed | -| `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | required workflows; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; workflow contract suites | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | partial: implicit-v1 dispatch and inherited-secret legacy remain | -| `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | partial: new routes require audit | -| `PRD-13` PII alternative controls | `TRD-SEC-01`, `TRD-RET-01` | `docs/automation/SECURITY.md`; workflow access and artifact retention settings | privacy/access review; no complete automated gate | purpose/audience/retention/deletion/access receipt | data/service owner / before processing business PII | proposed/partial | -| `PRD-14` quality/readability | verification §12, ADR-0012 | `pyproject.toml`; pinned requirement/hash manifests; owned Python/workflow source | complete `tests/`; compile/syntax; coverage; interrogate | exact-head tests, 100% owned production statement/branch and public docstrings | change author / before PR readiness | implemented policy; fresh proof required | +| `PRD-01` fresh live state | `TRD-EVT-01`, ADR-0007 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_agent_mention_sweep.py` | `scan-pr-queue` plus live PR/ruleset API snapshot | automation maintainer / every decision | `implemented_on_protected_main` | +| `PRD-02` exact head + live base | `TRD-REV-01`, ADR-0002/0014 | `scripts/ci/pr_head_replay_guard.py`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_head_replay_guard.py`; `tests/test_opencode_review_context.py`; `tests/test_pr_review_fix_scheduler.py` | exact-head Check Runs plus independently resolved current base and expected-head mutation response | workflow owners / before every write | `accepted_architecture` — implementation exists but path inventory and current-live-base receipts remain distributed | +| `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | `implemented_on_protected_main` with live-ruleset dependency | +| `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; merge scheduler policy | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_scheduler_independent_approval_gate.py` | live review decision, qualifying non-author formal review and ruleset | repository admins / before merge | `active_pr` for the strengthened exact-head scheduler gate; existing ruleset remains live authority | +| `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml`; external maintainer lease policy | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | `accepted_architecture` — no shared persisted lease service claimed | +| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; external hourly prompt contract | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; external continuation receipt | 15-minute queue sweep plus double fresh whole-queue exit sweep | automation operator / before run exit | `accepted_architecture` — no-soft-timeout/meta-intermediate contract is in this active documentation PR | +| `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py`; documentation contracts | source-backed finding/RCA, `remediation_candidate` evidence, exact-head repair and protected recovery receipt | incident owner / before closure | `accepted_architecture` | +| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; focused redaction contracts | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / clean-history replacement #888 + protected consumer canary | `active_pr` | +| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/**`; root `ARCHITECTURE.md`; repository entrypoints | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | +| `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/**`; `docs/automation/RUNBOOK.md` receipt template | workflow-specific contracts and real consumer canary | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | `accepted_architecture` — receipts remain distributed | +| `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | required workflows; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; workflow contract suites | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | `accepted_architecture` — implicit-v1 dispatch and inherited-secret legacy remain | +| `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | `implemented_on_protected_main` for existing provider boundaries; new routes require audit | +| `PRD-13` PII alternative controls | `TRD-SEC-01`, `TRD-RET-01` | `docs/automation/SECURITY.md`; workflow access and artifact retention settings | privacy/access review; documentation/security contracts | purpose/audience/retention/deletion/access receipt | data/service owner / before processing business PII | `accepted_architecture` — service-specific deployment evidence remains external | +| `PRD-14` quality/readability | verification §12, ADR-0012 | `pyproject.toml`; pinned requirement/hash manifests; owned Python/workflow source | complete `tests/`; compile/syntax; coverage; interrogate | exact-head tests, 100% owned production statement/branch and public docstrings | change author / before PR readiness | `implemented_on_protected_main` as policy; every head requires fresh proof | ## 2. Documentation coverage | Artifact | Required content | Machine gate | Status | |---|---|---|---| -| PRD | users, two modes, outcomes, degraded behavior, acceptance/non-goals | file/index/term contract | added in documentation PR | -| TRD | events, revisions, evidence authority, permissions, retries, leases, redaction | file/index/source-name contract | added in documentation PR | -| Architecture | contexts, components, planes, trust/failure boundaries | Mermaid and source-name contract | added in documentation PR | -| Data model / ERD | conceptual vs persisted and all evidence/governance entities | entity/naming/cardinality/link contract | added in documentation PR | -| UML | component, two sequences, state, authority, topology, retry, sandbox | diagram-section/fence contract | added in documentation PR | -| Security / threat model | required attack paths, privacy alternative, residual risk | term/link contract | added in documentation PR | -| Test strategy | realistic gate, security, performance, consumer proof | term/link contract | added in documentation PR | -| Operability / runbook | SLI/SLO, queue/provider failures, exact queries, rollback, retention, receipt, closure/reopen | term/link contract | added in documentation PR | -| ADR set | sixteen indexed decisions with alternatives, tests, rollback, supersession | index/file/section contract | added in documentation PR | +| PRD | users, two modes, outcomes, degraded behavior, acceptance/non-goals | file/index/term contract | `active_pr` | +| TRD | events, revisions, evidence authority, permissions, retries, leases, redaction | file/index/source-name contract | `active_pr` | +| Architecture | contexts, components, planes, trust/failure boundaries | Mermaid and source-name contract | `active_pr` | +| Data model / ERD | conceptual vs persisted and evidence/governance/remediation/continuation/documentation entities | entity/naming/cardinality/link contract | `active_pr` | +| UML | component, two sequences, state, authority, topology, retry, sandbox, lease/continuation | diagram-section/fence contract | `active_pr` | +| Security / threat model | required attack paths, privacy alternative, residual risk | term/link contract | `active_pr` | +| Test strategy | realistic gate, security, performance, consumer proof, documentation fitness | term/link/standards contract | `active_pr` | +| Operability / runbook | SLI/SLO, queue/provider failures, exact queries, rollback, retention, receipt, closure/reopen | term/link contract | `active_pr` | +| Whole-conversation audit | maturity states, central/leaf ownership, no-soft-timeout, fitness findings | indexed file + documentation contract | `active_pr` | +| ADR set | sixteen indexed decisions with alternatives, tests, rollback, supersession | index/file/section contract | `active_pr` | +| Standards doctoring | final/draft version discipline and APA 7 references | standards-freshness contract | `active_pr` | ## 3. Redaction incident lineage -- `ContextualWisdomLab/.github#841` is the closed, unmerged Jules/Sentinel branch. It established the original disclosure concern but mixed an unrelated SSRF slice and omitted publication paths; it is historical evidence, not the integration vehicle. -- `ContextualWisdomLab/.github#842` is the clean current-main replacement for the sandbox evidence-redaction slice. It owns completed/timeout/exception output, service tails, command/result metadata, explicit allowed values, canonicalization, structural JSON handling, diagnostic preservation, and exhaustive hostile forms. -- Output-memory and service-file quotas remain a separate resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. -- Every head change on #842 invalidates predecessor test counts, coverage counts, reviews, and run IDs. The PR body and redaction doctoring must be updated from the final exact-head evidence before merge. +- `ContextualWisdomLab/.github#841` is closed/unmerged historical origin evidence. It established the disclosure concern but mixed unrelated scope; it is `superseded` as an integration path. +- `ContextualWisdomLab/.github#842` is closed/unmerged historical RED→GREEN and exhaustive-boundary evidence. Its final blobs removed current-source defects but its reachable PR history retained secret-shaped test fixtures that kept Secret Scan red; it is `superseded` as an integration path. +- `ContextualWisdomLab/.github#888` is the `active_pr` clean-history replacement from protected main. It carries the final ten #842 blobs without the secret-shaped predecessor history. Exact-head Sandbox Log Redaction, Secret Scan, Strix, Security Scan, SAST, OSV, Scorecard, SBOM and CodeQL evidence were successful at the latest observation; any queued gate remains non-passing until complete. +- Output-memory and service-file quotas remain a separate `planned` resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. +- Every #888 head change invalidates predecessor test counts, coverage counts, reviews, and run IDs. Protected-main/consumer synthetic credential canaries are required before operational closure. -## 4. Known gaps and next evidence +## 4. Whole-conversation and documentation governance traceability -| Gap | Risk | Next bounded evidence | -|---|---|---| -| No persisted cross-repository continuation/writer ledger | collision and queue-starvation reconstruction | decide whether GitHub Project/artifact state is sufficient before proposing persistence | -| SLI aggregation incomplete | buyer cannot quantify reliability | emit bounded run metrics keyed by repo/head/workflow/attempt | -| Sandbox total-output/service-file quota incomplete | memory/disk DoS | fail-first hostile-output quota tests and separate implementation PR | -| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | explicit secret interface plus real deployment positive/negative canary | -| Operational receipts distributed | incident closure hard to audit | dated protected-main/consumer receipt index | -| Project #1 requires GraphQL/`gh` project scope | agents without that capability cannot acquire visible project item | add supported connector or ensure native PR auto-add; never invent state | -| Master context current-state section is dated 2026-07-08 | stale operational narrative | separate timeless context from generated/daily live-state appendix | +| Decision / evidence source | Canonical artifact | Maturity | Rule | +|---|---|---|---| +| Repeated instruction to continue after merge/review/check/doc work | ADR-0007; `DOCUMENTATION_AUDIT.md` | `active_pr` | Meta/control actions and one substantive result are intermediate while a safe lane remains; hourly recurrence is not a soft timeout. | +| Conversation/planning packs across TEPP, OriginWeave, EmbedRelay, MHTML ETL, LifeOS and leaf products | `DOCUMENTATION_AUDIT.md`; central/leaf Architecture | `active_pr` | Revalidate shared automation decisions; leaf product semantics remain `out_of_scope` here and are not copied into central architecture. | +| ADR/PRD/TRD/UML/ERD completeness request | documentation graph + this traceability matrix | `active_pr` | A prose assessment alone is insufficient; gaps become canonical GitHub mutations and machine contracts. | +| Architecture/requirements/product-quality standards | `docs/doctoring/automation-control-plane-standards.md` | `active_pr` | Final publications are normative baselines; drafts are informative only. | +| Logical evidence-store relationships | `DATA_MODEL.md`; `ERD.md` | `active_pr` | Conceptual model only; material persistence requires a separate ADR and privacy/tenancy/DR design. | + +## 5. Known gaps and next evidence + +| Gap | Risk | Next bounded evidence | Maturity | +|---|---|---|---| +| No persisted cross-repository continuation/writer ledger | collision and queue-starvation reconstruction | decide whether GitHub Project/artifact state is sufficient before proposing persistence | `planned` | +| SLI aggregation incomplete | buyer cannot quantify reliability | emit bounded run metrics keyed by repo/head/workflow/attempt | `planned` | +| Sandbox total-output/service-file quota incomplete | memory/disk DoS | fail-first hostile-output quota tests and separate implementation PR | `planned` | +| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | explicit secret interface plus real deployment positive/negative canary | `planned` | +| Operational receipts distributed | incident closure hard to audit | dated protected-main/consumer receipt index | `planned` | +| Project #1 requires GraphQL/`gh` project scope | agents without that capability cannot acquire visible project item | add supported connector or ensure native PR auto-add; never invent state | `planned` | +| Master context current-state section is dated | stale operational narrative | separate timeless context from generated/daily live-state appendix | `planned` | +| Explicit dispatch/result schema versions not universal | compatibility ambiguity | version all central dispatch/result contracts with migration/negative tests | `planned` | + +## 6. Standards traceability + +Research status, normative version choices, applied implications, and APA 7 references are maintained in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). The current baseline explicitly distinguishes: -## 5. Standards traceability +- SLSA 1.2 as the current SLSA specification; +- NIST SP 800-218 SSDF 1.1 as current final versus the SSDF 1.2 / SP 800-218 Rev. 1 initial public draft as informative; +- ISO/IEC/IEEE 42010:2022 for architecture description; +- ISO/IEC/IEEE 29148:2018 as current published requirements engineering versus Edition 3 DIS as informative until publication; +- ISO/IEC 25010:2023 for product quality; and +- the named final testing/security/AI/observability/assurance sources in the doctoring record. -Research status, normative version choices, and APA 7 references are maintained in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). In particular, NIST SSDF 1.1 remains the final normative baseline while SSDF 1.2 is an initial public draft; SLSA 1.2 is the current SLSA specification at this review date. +No citation alone establishes certification, formal conformance, a SLSA level, SOC 2, CSAP, or operating effectiveness. \ No newline at end of file From 7f4e6d5516c8a7bb4696ea482976f62b6ec8b600 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:16:11 +0900 Subject: [PATCH 12/69] docs(uml): add writer rotation and documentation continuation flows --- docs/automation/UML.md | 60 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/docs/automation/UML.md b/docs/automation/UML.md index 1d52655d5..177f42c35 100644 --- a/docs/automation/UML.md +++ b/docs/automation/UML.md @@ -5,6 +5,8 @@ Last reviewed: 2026-08-09 These diagrams are normative at the boundary level. Detailed step names remain in workflow source and are checked through [TRACEABILITY.md](TRACEABILITY.md). +An editable FigJam companion board is available at . It is **supplemental visualization**, not an authority source: Mermaid and the Git-tracked contracts in this directory remain canonical. The board currently visualizes the central control plane, PR-governance sequence, evidence state machine, and conceptual evidence entities; it must be refreshed or labelled historical when those canonical diagrams change materially. + ## 1. Bounded-context component view ```mermaid @@ -213,3 +215,61 @@ stateDiagram-v2 heads remain reviewable, but central automation neither direct-merges nor queues auto-merge for them; it records the maintainer prerequisite and continues other work. + +## 11. Writer lease and branch rotation + +```mermaid +flowchart TB + Select["Select highest-value executable lane"] + Refetch["Re-fetch target head, base, blob/ref, writer state"] + Lease{"Branch writer available and identity unchanged?"} + Write["Acquire branch-scoped writer_lease and mutate"] + Verify["Run exact-head tests and evidence"] + Defer["Freeze only this exact branch/head/action"] + Rotate["Rotate to another non-conflicting lane"] + Sweep["Fresh whole-queue sweep"] + + Select --> Refetch + Refetch --> Lease + Lease -->|yes| Write + Lease -->|no| Defer + Write --> Verify + Verify --> Rotate + Defer --> Rotate + Rotate --> Select + Select -->|no immediate lane| Sweep + Sweep -->|work exists| Select + Sweep -->|none; first sweep only| Sweep +``` + +The lease is branch-scoped, not an implicit repository- or organization-wide mutex. A pending review/check, external approval, or another branch writer blocks only that exact lane. A second fresh all-lanes-nonactionable sweep—or genuine practical run/tool-budget exhaustion—is required before termination. + +## 12. Documentation assessment to repository mutation and continuation + +```mermaid +flowchart LR + Evidence["Conversation, planning, PR, incident, protected-main evidence"] + Revalidate["Revalidate ownership and live implementation"] + Fitness{"Canonical artifact fitness"} + Docs["Repair PRD/TRD/Architecture/UML/ERD/ADR/traceability"] + Tests["Add documentation fitness tests"] + SourceGap{"Implementation gap discovered?"} + Source["Create or repair source/test/workflow/issue slice"] + Review["Exact-head checks and review"] + Continue["Return to live executable queue"] + Leaf["Hand leaf product semantics to owning repository"] + + Evidence --> Revalidate + Revalidate --> Fitness + Fitness -->|stale, partial, missing| Docs + Fitness -->|leaf-owned| Leaf + Docs --> Tests + Tests --> SourceGap + SourceGap -->|yes| Source + SourceGap -->|no| Review + Source --> Review + Review --> Continue + Leaf --> Continue +``` + +A documentation assessment or update is never completion by itself. Candidate conversation/planning evidence becomes canonical only after revalidation, maturity classification, Git-tracked mutation, and machine/reviewer checks. Product-specific architecture remains with its leaf repository; central documentation owns only shared automation and interface contracts. \ No newline at end of file From 5e7d703f20443bc5ab5347317a692c102c4fa007 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:21:48 +0900 Subject: [PATCH 13/69] docs(audit): close repaired documentation-family gaps --- docs/automation/DOCUMENTATION_AUDIT.md | 28 +++++++++++++++----------- 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md index 857491e4a..09588a971 100644 --- a/docs/automation/DOCUMENTATION_AUDIT.md +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -46,19 +46,21 @@ Every durable claim in this documentation graph MUST map to one of these states: ## Documentation family fitness -| Artifact family | Fitness on this PR | Required residual action | +This table assesses the **active PR documentation baseline**, not protected-main runtime implementation. An `ADEQUATE` documentation family may still describe `accepted_architecture`, `planned`, or `active_pr` behavior whose runtime evidence is incomplete. + +| Artifact family | Fitness on this PR | Ongoing control | |---|---|---| | PRD | ADEQUATE | Keep the two operating modes, degraded behavior and measurable acceptance aligned with live implementation. | | TRD | ADEQUATE | Preserve explicit event, identity, authority, secret, retry, timeout, lease, result and compatibility contracts. | | Architecture | ADEQUATE | Keep central/leaf bounded contexts, data/control planes, trust and failure domains current. | -| UML | ADEQUATE | Machine-check all required component, sequence, state, authority, retry, deployment, lease and continuation flows. | -| ERD / data model | PARTIAL | Add first-class aliases/entities for remediation alternatives, continuation handoff, documentation artifacts and traceability records; remain conceptual until a persistence ADR is accepted. | +| UML | ADEQUATE | Machine-check component, PR-maintenance/product-development sequences, evidence state, authority, deployment, retry, mention/sandbox, writer rotation and documentation-continuation flows. The FigJam companion remains supplemental only. | +| ERD / data model | ADEQUATE | `remediation_candidate`, `continuation_handoff`, `documentation_artifact`, `traceability_record`, PR-base/live-base identity and evidence-authority relationships are explicit; the model remains conceptual until a persistence ADR is accepted. | | Security / threat model | ADEQUATE | Keep credential, confused-deputy, untrusted-source, prompt-injection, stale-evidence, PII and supply-chain cases current. | -| Test strategy | ADEQUATE | Add documentation-fitness regression coverage for this audit, maturity mapping and standards freshness. | +| Test strategy | ADEQUATE | Documentation-fitness regression coverage now binds this audit, maturity mapping, logical entities and standards freshness; semantic review remains required. | | Operability / incident runbook | ADEQUATE | Preserve protected-main/consumer closure and queue/provider/runner/DNS recovery evidence. | -| ADR set | ADEQUATE | Extend work-conservation wording with the explicit intermediate-event/no-soft-timeout invariant. | -| Traceability | PARTIAL | Add explicit documentation-artifact and continuation/remediation mappings and current requirements/product-quality standards. | -| Standards doctoring | PARTIAL | Keep SLSA 1.2 and final-vs-draft SSDF handling; add ISO/IEC/IEEE 29148:2018 and ISO/IEC 25010:2023. | +| ADR set | ADEQUATE | ADR-0007 now explicitly classifies prompt/docs/status/review/dispatch/Draft/Ready/auto-merge/commit/merge as intermediate and forbids a voluntary soft timeout. | +| Traceability | ADEQUATE | Controlled maturity, current redaction lineage, whole-conversation governance, standards, implementation/test/gate ownership and explicit product/control-plane debt are mapped. | +| Standards doctoring | ADEQUATE | Current final baselines include SLSA 1.2, ISO/IEC/IEEE 42010:2022, ISO/IEC/IEEE 29148:2018, ISO/IEC 25010:2023 and NIST SSDF 1.1; newer drafts are labelled informative/non-normative until final. | ## No-soft-timeout continuation invariant @@ -86,10 +88,12 @@ flowchart LR ## Exit condition for this documentation repair -This audit is sufficient only when: +The documentation family is review-ready only when: -1. it is indexed by `docs/automation/README.md`; -2. machine tests validate the controlled maturity vocabulary, whole-conversation ownership boundary, double-exit/no-soft-timeout rule, conceptual-vs-persisted boundary, standards freshness and local links; -3. the residual ERD/traceability/standards gaps above are repaired on the same canonical documentation line; +1. `docs/automation/README.md` indexes the complete canonical graph and whole-conversation audit; +2. machine tests validate controlled maturity, whole-conversation ownership, double-exit/no-soft-timeout behavior, conceptual-vs-persisted boundaries, remediation/continuation/documentation/traceability entities, standards freshness, local links and Mermaid/code fences; +3. PRD/TRD/Architecture/UML/ERD/Data Model/Security/Test/Operability/ADR/Traceability/standards doctoring have no known family-level `PARTIAL` or `MISSING` gap on the same exact head; 4. exact-head documentation/security/supply-chain checks and current review are clean; and -5. after protected merge, the canonical index is read from protected `main` before any `implemented_on_protected_main` claim is made. +5. after protected merge, the canonical index is refetched from protected `main` before any `implemented_on_protected_main` claim is made. + +Residual **product/control-plane implementation debt** remains tracked in `TRACEABILITY.md` and the PR body. Documentation sufficiency does not convert that planned or active implementation work into shipped behavior. \ No newline at end of file From 3edd408ae5c7cab24aedd429423cdb6025bbd067 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:25:32 +0900 Subject: [PATCH 14/69] test(docs): make ownership contract semantic and bind continuation UML --- tests/test_automation_documentation_contract.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index dfd9e723f..8a06a4b10 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -216,8 +216,9 @@ def test_whole_conversation_audit_has_controlled_maturity_and_ownership() -> Non "AppGuardrail", ): assert leaf_product in audit - assert "candidate evidence" in audit - assert "product-specific detail remains owned by the product repository" in audit + audit_normalized = " ".join(audit.casefold().split()) + assert "candidate evidence" in audit_normalized + assert "product-specific detail remains owned by the product repository" in audit_normalized assert "DOCUMENTATION_AUDIT.md" in index @@ -226,6 +227,7 @@ def test_work_conservation_forbids_meta_action_and_soft_timeout_completion() -> adr = read_document("adr/0007-work-conserving-maintenance.md") audit = read_document("DOCUMENTATION_AUDIT.md") + uml = read_document("UML.md") corpus = adr + "\n" + audit for phrase in ( "prompt update", @@ -238,6 +240,9 @@ def test_work_conservation_forbids_meta_action_and_soft_timeout_completion() -> ): assert phrase.casefold() in corpus.casefold() assert "intermediate event" in corpus + assert "## 11. Writer lease and branch rotation" in uml + assert "## 12. Documentation assessment to repository mutation and continuation" in uml + assert "supplemental visualization" in uml def test_remediation_continuation_and_documentation_entities_are_explicit() -> None: From e7771a0c1d1df74f2f5fce242931db0bd65d6190 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:36:31 +0900 Subject: [PATCH 15/69] docs(automation): version central event and result contracts --- docs/automation/EVENT_CONTRACTS.md | 192 +++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 docs/automation/EVENT_CONTRACTS.md diff --git a/docs/automation/EVENT_CONTRACTS.md b/docs/automation/EVENT_CONTRACTS.md new file mode 100644 index 000000000..df137ba07 --- /dev/null +++ b/docs/automation/EVENT_CONTRACTS.md @@ -0,0 +1,192 @@ +# Event, payload, and result contracts + +Status: active_pr documentation baseline +Last reviewed: 2026-08-09 +Scope: central automation envelopes crossing workflow, repository, sandbox, and scheduler boundaries + +## 1. Purpose + +This document defines versioning, identity, strict parsing, replay, compatibility, and migration rules for central control-plane messages. It does **not** make an unversioned protected-main payload versioned by documentation alone. A runtime schema is `implemented_on_protected_main` only after the producing and consuming code, negative compatibility tests, review, protected merge, and required operational acceptance exist. + +GitHub event envelopes remain platform-owned. The schemas below cover the CWL-owned fields placed in GitHub workflow inputs, `repository_dispatch.client_payload`, bounded result JSON, and idempotency receipts. + +## 2. Global envelope invariants + +Every mutation-capable or review-routing message MUST satisfy all applicable invariants before a credential with broader authority than metadata read is materialized: + +1. `target_repository` is an exact allowlisted `ContextualWisdomLab/` identity, never a prefix/glob chosen by model output. +2. `pull_request_number` is a positive integer and is re-read from live GitHub state. +3. `source_head_sha` is an exact 40-hex source revision and equals the live open PR head at the consumer boundary. +4. `pr_base_snapshot_sha` records the base SHA supplied/observed with the PR/event for provenance only. +5. `live_base_tip_sha` is independently resolved from the current target base ref when the decision depends on the current protected base. +6. `base_branch` and source repository/ref are revalidated rather than inferred from an earlier event. +7. actor/sender/installation identity is authenticated through the event/provider boundary and compared with the configured policy. +8. unknown keys, duplicate JSON keys, wrong types, ambiguous null/empty values, unsupported schema versions, oversized payloads, and identity mismatch fail closed. +9. CWL payloads never carry credential values, raw model prompts, unrestricted logs, or business payloads merely for convenience. +10. idempotency identity binds every field whose change could alter authority or side effects. + +A CWL envelope cannot promote a check, status, model result, or comment into a formal human review. Authority class is preserved outside the payload schema. + +## 3. Version lifecycle + +| State | Meaning | Consumer behavior | +|---|---|---| +| `legacy_implicit_v1` | Protected-main path has no explicit schema field but has a bounded parser/contract. | Continue only where required for compatibility; do not extend with new optional authority fields. | +| `active_pr` | Explicit schema exists only on an open PR. | Never advertise as protected-main support; predecessor consumers reject it unless the PR intentionally contains a dual-parser migration. | +| `implemented_on_protected_main` | Producer and consumer are merged, machine/review gates passed, and protected-source identity is observable. | Accept exact documented version; retain rollback/migration path as specified. | +| `superseded` | A newer version replaced this version after a reviewed migration window. | Reject new production use; retain only bounded historical decoding if incident/replay evidence requires it. | + +A major schema change is any change to required fields, field meaning, authority, idempotency identity, rejection semantics, maximum sizes, or side effects. Major changes require a new schema identifier and ADR/traceability update. Additive fields are not automatically backward-compatible: the receiving strict parser decides whether they are allowed. + +## 4. Review-agent invocation envelope + +### 4.1 Version 2 — `cwl.agent-invocation/v2` + +Maturity: `active_pr` while the complete envelope work remains unmerged. + +The intended canonical CWL payload has exactly: + +```json +{ + "schema": "cwl.agent-invocation/v2", + "claim": { + "repository": "ContextualWisdomLab/example", + "pr_number": 17, + "head_sha": "<40 hex>", + "base_sha": "<40 hex PR snapshot base>", + "base_branch": "main", + "agent": "opencode-agent", + "comment_id": 123456, + "actor": "trusted-maintainer", + "trigger_reviews": true, + "review_dispatch_limit": "1", + "enable_auto_merge": false, + "update_branches": false, + "merge_mode": "disabled" + }, + "agent_invocation_key": "" +} +``` + +For Noema, policy-specific fields not used by that transport remain governed by its own strict wrapper contract; a consumer may not reinterpret an OpenCode policy field as Noema authority. + +The outer `client_payload` must stay within GitHub's repository-dispatch constraints and the consumer enforces a stricter exact-key/type/cardinality contract before forwarding. The SHA-256 invocation key is an idempotency/fencing identity, not a credential or approval. + +### 4.2 Legacy mention path + +Maturity: `implemented_on_protected_main` for the currently protected router/wrapper shapes that predate complete v2 dispatch binding. + +Legacy schema-free fields remain compatibility evidence only. They must not silently accept an object claiming `cwl.agent-invocation/v2`, nor may new authority-bearing fields be added to the legacy path. Migration closes only when the protected producer and every authoritative downstream consumer use the explicit version or the legacy path is intentionally retained with a separately documented reason. + +## 5. Merge-scheduler review-only dispatch + +### 5.1 `merge-scheduler-agent-review-v2` + +Maturity: `active_pr` when supplied through the versioned review-agent envelope. + +Normative properties: + +- review-only: no branch update, auto-merge, direct merge, release, or deployment; +- exact source repository/PR/head/base/base-branch identity bound to the upstream invocation claim; +- two fresh live PR snapshot checks before the mutation-free review decision; +- a different-head active run may suppress duplicate/stale dispatch but cannot validate old evidence; +- receiver concurrency isolation may serialize/idempotently suppress the same invocation but may not cancel a newer valid exact-head request because an older event arrived late. + +### 5.2 Legacy merge-scheduler dispatch + +Maturity: `legacy_implicit_v1`. + +Schema-free scheduler requests remain an explicit compatibility path. They cannot accept the v2 canonical claim by duck typing. Any migration that removes the legacy path must prove every current producer moved first and must retain rollback for at least one protected release interval unless an incident requires immediate fail-closed removal. + +## 6. Merge scheduler result receipt + +### `pr-review-merge-scheduler/v2` + +Maturity: `implemented_on_protected_main` where the current scheduler emits this versioned bounded result. + +The receipt records deterministic scheduler observation/decision evidence. At minimum consumers bind: + +- schema/version; +- target repository and PR; +- exact expected source head; +- observed base/merge state relevant to the decision; +- decision/action and finite reason code; +- action error separately from ordinary wait/block/defer policy states; and +- attempt/run identity where needed to distinguish retries. + +A result receipt is not formal review evidence and does not prove a mutation succeeded unless the GitHub mutation response and subsequent live state confirm it. A future change to terminal workflow policy for `action_error` must version or explicitly preserve the receipt semantics rather than overload `success`. + +## 7. Sandbox result envelopes + +### 7.1 `SANDBOXED_VERIFY_RESULT` + +Maturity: `implemented_on_protected_main`; redaction improvements may be `active_pr` independently. + +The bounded JSON result preserves stable fields including `exit_code`, `elapsed_seconds`, `allowed_env`, and `evidence_note` as defined in [TRD.md](TRD.md). Child process stdout/stderr is diagnostic input, not schema authority. A parse/setup failure uses the documented stable exit/result failure semantics and never publishes unsafely parsed raw evidence. + +### 7.2 `SANDBOXED_WEB_E2E_RESULT` + +Maturity: `implemented_on_protected_main`; redaction/resource-bound improvements may be `active_pr` independently. + +The envelope retains stable stage/exit/timing/evidence metadata while backend/frontend logs are separately bounded and redacted. Result JSON must remain valid after redaction; credentials or arbitrary source text are not added as structured fields. + +Breaking field/type/exit-code changes require an explicit new result schema/version and migration tests. + +## 8. Reusable-workflow contract versioning + +Reusable workflows are APIs even when their YAML path is stable. + +- Required inputs and secrets form the public interface. +- A caller pinned to an immutable commit receives exactly that reviewed contract. +- Removing/renaming an input or secret, broadening `secrets: inherit`, changing mutation authority, or changing trigger/source identity is a breaking interface change. +- New credentials require explicit purpose and job scope before they are callable. +- A reusable workflow's internal job name may be externally significant when branch/ruleset/check policy relies on it; such names are tracked in TRD/Traceability rather than renamed casually. + +The planned `deploy-pages.yml` explicit Cloudflare-secret migration is a contract hardening of an older implicit inheritance interface. It is tracked separately and must be implemented test-first; this document does not make that planned runtime change effective. + +## 9. Size, strictness, and GitHub platform bounds + +CWL producers MUST validate their stricter schema and bounded string/list sizes before invoking GitHub. They MUST also remain below the platform's repository-dispatch top-level-property and payload-size limits documented by GitHub. A message that would exceed either bound is rejected before dispatch; truncating identity/security fields is forbidden. + +Where a payload includes model-generated or source-derived display evidence, the system passes an identifier/digest or a separately bounded artifact rather than expanding the authority envelope. + +## 10. Replay and idempotency + +- Redelivery of an identical GitHub event may be processed multiple times at the transport layer but must not create duplicate mutation/review authority. +- `invocation_claim`/artifact-ledger identity is exact and stateful; a completed exact request does not forward twice. +- Failed-before-forward recoverability is a separate reliability contract and must not be achieved by weakening the completed-request idempotency key. +- Head/base/actor/comment/policy movement produces a distinct claim or fails live comparison; it never reuses the old completed claim as current authority. +- Rerunning a workflow creates a new run/attempt identity while the exact source/review claim may remain the same. Consumers distinguish retry transport identity from business/idempotency identity. + +## 11. Compatibility and rollback matrix + +| Change | Migration rule | Rollback rule | +|---|---|---| +| Add explicit version to legacy payload | Dual parser only if producers/consumers cannot move atomically; versioned object must never be accepted as legacy. | Revert producer first or keep dual consumer until all callers are restored. | +| Remove legacy payload | Prove zero live callers through code search + protected consumer evidence. | Restore immutable prior consumer/source and rerun negative/positive canaries. | +| Change required field/type | New schema identifier. | Keep prior parser only for the documented compatibility window. | +| Change idempotency key inputs | New schema/claim version and collision/replay tests. | Do not reinterpret old artifacts under the new key definition. | +| Change reviewer/mutation authority | ADR + security/threat/ruleset review; not a schema-only edit. | Restore previous authority contract; never synthesize missing approvals. | +| Change reusable secret contract | Explicit caller inventory and migration; no blanket inheritance fallback. | Revert callers and called workflow to the same immutable compatible pair. | + +## 12. Negative contract tests + +Every versioned message family needs tests for: + +- missing/extra/duplicate keys; +- wrong scalar/container types and null/empty ambiguity; +- unsupported/newer/older schema values; +- malformed repository/ref/SHA/PR/actor identifiers; +- source-head and live-base movement; +- stale, replayed, duplicate, and out-of-order events; +- payload size/cardinality limits; +- cross-repository confused deputy attempts; +- untrusted model/source text pretending to be schema control fields; +- credential-shaped content rejection/redaction at publication boundaries; and +- legacy/versioned parser non-conflation. + +## 13. Traceability and supersession + +Runtime schemas are linked from [TRACEABILITY.md](TRACEABILITY.md) to their producers, consumers, tests, maturity state, and protected acceptance receipts. If code and this document disagree, current live source is the observation authority and the mismatch is a documentation or implementation defect—not permission to guess which behavior was intended. + +A schema is superseded only after producers, consumers, negative compatibility tests, operator runbook, rollback, and protected-main/consumer acceptance have moved to the successor. Historical payload examples remain non-authorizing evidence. \ No newline at end of file From 9b63d2a9eaad66aa366313a5361a86111ec5b2ef Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:37:14 +0900 Subject: [PATCH 16/69] docs(automation): index versioned event contracts --- docs/automation/README.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/docs/automation/README.md b/docs/automation/README.md index 51a743b78..f086ef9f9 100644 --- a/docs/automation/README.md +++ b/docs/automation/README.md @@ -10,18 +10,19 @@ This directory is the durable specification for the ContextualWisdomLab automati 1. [PRD.md](PRD.md) — operator and buyer outcomes, scope, and acceptance. 2. [TRD.md](TRD.md) — normative event, evidence, identity, retry, secret, and compatibility contracts. -3. [ARCHITECTURE.md](ARCHITECTURE.md) — viewpoints, bounded contexts, trust boundaries, and failure domains. -4. [DATA_MODEL.md](DATA_MODEL.md) and [ERD.md](ERD.md) — conceptual evidence +3. [EVENT_CONTRACTS.md](EVENT_CONTRACTS.md) — versioned dispatch, reusable-workflow, scheduler-result, sandbox-result, replay, migration, and rollback contracts; legacy runtime paths remain explicitly classified rather than upgraded by prose. +4. [ARCHITECTURE.md](ARCHITECTURE.md) — viewpoints, bounded contexts, trust boundaries, and failure domains. +5. [DATA_MODEL.md](DATA_MODEL.md) and [ERD.md](ERD.md) — conceptual evidence model and exact-path logical ERD; neither claims a deployed database. -5. [UML.md](UML.md) — component, sequence, state, authority, deployment, and incident flows. -6. [SECURITY.md](SECURITY.md) and [THREAT_MODEL.md](THREAT_MODEL.md) — security objectives, controls, abuse cases, and residual risk. -7. [TEST_STRATEGY.md](TEST_STRATEGY.md) — deterministic, security, compatibility, documentation, and protected-main acceptance gates. -8. [OPERABILITY.md](OPERABILITY.md), [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md), +6. [UML.md](UML.md) — component, sequence, state, authority, deployment, and incident flows. +7. [SECURITY.md](SECURITY.md) and [THREAT_MODEL.md](THREAT_MODEL.md) — security objectives, controls, abuse cases, and residual risk. +8. [TEST_STRATEGY.md](TEST_STRATEGY.md) — deterministic, security, compatibility, documentation, and protected-main acceptance gates. +9. [OPERABILITY.md](OPERABILITY.md), [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md), and executable [RUNBOOK.md](RUNBOOK.md) — service objectives, observability, diagnosis, ownership, recovery, retention, rollback, and acceptance receipts. -9. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. -10. [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md) — whole-conversation fitness, central-versus-leaf ownership, maturity states, and no-soft-timeout continuation contract. -11. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. +10. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. +11. [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md) — whole-conversation fitness, central-versus-leaf ownership, maturity states, and no-soft-timeout continuation contract. +12. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. ## Maturity and authority discipline From 3009af20e86ae5ba0d582b54ae5af5efe08fc962 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:38:03 +0900 Subject: [PATCH 17/69] test(docs): require versioned event contract baseline --- .../test_automation_documentation_contract.py | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 8a06a4b10..158c140fe 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -13,6 +13,7 @@ "README.md", "PRD.md", "TRD.md", + "EVENT_CONTRACTS.md", "ARCHITECTURE.md", "DATA_MODEL.md", "ERD.md", @@ -284,6 +285,31 @@ def test_standards_baseline_keeps_final_and_draft_versions_distinct() -> None: assert "informative until final" in standards +def test_event_contracts_version_legacy_and_explicit_interfaces() -> None: + """Versioned messages remain distinct from explicit legacy compatibility paths.""" + + contracts = read_document("EVENT_CONTRACTS.md") + index = read_document("README.md") + for schema_name in ( + "cwl.agent-invocation/v2", + "merge-scheduler-agent-review-v2", + "pr-review-merge-scheduler/v2", + "SANDBOXED_VERIFY_RESULT", + "SANDBOXED_WEB_E2E_RESULT", + ): + assert schema_name in contracts + for identity_field in ( + "source_head_sha", + "pr_base_snapshot_sha", + "live_base_tip_sha", + ): + assert identity_field in contracts + assert "legacy_implicit_v1" in contracts + assert "versioned object must never be accepted as legacy" in contracts + assert "no blanket inheritance fallback" in contracts + assert "EVENT_CONTRACTS.md" in index + + def test_exact_head_and_stable_interface_contracts_are_explicit() -> None: """Critical invalidation, dispatch, marker, and receipt shapes stay named.""" @@ -336,6 +362,7 @@ def test_security_and_operations_contracts_keep_authorities_separate() -> None: for relative_path in ( "PRD.md", "TRD.md", + "EVENT_CONTRACTS.md", "SECURITY.md", "THREAT_MODEL.md", "OPERABILITY.md", @@ -365,6 +392,7 @@ def test_timeless_architecture_does_not_embed_transient_run_identity() -> None: for relative_path in ( "PRD.md", "TRD.md", + "EVENT_CONTRACTS.md", "ARCHITECTURE.md", "DATA_MODEL.md", "UML.md", From c2fa86d5e131bd1be09dfaa9251baf9bed561ee8 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:39:00 +0900 Subject: [PATCH 18/69] docs(threats): model premature termination and split authority --- docs/automation/AUTONOMY_THREATS.md | 118 ++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/automation/AUTONOMY_THREATS.md diff --git a/docs/automation/AUTONOMY_THREATS.md b/docs/automation/AUTONOMY_THREATS.md new file mode 100644 index 000000000..70c4680f9 --- /dev/null +++ b/docs/automation/AUTONOMY_THREATS.md @@ -0,0 +1,118 @@ +# Autonomous-maintenance threat addendum + +Status: active_pr threat-model companion +Last reviewed: 2026-08-09 +Scope: availability, integrity, and governance threats created by long-running autonomous maintenance itself + +This companion is part of the automation threat-model family. The general trust boundaries remain in [THREAT_MODEL.md](THREAT_MODEL.md); this file makes two whole-conversation threats explicit because they are easy to miss in a conventional credential/source threat model: **premature termination** and **split-brain authority**. + +## 1. AT-01 — premature termination / false quiescence + +### Threat + +An automation run observes one blocked PR, queued review, long model call, completed merge, documentation update, prompt change, or apparent wall-clock budget and incorrectly concludes that the control plane has no more executable work. Safe work in another branch, issue, documentation lane, protected-main acceptance lane, or buyer/control-plane gap is starved until the next recurrence. + +This is simultaneously: + +- an availability threat because repair/merge/product queues idle; +- an integrity threat because `done` or an implicit terminal state misrepresents the live queue; and +- an accountability threat because the lack of a fresh exit proof makes later reconstruction ambiguous. + +### Attacker/failure stories + +1. A provider or reviewer delays one PR; the maintainer keeps polling it and never repairs another repository-owned defect. +2. A prompt update, documentation audit, review request, CI dispatch, Draft/Ready transition, auto-merge enablement, commit, merge, or single buyer slice is treated as terminal progress even though another safe lane exists. +3. A long but still valid invocation crosses an informal elapsed-time threshold and exits without checking the rest of the queue. +4. A malicious/noisy PR consumes attention through repeated changing status while unrelated high-leverage work remains executable. + +### Required controls + +- exact-identity deferred set for external/pending waits; +- branch-local writer lease rather than global serialization; +- live executable queue ordered by risk/leverage; +- no-report-as-completion and no-soft-timeout invariant from ADR-0007; +- meta/control events carry zero terminal credit; +- fresh whole-queue exit sweep covering PRs, issues, docs, tests, security, protected-main acceptance, writers, release evidence, and bounded product gaps; +- if the first exit sweep finds work, execute it and sweep again; +- termination only on genuine practical execution/tool-budget exhaustion or a second fresh all-lanes-nonactionable sweep; and +- `continuation_handoff` only for genuine finite-run budget exhaustion, carrying exact deferred identities and next executable actions. + +### Negative tests / evidence + +- one pending review cannot end the run when another safe issue exists; +- prompt/doc/status/review/dispatch/Draft/Ready/auto-merge/commit/merge events cannot satisfy the exit predicate; +- elapsed time alone cannot satisfy the exit predicate; +- the first sweep finding work forces another action; +- the second sweep cannot reuse the first sweep's stale state; and +- a continuation handoff without a real budget boundary is invalid. + +### Residual risk + +The ChatGPT/connector execution environment can impose a real finite tool/run budget. The control plane cannot remove that external bound. It can only preserve exact continuation state and refuse to call budget exhaustion product completion. + +## 2. AT-02 — split-brain authority / evidence conflation + +### Threat + +Two or more evidence or actor classes believe they authorize the same operation: for example a model verdict is treated as human approval, a success status replaces a required Check Run, a PR-base snapshot is mistaken for the current live base tip, an external writer and scheduler mutate the same branch, or a documentation acceptance state is treated as protected-main implementation. + +### Failure stories + +1. An automated reviewer emits approval-like prose and a merge scheduler counts it as qualifying independent human review. +2. A stale successful Check Run or Commit Status from a predecessor head survives while the PR head moved. +3. A PR's `.base.sha` snapshot is used as if it were the current protected base tip after the base branch advances. +4. A source PR passes but protected-main/consumer behavior is still broken; incident status nevertheless changes to closed. +5. A branch writer and an autofix workflow each use separately read old heads and both push. +6. An `active_pr` documentation contract is presented as `implemented_on_protected_main`. + +### Required controls + +- distinct `check_evidence`, `status_evidence`, `review_evidence`, `model_evidence`, `workflow_evidence`, `dependency_evidence`, and security evidence classes; +- qualifying human review requires live formal review identity and current-head/ruleset eligibility; no model/check/status wording can synthesize it; +- `source_revision`, `pr_base_snapshot_sha`, independently resolved `live_base_tip_sha`, and protected `merge_revision` remain separate; +- expected-head mutations and immediate live refresh before writes; +- branch-scoped `writer_lease` and rotation when another writer becomes active; +- protected-main or real-consumer `operational_acceptance` attaches to `merge_revision`, not the source PR; +- controlled documentation maturity vocabulary; and +- strict versioned event/result envelopes with legacy/versioned parser non-conflation in [EVENT_CONTRACTS.md](EVENT_CONTRACTS.md). + +### Negative tests / evidence + +- model/comment/status-only `APPROVED` does not satisfy independent human review; +- stale-head evidence invalidates when source head or current live base moves; +- schema-free legacy dispatch cannot accept a versioned claim; +- a moved expected head rejects mutation even if all predecessor checks were green; +- only one active branch writer is permitted by the current lease decision; +- protected-source acceptance cannot be derived from a source-branch run; and +- `active_pr`/`accepted_architecture` cannot be emitted as `implemented_on_protected_main` without protected evidence. + +### Residual risk + +An organization owner or external platform authority can intentionally reconfigure protection outside repository-controlled code. Scheduled ruleset audit, provider/actor receipts, and protected-main acceptance reduce detection/reconstruction time but do not cryptographically prevent a fully authorized platform owner from changing policy. + +## 3. Combined failure mode + +Premature termination and split-brain authority can reinforce one another: a run sees an apparently green status from the wrong authority, stops early, and never reaches the lane that would have discovered the missing formal review or protected-main failure. Therefore work conservation and authority separation are one safety property, not independent conveniences. + +```mermaid +flowchart LR + A[Live queue and evidence] --> B[Normalize authority and exact identity] + B --> C{Any safe executable lane?} + C -->|yes| D[Acquire branch-local lease and act] + D --> E[Exact-head / protected evidence] + E --> A + C -->|no| F[Fresh exit sweep] + F -->|work found| D + F -->|none| G[Second fresh sweep] + G -->|work found| D + G -->|none or real budget exhausted| H[Termination / continuation handoff] +``` + +## 4. Ownership and reopening + +- automation maintainer owns work-conservation and continuation semantics; +- governance maintainer owns evidence/reviewer/merge authority; +- repository writer owns branch-local lease compliance; +- service owner owns protected-main/consumer acceptance for runtime incidents. + +Reopen this threat family when scheduling, reviewer eligibility, evidence classes, event schemas, writer authority, or protected-main acceptance changes. \ No newline at end of file From 7270fedaa5c53afab68004b5d592af4da113ffd2 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:39:37 +0900 Subject: [PATCH 19/69] docs(operability): define defer wait and termination receipts --- docs/automation/CONTINUATION_RUNBOOK.md | 161 ++++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 docs/automation/CONTINUATION_RUNBOOK.md diff --git a/docs/automation/CONTINUATION_RUNBOOK.md b/docs/automation/CONTINUATION_RUNBOOK.md new file mode 100644 index 000000000..0471e5d3f --- /dev/null +++ b/docs/automation/CONTINUATION_RUNBOOK.md @@ -0,0 +1,161 @@ +# Autonomous continuation runbook + +Status: active_pr operability companion +Last reviewed: 2026-08-09 +Scope: work-conserving queue rotation, wait/defer semantics, real run-budget exhaustion, and observable termination evidence + +This runbook operationalizes ADR-0007, [AUTONOMY_THREATS.md](AUTONOMY_THREATS.md), and the logical `continuation_handoff` model. It does not replace workflow-specific timeouts or GitHub rulesets. + +## 1. Operator rule + +A blocked or waiting item is **local state**, not run completion. The automation defers the exact identity and selects another safe lane. Hourly recurrence means continuation after a genuine practical run/tool limit, not “stop after about an hour.” + +## 2. Observable lane states and reason codes + +| Reason code | Meaning | Allowed action | Run terminal? | +|---|---|---|---| +| `EXECUTABLE_NOW` | Safe mutation/test/merge/operational action exists and writer lease is available. | Execute highest-value item. | No | +| `WAIT_CHECK_PENDING` | Exact-head required check is queued/in-progress. | Defer exact PR/head/check identity; rotate. | No | +| `WAIT_REVIEW_PENDING` | Current-head automated or human review is pending. | Defer exact PR/head/reviewer identity; rotate. | No | +| `WAIT_EXTERNAL_APPROVAL` | Qualifying independent human approval is the local remaining gate. | Preserve expected-head-safe merge posture if policy allows; rotate. | No | +| `WAIT_PROVIDER_COOLDOWN` | Provider/reviewer is rate-limited/unavailable. | Record bounded provider identity/time; rotate. | No | +| `WAIT_DEPENDENCY` | Another PR/release/protected-main change is an exact prerequisite. | Bind dependency identity; rotate to disjoint work. | No | +| `WAIT_WRITER_LEASE` | Another write-capable actor owns the exact branch/head. | Freeze only that branch; do not race. | No | +| `BLOCK_POLICY` | Live ruleset/permission/security policy forbids proposed action. | RCA; execute a feasible policy/source remedy if authorized, otherwise defer exact external prerequisite. | No by itself | +| `BLOCK_SOURCE_DEFECT` | Current-head source/test/security defect is proven. | Test-first repair or hand to the active exact-branch writer. | No | +| `BLOCK_INFRA_PERMANENT` | Auth/integrity/TLS/ref/schema/policy failure is fail-closed. | Do not retry as transient; repair root cause or defer external owner. | No | +| `RETRY_INFRA_TRANSIENT` | Bounded evidence class such as reset/DNS/5xx/capacity is eligible for limited retry. | Retry within component budget, then defer/rotate. | No | +| `META_INTERMEDIATE` | Prompt/doc/status/comment/review request/dispatch/Draft/Ready/auto-merge/commit/merge/document completion occurred. | Re-scan executable queue immediately. | No | +| `SWEEP1_EMPTY` | First fresh whole-queue sweep found no executable lane. | Perform second fresh sweep from new live reads. | No | +| `SWEEP2_EMPTY` | Second fresh whole-queue sweep independently found no executable lane. | Termination is permitted if no real budget/error boundary requires a handoff. | Yes | +| `RUN_BUDGET_EXHAUSTED` | External execution/tool boundary prevents further safe calls in this finite invocation. | Emit exact continuation handoff; next recurrence resumes. | Yes, as continuation—not completion | +| `SAFETY_BOUNDARY` | Policy/safety tool denies further operation across all applicable lanes. | Preserve evidence and stop; user/operator escalation only if needed. | Yes | + +“Elapsed time exceeds N minutes” is intentionally absent. Component timeouts may have explicit finite budgets, but the outer maintenance run cannot invent a soft elapsed-time terminal reason. + +## 3. Defer key + +A deferred item must be precise enough not to suppress unrelated work: + +```json +{ + "repository": "ContextualWisdomLab/.github", + "work_kind": "pull_request_check", + "pull_request_number": 123, + "source_head_sha": "<40 hex>", + "live_base_tip_sha": "<40 hex when material>", + "external_identity": "workflow/check/reviewer/dependency identity", + "reason_code": "WAIT_CHECK_PENDING", + "observed_at": "" +} +``` + +Do not defer an entire repository merely because one branch or provider item waits. A changed head/base/external identity invalidates the old defer key and requires a fresh observation. + +## 4. Continuation handoff receipt + +Only `RUN_BUDGET_EXHAUSTED` needs a `continuation_handoff` when executable work remains. The bounded receipt contains identifiers and next actions, not raw logs or credentials. + +```json +{ + "schema": "cwl.automation-continuation/v1", + "run_identity": "", + "reason_code": "RUN_BUDGET_EXHAUSTED", + "recorded_at": "", + "deferred_items": [ + { + "repository": "ContextualWisdomLab/.github", + "pull_request_number": 123, + "source_head_sha": "<40 hex>", + "reason_code": "WAIT_REVIEW_PENDING", + "external_identity": "" + } + ], + "next_executable_lanes": [ + "re-fetch exact PR heads/live bases and resume highest-value executable work" + ] +} +``` + +The receipt is not persisted in a new database by this document. A GitHub artifact, automation task state, or other existing bounded continuation store may represent it. Persistence changes require the data-model ADR gate. + +## 5. Double exit sweep + +### Sweep 1 + +Freshly enumerate: + +- every open PR and exact current head; +- independently resolved live base tips where material; +- open issues and dependencies; +- required checks/statuses/reviews/threads/security evidence; +- active writers and branch-local leases; +- protected-main/consumer acceptance still owed by merged repairs; +- canonical documentation versus live implementation; +- tests/coverage/docstrings/security/supply-chain/release debt; and +- one bounded buyer/control-plane gap if existing work is non-actionable. + +If any safe action exists, classify `EXECUTABLE_NOW`, execute it, and the exit sequence resets. + +### Sweep 2 + +Only after `SWEEP1_EMPTY`, perform the same inventory from **new live reads**. Do not reuse cached PR/check/reviewer state as proof. If still empty, `SWEEP2_EMPTY` permits termination. + +A queued check or external approval on one PR does not make the sweep empty if any other safe lane exists. + +## 6. Retry and timeout interaction + +Workflow/component timeouts remain bounded and are separate from maintenance-run termination. + +- provider attempt timeout → classify provider result and continue/fallback/defer according to provider policy; +- network/bootstrap transient retry → bounded retry count/time and then rotate; +- integrity/auth/TLS/ref/schema failure → immediate permanent/fail-closed class, no transient retry; +- long OpenCode/Noema/Strix job → defer exact run identity and continue other work; +- GitHub Actions queue saturation → do not poll; advance source/docs/issues that do not conflict. + +## 7. Split authority operational check + +Before merge, mutation, or protected incident closure, operator evidence must answer separately: + +- Which exact source head was evaluated? +- What PR-base snapshot was observed? +- What is the independently resolved current live base tip? +- Which required Check Runs passed on the exact head? +- Which Commit Statuses exist and who created them? +- Which formal reviews are current and which reviewer, if any, qualifies independently? +- Which model judgments exist and what authority class do they have? +- Which branch writer owns the mutation lane? +- Did GitHub accept the expected-head mutation? +- What protected merge revision resulted? +- What protected-main/consumer acceptance scenario proves runtime closure? + +A missing answer cannot be inferred from another channel. + +## 8. Monitoring + +Useful finite-cardinality measures: + +- oldest executable-lane age; +- oldest deferred-lane age by reason code; +- count of branch-local writer collisions avoided; +- transient retry attempts/exhaustions by failure class; +- time from exact-head gate-clean to protected merge; +- time from protected merge to operational acceptance; +- count of `META_INTERMEDIATE` events followed by another substantive action; +- count of first exit sweeps that discovered work; and +- number of `RUN_BUDGET_EXHAUSTED` continuation handoffs with executable lanes remaining. + +Do not place repository source text, comment bodies, PII, model output, or credentials in metric labels. + +## 9. Reopening and escalation + +Reopen continuation incidents when: + +- a run terminates after a meta/control event while safe work existed; +- a waiting PR starves an unrelated lane; +- a defer key suppresses a changed head/base or unrelated branch; +- a second exit sweep reused stale evidence; +- `RUN_BUDGET_EXHAUSTED` is reported as software/product completion; or +- one evidence authority is used to fill a missing field from another authority. + +Escalate to the user/operator only when a genuinely external permission/governance/safety decision is required **and** the fresh whole queue has no other safe executable work. \ No newline at end of file From 940fd5fa7221d55ef3c140fa70c6e7d3d3b6952f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:40:15 +0900 Subject: [PATCH 20/69] docs(automation): index autonomy threat and continuation runbook --- docs/automation/README.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/automation/README.md b/docs/automation/README.md index f086ef9f9..ca79b184b 100644 --- a/docs/automation/README.md +++ b/docs/automation/README.md @@ -14,12 +14,10 @@ This directory is the durable specification for the ContextualWisdomLab automati 4. [ARCHITECTURE.md](ARCHITECTURE.md) — viewpoints, bounded contexts, trust boundaries, and failure domains. 5. [DATA_MODEL.md](DATA_MODEL.md) and [ERD.md](ERD.md) — conceptual evidence model and exact-path logical ERD; neither claims a deployed database. -6. [UML.md](UML.md) — component, sequence, state, authority, deployment, and incident flows. -7. [SECURITY.md](SECURITY.md) and [THREAT_MODEL.md](THREAT_MODEL.md) — security objectives, controls, abuse cases, and residual risk. +6. [UML.md](UML.md) — component, sequence, state, authority, deployment, writer-rotation, and documentation-continuation flows. +7. [SECURITY.md](SECURITY.md), [THREAT_MODEL.md](THREAT_MODEL.md), and [AUTONOMY_THREATS.md](AUTONOMY_THREATS.md) — security objectives, general abuse cases, premature-termination/split-authority threats, and residual risk. 8. [TEST_STRATEGY.md](TEST_STRATEGY.md) — deterministic, security, compatibility, documentation, and protected-main acceptance gates. -9. [OPERABILITY.md](OPERABILITY.md), [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md), - and executable [RUNBOOK.md](RUNBOOK.md) — service objectives, observability, - diagnosis, ownership, recovery, retention, rollback, and acceptance receipts. +9. [OPERABILITY.md](OPERABILITY.md), [INCIDENT_RUNBOOK.md](INCIDENT_RUNBOOK.md), executable [RUNBOOK.md](RUNBOOK.md), and [CONTINUATION_RUNBOOK.md](CONTINUATION_RUNBOOK.md) — service objectives, observability, diagnosis, defer/wait reason codes, double-exit sweeps, ownership, recovery, retention, rollback, and acceptance receipts. 10. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. 11. [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md) — whole-conversation fitness, central-versus-leaf ownership, maturity states, and no-soft-timeout continuation contract. 12. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. From 92baf6f598bcbd3e275892ae84b6af9ab3b302db Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:41:03 +0900 Subject: [PATCH 21/69] test(docs): bind autonomy threat and continuation runbook --- .../test_automation_documentation_contract.py | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 158c140fe..5dfd52567 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -20,10 +20,12 @@ "UML.md", "SECURITY.md", "THREAT_MODEL.md", + "AUTONOMY_THREATS.md", "TEST_STRATEGY.md", "OPERABILITY.md", "INCIDENT_RUNBOOK.md", "RUNBOOK.md", + "CONTINUATION_RUNBOOK.md", "TRACEABILITY.md", "DOCUMENTATION_AUDIT.md", ) @@ -246,6 +248,34 @@ def test_work_conservation_forbids_meta_action_and_soft_timeout_completion() -> assert "supplemental visualization" in uml +def test_autonomy_threats_and_continuation_reason_codes_are_explicit() -> None: + """Premature termination and split authority stay tied to operational ledger semantics.""" + + threats = read_document("AUTONOMY_THREATS.md") + runbook = read_document("CONTINUATION_RUNBOOK.md") + for threat_name in ( + "premature termination", + "false quiescence", + "split-brain authority", + ): + assert threat_name in threats + for reason_code in ( + "EXECUTABLE_NOW", + "WAIT_CHECK_PENDING", + "WAIT_EXTERNAL_APPROVAL", + "WAIT_WRITER_LEASE", + "META_INTERMEDIATE", + "SWEEP1_EMPTY", + "SWEEP2_EMPTY", + "RUN_BUDGET_EXHAUSTED", + ): + assert reason_code in runbook + assert "cwl.automation-continuation/v1" in runbook + assert "Elapsed time exceeds N minutes" in runbook + assert "source_head_sha" in runbook + assert "live_base_tip_sha" in runbook + + def test_remediation_continuation_and_documentation_entities_are_explicit() -> None: """RCA alternatives, handoff, documentation and traceability stay first-class logical concepts.""" @@ -365,7 +395,9 @@ def test_security_and_operations_contracts_keep_authorities_separate() -> None: "EVENT_CONTRACTS.md", "SECURITY.md", "THREAT_MODEL.md", + "AUTONOMY_THREATS.md", "OPERABILITY.md", + "CONTINUATION_RUNBOOK.md", "INCIDENT_RUNBOOK.md", "TRACEABILITY.md", ) @@ -397,6 +429,8 @@ def test_timeless_architecture_does_not_embed_transient_run_identity() -> None: "DATA_MODEL.md", "UML.md", "SECURITY.md", + "AUTONOMY_THREATS.md", + "CONTINUATION_RUNBOOK.md", ) ) assert re.search(r"\b[0-9a-f]{40}\b", stable_corpus) is None From 23d897b2700463326745976e53a221d63ce6f21f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:56:26 +0900 Subject: [PATCH 22/69] docs(threats): align maturity and redaction lineage --- docs/automation/THREAT_MODEL.md | 36 ++++++++++++++++----------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/automation/THREAT_MODEL.md b/docs/automation/THREAT_MODEL.md index a274aac6f..ce95f981c 100644 --- a/docs/automation/THREAT_MODEL.md +++ b/docs/automation/THREAT_MODEL.md @@ -166,24 +166,24 @@ Threats are reopened whenever source, identity, permissions, provider, retention ## 7. Exact source and test traceability -Paths below are the current concrete control and regression locations. A listed path is not proof that every branch is implemented; the status column names material gaps. +Paths below are the current concrete control and regression locations. A listed path is not proof that every branch is implemented; the closure and maturity columns name material gaps. -| Threat | Exact control source | Exact regression evidence | Current status | -|---|---|---|---| -| TM-01 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Protected integration and consumer proof tracked by PR #842. | -| TM-02 | `.github/workflows/deploy-pages.yml`; `docs/automation/SECURITY.md` | `tests/test_automation_documentation_contract.py` | Partial: documentation guard only; explicit-secret migration and a runtime negative contract remain open. | -| TM-03 | `.github/workflows/agent-mention-router.yml`; `.github/workflows/opencode-review-dispatch.yml`; `scripts/ci/agent_mention_router.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_agent_mention_complete_payload_binding.py`; `tests/test_agent_mention_idempotency.py`; `tests/test_agent_mention_workflow_contract.py`; `tests/test_opencode_security_boundaries.py` | Implemented for named paths; every new dispatch type needs equivalent binding. | -| TM-04 | `scripts/ci/pr_head_replay_guard.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_head_replay_guard.py`; `tests/test_pr_review_merge_scheduler.py` | Implemented with live-ruleset dependency. | -| TM-05 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/opencode_existing_approval_gate.py`; `scripts/ci/noema_review_gate.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_opencode_existing_approval_gate.py`; `tests/test_noema_review_gate.py` | Implemented with live reviewer/ruleset configuration dependency. | -| TM-06 | `.github/workflows/opencode-review.yml`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/security-scan.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_review_execution_contracts.py`; `tests/test_required_workflow_queue_contract.py` | Implemented for inventoried required entrypoints; workflow inventory must remain current. | -| TM-07 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/opencode_adversarial_receipts.py`; `scripts/ci/review_execution_contracts.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_review_execution_contracts.py` | Defense in depth; deterministic and human gates remain mandatory. | -| TM-08 | `scripts/ci/compile_opencode_review_lock.sh`; `scripts/ci/materialize_base_python_requirements.py`; `.github/workflows/trusted-uv-materializer-quality-ci.yml` | `tests/test_trusted_uv_download_contract.py`; `tests/test_uv_redirect_boundary.py`; `tests/test_strix_workflow_dependency_hashes.py` | Implemented for covered toolchains; new ecosystems require inventory. | -| TM-09 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_noema_review_gate.py`; `tests/test_opencode_security_boundaries.py` | Partial: routing is tested; repository data-owner consent/classification remains governance evidence. | -| TM-10 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Partial: algorithmic fixtures exist; total capture and service-file quotas remain open. | -| TM-11 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/pr_review_fix_scheduler.py`; `scripts/ci/pr_auto_rebase.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py`; `tests/test_pr_auto_rebase.py` | Partial: expected-head guards exist; no single persisted fleet writer-lease service. | -| TM-12 | `scripts/ci/opencode_dispatch_status.py`; `scripts/ci/agent_mention_router.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_agent_mention_receipt_authority.py`; `tests/test_agent_mention_artifact_ledger.py`; `tests/test_pr_head_replay_guard.py` | Partial: receipts are distributed and retention can expire them. | -| TM-13 | `docs/automation/SECURITY.md`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_automation_documentation_contract.py` | Partial: public/private routing and policy are checked; no complete access/retention audit gate. | -| TM-14 | `scripts/ci/run_opencode_review_model_pool.sh`; `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/strix_model_utils.sh` | `tests/test_opencode_model_pool_runner.py`; `tests/test_pr_review_merge_scheduler.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | Implemented for covered retry loops; each new retry path requires classification fixtures. | -| TM-15 | `docs/automation/RUNBOOK.md`; `docs/automation/INCIDENT_RUNBOOK.md`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_automation_documentation_contract.py` | Partial: guarded merge tests exist; fleet rollback rehearsal and consumer receipt remain operational evidence. | +| Threat | Exact control source | Exact regression evidence | Closure boundary | Current maturity | +|---|---|---|---|---| +| TM-01 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Clean-history integration and consumer proof are proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888). | `active_pr` | +| TM-02 | `.github/workflows/deploy-pages.yml`; `docs/automation/SECURITY.md` | `tests/test_automation_documentation_contract.py` | Documentation detects known debt; explicit workflow-call secret declarations and a runtime negative contract remain open. | `accepted_architecture` | +| TM-03 | `.github/workflows/agent-mention-router.yml`; `.github/workflows/opencode-review-dispatch.yml`; `scripts/ci/agent_mention_router.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_agent_mention_complete_payload_binding.py`; `tests/test_agent_mention_idempotency.py`; `tests/test_agent_mention_workflow_contract.py`; `tests/test_opencode_security_boundaries.py` | Named router/wrapper controls exist; end-to-end strict snapshot preservation remains proposed by PR #840. | `active_pr` | +| TM-04 | `scripts/ci/pr_head_replay_guard.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_head_replay_guard.py`; `tests/test_pr_review_merge_scheduler.py` | Enforcement still depends on current live ruleset configuration. | `implemented_on_protected_main` | +| TM-05 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/opencode_existing_approval_gate.py`; `scripts/ci/noema_review_gate.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_opencode_existing_approval_gate.py`; `tests/test_noema_review_gate.py` | The software enforces eligibility; human reviewer capacity remains external governance work. | `implemented_on_protected_main` | +| TM-06 | `.github/workflows/opencode-review.yml`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/security-scan.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_review_execution_contracts.py`; `tests/test_required_workflow_queue_contract.py` | Covered entrypoints are protected; every new entrypoint must join the inventory. | `implemented_on_protected_main` | +| TM-07 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/opencode_adversarial_receipts.py`; `scripts/ci/review_execution_contracts.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_review_execution_contracts.py` | Defense in depth does not replace deterministic and qualifying-human gates. | `implemented_on_protected_main` | +| TM-08 | `scripts/ci/compile_opencode_review_lock.sh`; `scripts/ci/materialize_base_python_requirements.py`; `.github/workflows/trusted-uv-materializer-quality-ci.yml` | `tests/test_trusted_uv_download_contract.py`; `tests/test_uv_redirect_boundary.py`; `tests/test_strix_workflow_dependency_hashes.py` | Covered toolchains are pinned; new ecosystems require an inventory extension. | `implemented_on_protected_main` | +| TM-09 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_noema_review_gate.py`; `tests/test_opencode_security_boundaries.py` | Routing is tested; repository data-owner consent/classification remains governance evidence. | `accepted_architecture` | +| TM-10 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Redaction is active work; total capture and service-file quotas remain separate planned work. | `active_pr` | +| TM-11 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/pr_review_fix_scheduler.py`; `scripts/ci/pr_auto_rebase.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py`; `tests/test_pr_auto_rebase.py` | Expected-head guards exist; a shared cross-workflow lease is planned in issue #890. | `planned` | +| TM-12 | `scripts/ci/opencode_dispatch_status.py`; `scripts/ci/agent_mention_router.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_agent_mention_receipt_authority.py`; `tests/test_agent_mention_artifact_ledger.py`; `tests/test_pr_head_replay_guard.py` | Receipts are distributed and retention can expire them; recoverable mention claims are planned. | `accepted_architecture` | +| TM-13 | `docs/automation/SECURITY.md`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_automation_documentation_contract.py` | Public/private routing is checked; no complete access/retention audit gate exists. | `accepted_architecture` | +| TM-14 | `scripts/ci/run_opencode_review_model_pool.sh`; `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/strix_model_utils.sh` | `tests/test_opencode_model_pool_runner.py`; `tests/test_pr_review_merge_scheduler.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | Covered retry loops are classified; new paths require equivalent fixtures. | `implemented_on_protected_main` | +| TM-15 | `docs/automation/RUNBOOK.md`; `docs/automation/INCIDENT_RUNBOOK.md`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_automation_documentation_contract.py` | Guarded merge tests exist; fleet rollback rehearsal and consumer receipts remain operational evidence. | `accepted_architecture` | Cross-document requirement status remains in [TRACEABILITY.md](TRACEABILITY.md). Incident commands and evidence templates are in [RUNBOOK.md](RUNBOOK.md). From b2f5c06a047378edd6f61141f60e75a9eb354274 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:56:45 +0900 Subject: [PATCH 23/69] docs(adr): point redaction decision to clean successor --- docs/automation/adr/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/automation/adr/README.md b/docs/automation/adr/README.md index a7d5a1b2d..4de778384 100644 --- a/docs/automation/adr/README.md +++ b/docs/automation/adr/README.md @@ -14,7 +14,7 @@ ADRs record durable decisions, not transient run status. A changed exact head, r | [ADR-0006](0006-protected-main-operational-acceptance.md) | Protected-main/consumer evidence closes operational incidents | Accepted | | [ADR-0007](0007-work-conserving-maintenance.md) | Work-conserving automation; reporting is not completion | Accepted | | [ADR-0008](0008-central-control-plane-and-thin-consumers.md) | Central control-plane ownership and thin product consumers | Accepted | -| [ADR-0009](0009-sandbox-evidence-redaction-boundary.md) | One complete sandbox evidence-redaction boundary with diagnostic preservation | Accepted; integration proof pending in `.github#842` | +| [ADR-0009](0009-sandbox-evidence-redaction-boundary.md) | One complete sandbox evidence-redaction boundary with diagnostic preservation | Accepted; clean-history integration proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) | | [ADR-0010](0010-agent-mention-routing-and-idempotency-ledger.md) | Authenticated agent-mention routing with an idempotency ledger | Accepted | | [ADR-0011](0011-provider-routing-and-credential-isolation.md) | Ordered provider routing with per-purpose credential isolation | Accepted | | [ADR-0012](0012-hash-pinned-toolchains-and-exact-base-materialization.md) | Hash-pinned toolchains and independently resolved live-base identity | Accepted | From 89c361b5db10866adeea75240803c8510218af2a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:56:49 +0900 Subject: [PATCH 24/69] docs(adr): correct redaction integration lineage --- .../adr/0009-sandbox-evidence-redaction-boundary.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md index 52e764915..c679e0b58 100644 --- a/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md +++ b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md @@ -1,6 +1,6 @@ # ADR-0009: Redact the complete sandbox evidence boundary while preserving diagnosis -Status: Accepted; protected integration and consumer proof pending in `ContextualWisdomLab/.github#842` +Status: Accepted; protected integration and consumer proof proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) Date: 2026-08-09 Decision owners: CWL security and CI maintainers @@ -8,7 +8,7 @@ Decision owners: CWL security and CI maintainers `sandboxed_verify.py` and `sandboxed_web_e2e.py` execute proof commands and publish completed output, timeout payloads, exceptions, backend/frontend service tails, command metadata, and structured result JSON. Applying redaction to only some print calls leaves credentials reachable through JSON, terminal/Unicode evasion, nested commands, explicit allowed environment values, or stream/result boundaries. Over-broad pattern matching can instead erase benign keys and the diagnostics needed to fix CI. -`ContextualWisdomLab/.github#841` identified the disclosure class but mixed an unrelated SSRF slice and missed publication paths. It was closed unmerged. #842 is the clean replacement and preserves #841 as historical evidence. +`ContextualWisdomLab/.github#841` identified the disclosure class but mixed an unrelated SSRF slice and missed publication paths. It was closed unmerged. Its first broad successor was also closed unmerged because secret-shaped fixtures remained in reachable history. [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) is the clean-history active replacement and preserves both predecessors as historical evidence. ## Decision drivers @@ -62,7 +62,7 @@ This reduces credential disclosure and log-injection risk while preserving audit ## Migration and rollback -Integrate #842 through normal protection, then run a synthetic credential fixture from protected main and an affected consumer. Update the PR body, doctoring, and CHANGELOG with only final exact-head numbers. If regression requires rollback, disable the affected evidence publication or revert to a reviewed fail-closed no-output path; do not restore a known disclosure path. +Integrate [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) through normal protection, then run a synthetic credential fixture from protected main and an affected consumer. Update the PR body, doctoring, and CHANGELOG with only final exact-head numbers. If regression requires rollback, disable the affected evidence publication or revert to a reviewed fail-closed no-output path; do not restore a known disclosure path. ## Supersession conditions From b822cbc39107b45857564a0ce60c27fb0a32f6a4 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:57:38 +0900 Subject: [PATCH 25/69] docs(doctoring): index control-plane research authority --- docs/doctoring/README.md | 45 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 docs/doctoring/README.md diff --git a/docs/doctoring/README.md b/docs/doctoring/README.md new file mode 100644 index 000000000..048333589 --- /dev/null +++ b/docs/doctoring/README.md @@ -0,0 +1,45 @@ +# Doctoring and reference authority + +Status: active_pr reference index +Last reviewed: 2026-08-09 +Scope: source-backed technical and standards evidence used by the `ContextualWisdomLab/.github` automation control plane + +Doctoring records explain **why** a control, algorithm, compatibility boundary, incident repair, or version decision exists. They do not override live source, GitHub rulesets, exact-head evidence, the canonical automation PRD/TRD/Architecture/ADR graph, or product-owned specifications in leaf repositories. + +## Canonical standards baseline + +- [Automation control-plane standards baseline](automation-control-plane-standards.md) — current final versus draft version discipline; NIST SSDF and CI/CD supply-chain guidance; SLSA; ISO/IEC/IEEE architecture, requirements, testing and product-quality baselines; ISO/IEC security/AI management references; GitHub secure-use/OIDC; OpenTelemetry; SOC 2 evidence limits; conditional CSAP readiness; APA 7 references. + +This file is the discoverable standards authority for the central control plane. When a final/draft status changes, update the standards baseline and its documentation-fitness test in the same reviewed change; do not silently replace the normative reference from a PR body or prompt. + +## Implementation-specific doctoring + +Scenario-specific doctoring remains close to the implementation it justifies. For example: + +- [Trusted uv lock materialization](trusted-uv-lock-materialization.md) — exact-base Python dependency materialization and trust boundaries. + +Other implementation-specific doctoring files may be added by focused PRs and linked from their ADR, traceability row, or owning workflow/test contract. They are not automatically normative for unrelated subsystems merely because they live under this directory. + +## Evidence hierarchy + +For current decisions, use this order: + +1. live GitHub repository/PR/ruleset/check/review/workflow evidence; +2. protected-main implementation source for observed runtime behavior; +3. `docs/automation/**` and accepted ADRs for intended central contracts; +4. this doctoring set for standards/research rationale and version decisions; +5. dated PR/incident evidence for historical reconstruction; and +6. conversation, prompts, planning packs, and model output as candidate evidence only until revalidated. + +## APA 7 and source-quality rules + +- Prefer current official final standards, primary technical documentation, and peer-reviewed primary research where material. +- Cite draft standards only when the draft itself is relevant and label its status explicitly; a draft does not silently supersede the current final publication. +- Record retrieval dates for undated or rapidly changing online technical documentation when the version/page does not provide a durable publication date. +- Use APA 7 references in doctoring and ADR material where external evidence materially supports a design decision. +- Separate a source's factual statement from CWL's inference or control decision. +- Never use citations to imply certification, formal conformance, market value, or operating effectiveness that has not been independently established. + +## Central versus leaf ownership + +This index covers the shared automation control plane only. Product-specific research/design for TEPP/psychometrics, fast-mlsirm, OriginWeave, EmbedRelay, MHTML ETL, LifeOS, BandScope, Inkspan, pg-erd-cloud, naruon, AppGuardrail, and other leaf products belongs in each owning repository. Central doctoring may cite those systems only when their behavior is evidence for a reusable cross-repository automation/interface decision. \ No newline at end of file From 76f0d38726a915024c764d8dc06f253a2e9dc994 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:58:11 +0900 Subject: [PATCH 26/69] docs(automation): link doctoring reference authority --- docs/automation/README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/automation/README.md b/docs/automation/README.md index ca79b184b..cfcbbd8bd 100644 --- a/docs/automation/README.md +++ b/docs/automation/README.md @@ -21,6 +21,7 @@ This directory is the durable specification for the ContextualWisdomLab automati 10. [TRACEABILITY.md](TRACEABILITY.md) — requirement-to-code-to-test-to-operation evidence and known gaps. 11. [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md) — whole-conversation fitness, central-versus-leaf ownership, maturity states, and no-soft-timeout continuation contract. 12. [adr/README.md](adr/README.md) — accepted and proposed architecture decisions. +13. [../doctoring/README.md](../doctoring/README.md) — discoverable standards/research authority, APA 7 source-quality rules, and links to implementation-specific doctoring. ## Maturity and authority discipline @@ -31,8 +32,9 @@ The sources of truth are ordered as follows: 1. live GitHub repository, pull-request, ruleset, check, review, and workflow evidence for current state; 2. protected default-branch workflow and script source for implemented behavior; 3. this documentation set for intended cross-component contracts; -4. dated incident receipts and pull-request bodies for historical evidence only; and -5. conversation, prompts, planning packs, and model output as candidate evidence only until revalidated and canonicalized. +4. the indexed doctoring/reference authority for external standards, primary technical documentation, research rationale, and version decisions; +5. dated incident receipts and pull-request bodies for historical evidence only; and +6. conversation, prompts, planning packs, and model output as candidate evidence only until revalidated and canonicalized. Historical SHAs, run IDs, check counts, and performance numbers belong in dated evidence or pull-request records, not timeless architecture. A head change makes predecessor-head checks and reviews historical. A base-branch change requires independently resolving the new live base tip and regenerating evidence. From 5ee7554d40f44b8608e083cb7dab5b32ed941f7a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:58:13 +0900 Subject: [PATCH 27/69] docs(data-model): make run parents optional and map issuers --- docs/automation/DATA_MODEL.md | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/docs/automation/DATA_MODEL.md b/docs/automation/DATA_MODEL.md index fcbda1a96..24d4a575b 100644 --- a/docs/automation/DATA_MODEL.md +++ b/docs/automation/DATA_MODEL.md @@ -39,6 +39,9 @@ erDiagram datetime started_at datetime completed_at string run_outcome + string issuer_kind + string external_run_locator + string source_not_applicable_reason } pull_request_snapshot { string pull_request_snapshot_uid PK @@ -81,10 +84,11 @@ erDiagram source_revision ||--o{ status_evidence : binds source_revision ||--o{ review_evidence : binds source_revision ||--o{ model_evidence : binds - source_revision ||--o{ workflow_evidence : binds + source_revision o|--o{ workflow_evidence : optionally_binds source_revision ||--o{ dependency_evidence : depends_on workflow_evidence ||--o{ check_evidence : reports workflow_evidence ||--o{ model_evidence : reports + merge_revision o|--o{ workflow_evidence : may_validate check_evidence { string check_evidence_uid PK @@ -122,6 +126,9 @@ erDiagram bigint workflow_run_id int workflow_run_attempt string workflow_conclusion + string evidence_issuer_kind + string external_run_locator + string subject_kind } dependency_evidence { string dependency_evidence_uid PK @@ -143,6 +150,23 @@ reviewer/model verdict is discussed in prose. The stored authority class remains model evidence unless GitHub independently records a formal review through a reviewer identity whose eligibility is validated separately. + +### 3.1 Runtime receipt and issuer aliases + +| Observed object | Canonical entity | Required issuer and identity mapping | +|---|---|---| +| GitHub Actions workflow/job/attempt | `workflow_evidence` | `evidence_issuer_kind=github_actions`, workflow name/source SHA, run ID, attempt, conclusion, and exact subject kind. | +| External hourly or manual bounded execution | `automation_run` | `issuer_kind`, auditable `external_run_locator`, run mode/outcome, and either exact source/base links or `source_not_applicable_reason`; an `orchestration_run` parent is optional. | +| Organization-wide scheduled invocation | `orchestration_run` plus repository-scoped `automation_run` children | External scheduler identity, start/end/outcome, and one child per repository; it is not itself a source revision or writer lease. | +| Protected merge API response | `merge_revision` | Merge commit, kind, protected ref, integration time, and optional validating `workflow_evidence`; it is never aliased to a source-head check. | +| Commit Status, Check Run, formal review, or model output | `status_evidence`, `check_evidence`, `review_evidence`, or `model_evidence` respectively | Preserve native issuer, object identity, observed revision, and authority class; similar display names do not permit conversion. | + +`workflow_evidence.subject_kind` is one of source revision, live-base +observation, merge revision, scheduled inventory, or explicit +`not_applicable`. A non-PR scheduled run therefore has no invented +`source_revision`; it records the protected workflow source and reason. +External run locators are identifiers or receipt URLs, never credential values. + ## 4. Operations, RCA, continuation, and documentation ERD ```mermaid @@ -245,7 +269,7 @@ claiming a database. erDiagram organization_target ||--o{ repository_target : governs organization_target ||--o{ orchestration_run : schedules - orchestration_run ||--o{ automation_run : contains + orchestration_run o|--o{ automation_run : optionally_contains organization_target ||--o{ ruleset_snapshot : observes repository_target ||--o{ ruleset_snapshot : applies_to automation_run ||--o{ dispatch_envelope : emits @@ -404,7 +428,7 @@ be collapsed into one pass/fail row. 13. A ruleset snapshot always has an observation time and digest; stale snapshots cannot authorize a current mutation. 14. One invocation idempotency key has at most one completed claim; retries link rather than overwrite their predecessor. 15. Checks, statuses, reviews, model results, scheduler decisions, security findings, threads, sandbox results, and SBOMs retain their distinct issuer and authority class. -16. An organization-wide invocation is one `orchestration_run` with repository-scoped `automation_run` children. Atomic evidence and mutations remain bound to one `repository_target`; a fleet parent does not imply a cross-repository writer lease. +16. An organization-wide invocation is one `orchestration_run` with repository-scoped `automation_run` children. A standalone automation_run created by one GitHub workflow, manual recovery, or external bounded action may have no orchestration parent. Atomic evidence and mutations remain bound to one `repository_target`; a fleet parent does not imply a cross-repository writer lease. ## 9. Future persistence decision gate From eedfb26ee4ddf70088fd5be735cd2607f4c89f44 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 22:58:39 +0900 Subject: [PATCH 28/69] test(docs): require discoverable doctoring authority --- ...est_automation_doctoring_index_contract.py | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 tests/test_automation_doctoring_index_contract.py diff --git a/tests/test_automation_doctoring_index_contract.py b/tests/test_automation_doctoring_index_contract.py new file mode 100644 index 000000000..9b4e5e728 --- /dev/null +++ b/tests/test_automation_doctoring_index_contract.py @@ -0,0 +1,53 @@ +"""Contracts for the discoverable automation doctoring/reference authority.""" + +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +DOCTORING_INDEX = ROOT / "docs" / "doctoring" / "README.md" +STANDARDS = ROOT / "docs" / "doctoring" / "automation-control-plane-standards.md" +AUTOMATION_INDEX = ROOT / "docs" / "automation" / "README.md" + + +def test_doctoring_authority_is_indexed_from_canonical_automation_docs() -> None: + """The canonical automation index links to one discoverable doctoring authority.""" + + assert DOCTORING_INDEX.is_file() + assert STANDARDS.is_file() + doctoring = DOCTORING_INDEX.read_text(encoding="utf-8") + automation = AUTOMATION_INDEX.read_text(encoding="utf-8") + assert "automation-control-plane-standards.md" in doctoring + assert "../doctoring/README.md" in automation + assert "APA 7" in doctoring + assert "candidate evidence" in doctoring + + +def test_doctoring_index_preserves_central_leaf_ownership_boundary() -> None: + """Central reference authority does not silently absorb product specifications.""" + + doctoring = DOCTORING_INDEX.read_text(encoding="utf-8") + assert "shared automation control plane only" in doctoring + for leaf_product in ( + "TEPP", + "fast-mlsirm", + "OriginWeave", + "EmbedRelay", + "MHTML ETL", + "LifeOS", + "BandScope", + "Inkspan", + "pg-erd-cloud", + "naruon", + "AppGuardrail", + ): + assert leaf_product in doctoring + + +def test_standards_reference_index_does_not_claim_certification() -> None: + """Reference indexing cannot be mistaken for certification or formal assurance.""" + + doctoring = DOCTORING_INDEX.read_text(encoding="utf-8") + standards = STANDARDS.read_text(encoding="utf-8") + assert "Never use citations to imply certification" in doctoring + assert "does not establish certification" in standards + assert "formal conformance" in standards From d93b0cf96e821f702233cfc93dc7345241858399 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:00:08 +0900 Subject: [PATCH 29/69] docs(security): register secrets and disclose mapping debt --- docs/automation/SECURITY.md | 54 ++++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/docs/automation/SECURITY.md b/docs/automation/SECURITY.md index 665780cc7..aa191e6fa 100644 --- a/docs/automation/SECURITY.md +++ b/docs/automation/SECURITY.md @@ -35,7 +35,59 @@ Last reviewed: 2026-08-09 | `github.token` | Same-repository capability under explicit job permissions | Assumed cross-repository authority | | `COPILOT_GITHUB_TOKEN` | None in autonomous development | Any autonomous development or review execution | -New reusable workflows must declare explicit secrets. `secrets: inherit` is prohibited unless a reviewed compatibility exception enumerates why every inherited secret is safe. The present `deploy-pages.yml` example is legacy debt and must not be copied into new contracts. +Reusable workflows must declare named secrets and callers must pass only +explicitly mapped secrets matching the callee's `on.workflow_call` contract. +`secrets: inherit` is prohibited for approved central consumers. This is the +target contract, not a claim that every protected-main path already conforms. + +### 3.1 Known protected-main mapping debt + +The registry proves name inventory, not reusable-call mapping. Source audit at +this review found secret expressions without matching reusable +`workflow_call.secrets` declarations in these exact paths: + +- `.github/workflows/pr-review-fix-scheduler.yml`; +- `.github/workflows/pr-review-merge-scheduler.yml`; +- `.github/workflows/pr-auto-rebase.yml`; and +- `.github/workflows/deploy-pages.yml`. + +The deploy contract has an active narrow repair in [PR #901](https://github.com/ContextualWisdomLab/.github/pull/901). +The other three remain planned migration debt and must not be reported as +explicitly mapped until their declarations, caller inventory, negative tests, +and protected consumer receipts exist. Runtime fallback credentials remain +purpose-separated; documenting debt does not authorize inheritance. + +### 3.2 Value-free workflow secret registry + +This registry is value-free and covers the exact union of literal +`secrets.NAME` and `secrets['NAME']` references in tracked `*.yml` and +`*.yaml` workflows at the reviewed revision. “Optional” means the feature +fails closed or uses a separately documented narrower authority; it never +permits synthetic success. Every new workflow secret updates this table in the +same change. + +| Secret name | Consumer and minimum scope | Requirement and owner | Rotation or revocation | +|---|---|---|---| +| `CLOUDFLARE_ACCOUNT_ID` | Cloudflare DNS/Pages account identifier | Required for the named Cloudflare operation; infrastructure owner | Update on account migration; remove from callers when retired. | +| `CLOUDFLARE_API_TOKEN` | Token limited to managed Cloudflare zones/projects | Required for apply/deploy; infrastructure owner | Revoke on exposure/role change; replace and dry-run before apply. | +| `GCP_SA_KEY` | Vertex provider service-account JSON | Optional provider path; security/model owner; long-lived migration risk | Revoke immediately on exposure and migrate toward workload identity/OIDC. | +| `NOEMA_GITHUB_APP_PRIVATE_KEY` | Noema App-token minting for the target repository | Optional when an approved narrower route exists; review-platform owner | Rotate App key, revoke affected installations, then run a non-destructive review probe. | +| `NOEMA_LLM_API_KEY` | Noema model endpoint only | Optional model path; review-platform owner | Rotate at provider and verify one bounded model call; no broader fallback on auth error. | +| `NOEMA_REVIEW_TOKEN` | Noema review-publication compatibility path | Optional App/OIDC fallback; review-platform owner | Revoke on exposure/eligibility change and audit review objects since exposure. | +| `NVIDIA_NIM_API_KEY` | Approved NVIDIA NIM model-call steps | Optional provider path; model-platform owner | Rotate at NVIDIA; never reuse for checkout, tests, or mutation. | +| `OPENAI_API_KEY` | Approved OpenAI-compatible model-call steps | Optional provider path; model-platform owner | Rotate at provider and verify only the provider-scoped step. | +| `OPENCODE_APPROVE_TOKEN` | Review publication and reviewed scheduler compatibility mutation | Optional App/merge-token fallback; automation owner | Revoke on exposure/scope change; audit reviews/writes and replace narrowly. | +| `OPENCODE_ZEN_API_KEY` | OpenCode Zen model-call step | Optional provider path; model-platform owner | Rotate at provider and verify one bounded model call. | +| `OPENROUTER_API_KEY` | OpenRouter model-call step | Optional provider path; model-platform owner | Rotate at provider and verify one bounded model call. | +| `PR_REVIEW_MERGE_TOKEN` | Named cross-repository scheduler/router/autofix operations | Required only where narrower App authority is unavailable; automation owner | Revoke on exposure/scope change, audit affected targets, and replace narrowly. | +| `SBOM_INVENTORY_TOKEN` | Organization repository read for SBOM aggregation | Optional App-token fallback; supply-chain owner | Revoke on exposure and prove read-only inventory scope before resuming. | +| `STRIX_GITHUB_MODELS_TOKEN` | GitHub Models call for Strix/OpenCode evidence | Optional provider token; security/model owner | Revoke on exposure; use only an independently configured provider fallback within budget. | +| `STRIX_OPENAI_API_KEY` | Strix-specific OpenAI provider call | Optional provider token; security owner | Rotate at provider and verify the Strix model boundary. | +| `VERTEX_LOCATION` | Vertex region identifier currently stored through secret context | Optional configuration; model-platform owner; migrate to non-secret configuration | Update on region/policy change; removal does not require credential rotation. | + +Each registry entry defines consumer, minimum scope, required/optional behavior, +owner, and rotation or revocation. Configuration identifiers migrate to +`vars` or another reviewed configuration store when secrecy is unnecessary. ## 4. Pull-request and workflow trust From c72c8bb3e6025803a2804a64bb00cfa368dcde16 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:00:27 +0900 Subject: [PATCH 30/69] docs(testing): narrow structural contract claims --- docs/automation/TEST_STRATEGY.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/automation/TEST_STRATEGY.md b/docs/automation/TEST_STRATEGY.md index e805ad669..aad177583 100644 --- a/docs/automation/TEST_STRATEGY.md +++ b/docs/automation/TEST_STRATEGY.md @@ -81,14 +81,17 @@ The current documentation gate verifies structural contracts: - required workflow/script names referenced by architecture exist; - data-model entities use two-or-more-word `snake_case` names and retain the selected many-observations-to-one-revision relationships; -- explicit secret contracts and the prohibition on `COPILOT_GITHUB_TOKEN` are present; -- traceability contains every PRD/TRD identifier and the exact source, test, - receipt, owner/target, and status columns; and +- the value-free registry exactly covers literal workflow secret names and the + prohibition on `COPILOT_GITHUB_TOKEN` is present; this registry does not + prove reusable `workflow_call.secrets` declarations or caller mapping; +- traceability contains every PRD/TRD identifier, locator/accountability + columns, and a controlled maturity value; repository-path resolution is + checked separately from semantic implementation proof; and - no timeless architecture depends on an unstable literal head SHA or run ID. -These tests do not prove Mermaid rendering, every ERD semantic, live GitHub -truth, standards correctness, or that a named operational receipt currently -exists. Rendering/review, live API/ruleset inspection, focused source tests, and +These tests do not prove Mermaid rendering, every ERD semantic, reusable-secret +mapping, traceability-row semantics, live GitHub truth, standards correctness, +or that a named operational receipt currently exists. Rendering/review, live API/ruleset inspection, focused source tests, and protected-main acceptance remain separate gates. The test name and PR summary must not overstate a structural assertion as runtime proof. From c3d302b32f75e1ad510b812d89301061679560bd Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:02:07 +0900 Subject: [PATCH 31/69] docs(traceability): use precise locators and live gap lineage --- docs/automation/TRACEABILITY.md | 53 ++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md index 8fea40ae8..49a93cc78 100644 --- a/docs/automation/TRACEABILITY.md +++ b/docs/automation/TRACEABILITY.md @@ -3,28 +3,29 @@ Status: living authoritative matrix Last reviewed: 2026-08-09 -Canonical maturity states are defined in [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md): `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, and `out_of_scope`. Descriptive qualifiers may follow one of these states, but must not replace the canonical state. A row marked `implemented_on_protected_main` still requires fresh exact-head evidence for each later change; a row marked `active_pr` is never shipped truth. +Canonical maturity states are defined in [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md): `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, and `out_of_scope`. Every maturity cell contains exactly one canonical state; explanatory qualifiers belong in the evidence or closure columns. A row marked `implemented_on_protected_main` still requires fresh exact-head evidence for each later change; a row marked `active_pr` is never shipped truth. ## 1. Product and technical requirements Receipt names describe the required evidence shape; they are not claims that a particular transient run is current. A mutation must re-fetch the live receipt. +Repository locators below are literal paths with no globs. External canaries and live API receipts are explicitly labeled rather than misrepresented as repository files. -| Requirement | TRD / ADR | Exact implementation source | Exact tests | Required gate / receipt authority | Owner / closure target | Status | +| Requirement | TRD / ADR | Implementation/source locator | Test/evidence locator | Required gate / receipt authority | Owner / closure target | Maturity | |---|---|---|---|---|---|---| | `PRD-01` fresh live state | `TRD-EVT-01`, ADR-0007 | `scripts/ci/pr_review_merge_scheduler.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_agent_mention_sweep.py` | `scan-pr-queue` plus live PR/ruleset API snapshot | automation maintainer / every decision | `implemented_on_protected_main` | -| `PRD-02` exact head + live base | `TRD-REV-01`, ADR-0002/0014 | `scripts/ci/pr_head_replay_guard.py`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_head_replay_guard.py`; `tests/test_opencode_review_context.py`; `tests/test_pr_review_fix_scheduler.py` | exact-head Check Runs plus independently resolved current base and expected-head mutation response | workflow owners / before every write | `accepted_architecture` — implementation exists but path inventory and current-live-base receipts remain distributed | -| `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | `implemented_on_protected_main` with live-ruleset dependency | -| `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; merge scheduler policy | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_scheduler_independent_approval_gate.py` | live review decision, qualifying non-author formal review and ruleset | repository admins / before merge | `active_pr` for the strengthened exact-head scheduler gate; existing ruleset remains live authority | -| `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml`; external maintainer lease policy | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | `accepted_architecture` — no shared persisted lease service claimed | -| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; external hourly prompt contract | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; external continuation receipt | 15-minute queue sweep plus double fresh whole-queue exit sweep | automation operator / before run exit | `accepted_architecture` — no-soft-timeout/meta-intermediate contract is in this active documentation PR | -| `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py`; documentation contracts | source-backed finding/RCA, `remediation_candidate` evidence, exact-head repair and protected recovery receipt | incident owner / before closure | `accepted_architecture` | -| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; focused redaction contracts | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / clean-history replacement #888 + protected consumer canary | `active_pr` | -| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/**`; root `ARCHITECTURE.md`; repository entrypoints | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | -| `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/**`; `docs/automation/RUNBOOK.md` receipt template | workflow-specific contracts and real consumer canary | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | `accepted_architecture` — receipts remain distributed | -| `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | required workflows; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; workflow contract suites | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | `accepted_architecture` — implicit-v1 dispatch and inherited-secret legacy remain | -| `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | `implemented_on_protected_main` for existing provider boundaries; new routes require audit | -| `PRD-13` PII alternative controls | `TRD-SEC-01`, `TRD-RET-01` | `docs/automation/SECURITY.md`; workflow access and artifact retention settings | privacy/access review; documentation/security contracts | purpose/audience/retention/deletion/access receipt | data/service owner / before processing business PII | `accepted_architecture` — service-specific deployment evidence remains external | -| `PRD-14` quality/readability | verification §12, ADR-0012 | `pyproject.toml`; pinned requirement/hash manifests; owned Python/workflow source | complete `tests/`; compile/syntax; coverage; interrogate | exact-head tests, 100% owned production statement/branch and public docstrings | change author / before PR readiness | `implemented_on_protected_main` as policy; every head requires fresh proof | +| `PRD-02` exact head + live base | `TRD-REV-01`, ADR-0002/0014 | `scripts/ci/pr_head_replay_guard.py`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/pr-review-autofix.yml` | `tests/test_pr_head_replay_guard.py`; `tests/test_opencode_review_context.py`; `tests/test_pr_review_fix_scheduler.py` | exact-head Check Runs plus independently resolved current base and expected-head mutation response | workflow owners / before every write | `accepted_architecture` | +| `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | `implemented_on_protected_main` | +| `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_scheduler_independent_approval_gate.py` | live review decision, qualifying non-author formal review and ruleset | repository admins / before merge | `active_pr` | +| `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | `accepted_architecture` | +| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; `AGENTS.md`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; external continuation receipt | 15-minute queue sweep plus double fresh whole-queue exit sweep | automation operator / before run exit | `accepted_architecture` | +| `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_automation_documentation_contract.py` | source-backed finding/RCA, `remediation_candidate` evidence, exact-head repair and protected recovery receipt | incident owner / before closure | `accepted_architecture` | +| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / clean-history replacement #888 + protected consumer canary | `active_pr` | +| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/DOCUMENTATION_AUDIT.md`; `docs/automation/README.md`; `docs/automation/EVENT_CONTRACTS.md`; `ARCHITECTURE.md` | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | +| `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/opencode-review.yml`; `.github/workflows/pr-review-merge-scheduler.yml`; `docs/automation/RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_pr_review_merge_scheduler.py`; real-consumer canary receipt | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | `accepted_architecture` | +| `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | `.github/workflows/opencode-review.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml`; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_automation_documentation_contract.py` | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | `accepted_architecture` | +| `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | `implemented_on_protected_main` | +| `PRD-13` PII alternative controls | `TRD-SEC-01`, `TRD-RET-01` | `docs/automation/SECURITY.md`; `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml` | `tests/test_opencode_agent_contract.py`; `tests/test_automation_documentation_contract.py`; service privacy/access review | purpose/audience/retention/deletion/access receipt | data/service owner / before processing business PII | `accepted_architecture` | +| `PRD-14` quality/readability | verification §12, ADR-0012 | `pyproject.toml`; `requirements-opencode-review-ci.txt`; `requirements-opencode-review-ci-hashes.txt`; `.github/workflows/trusted-uv-materializer-quality-ci.yml` | `tests/test_repository_branch_coverage_execution_sandboxes.py`; `tests/test_trusted_uv_materializer_quality_workflow_contract.py`; compile/coverage/interrogate receipts | exact-head tests, 100% owned production statement/branch and public docstrings | change author / before PR readiness | `implemented_on_protected_main` | ## 2. Documentation coverage @@ -46,7 +47,7 @@ Receipt names describe the required evidence shape; they are not claims that a p - `ContextualWisdomLab/.github#841` is closed/unmerged historical origin evidence. It established the disclosure concern but mixed unrelated scope; it is `superseded` as an integration path. - `ContextualWisdomLab/.github#842` is closed/unmerged historical RED→GREEN and exhaustive-boundary evidence. Its final blobs removed current-source defects but its reachable PR history retained secret-shaped test fixtures that kept Secret Scan red; it is `superseded` as an integration path. -- `ContextualWisdomLab/.github#888` is the `active_pr` clean-history replacement from protected main. It carries the final ten #842 blobs without the secret-shaped predecessor history. Exact-head Sandbox Log Redaction, Secret Scan, Strix, Security Scan, SAST, OSV, Scorecard, SBOM and CodeQL evidence were successful at the latest observation; any queued gate remains non-passing until complete. +- `ContextualWisdomLab/.github#888` is the `active_pr` clean-history replacement from protected main. It carries the final ten #842 blobs without the secret-shaped predecessor history. Its exact-head checks, reviews, and threads must be re-fetched at every decision; predecessor or queued evidence is non-authorizing. - Output-memory and service-file quotas remain a separate `planned` resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. - Every #888 head change invalidates predecessor test counts, coverage counts, reviews, and run IDs. Protected-main/consumer synthetic credential canaries are required before operational closure. @@ -67,13 +68,29 @@ Receipt names describe the required evidence shape; they are not claims that a p | No persisted cross-repository continuation/writer ledger | collision and queue-starvation reconstruction | decide whether GitHub Project/artifact state is sufficient before proposing persistence | `planned` | | SLI aggregation incomplete | buyer cannot quantify reliability | emit bounded run metrics keyed by repo/head/workflow/attempt | `planned` | | Sandbox total-output/service-file quota incomplete | memory/disk DoS | fail-first hostile-output quota tests and separate implementation PR | `planned` | -| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | explicit secret interface plus real deployment positive/negative canary | `planned` | +| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | [PR #901](https://github.com/ContextualWisdomLab/.github/pull/901) explicit interface plus real deployment positive/negative canary | `active_pr` | +| Undeclared scheduler/rebase reusable secrets | caller ambiguity and excess exposure | declare and map secrets in the three exact workflows registered by `SECURITY.md`; add negative caller tests and consumer receipts | `planned` | | Operational receipts distributed | incident closure hard to audit | dated protected-main/consumer receipt index | `planned` | | Project #1 requires GraphQL/`gh` project scope | agents without that capability cannot acquire visible project item | add supported connector or ensure native PR auto-add; never invent state | `planned` | | Master context current-state section is dated | stale operational narrative | separate timeless context from generated/daily live-state appendix | `planned` | | Explicit dispatch/result schema versions not universal | compatibility ambiguity | version all central dispatch/result contracts with migration/negative tests | `planned` | -## 6. Standards traceability +## 6. Audited implementation-gap lineage + +An issue records planned work; it does not implement the work it describes. + +| Gap | Live object | Current boundary and required closure | Maturity | +|---|---|---|---| +| `IG-001` dispatch snapshot preservation | [PR #840](https://github.com/ContextualWisdomLab/.github/pull/840) | The active PR proposes an end-to-end versioned envelope, live-base binding, and review-only route; protected-main consumer acceptance remains required. | `active_pr` | +| `IG-002` counted independent reviewer | [Issue #772](https://github.com/ContextualWisdomLab/.github/issues/772) | Tracks planned implementation of a non-author human reviewer path that GitHub counts. | `planned` | +| `IG-003` external-head convergence | [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889) | Tracks planned implementation of one safe review-or-reject contract across privileged entrypoints. | `planned` | +| `IG-004` cross-workflow writer fencing | [Issue #890](https://github.com/ContextualWisdomLab/.github/issues/890) | Tracks planned implementation of a shared repository/branch lease, TTL, heartbeat, takeover, and fencing. | `planned` | +| `IG-005` authoritative Strix result | [Issue #891](https://github.com/ContextualWisdomLab/.github/issues/891) | Tracks planned implementation of a fail-closed terminal gate when authoritative scan evidence is absent. | `planned` | +| `IG-006` merge mode and mutation authority | [Issue #892](https://github.com/ContextualWisdomLab/.github/issues/892) | Tracks planned implementation of one executable credential/mode authority table. | `planned` | +| `IG-007` recoverable mention claim | [Issue #893](https://github.com/ContextualWisdomLab/.github/issues/893) | Tracks planned recoverable claim states and fencing without weakening completed-request idempotency. | `planned` | +| `IG-008` truthful scheduler terminal result | [Issue #894](https://github.com/ContextualWisdomLab/.github/issues/894) | Tracks a non-passing terminal result after preserving the queue summary when a material action is `action_error`. | `planned` | + +## 7. Standards traceability Research status, normative version choices, applied implications, and APA 7 references are maintained in [`../doctoring/automation-control-plane-standards.md`](../doctoring/automation-control-plane-standards.md). The current baseline explicitly distinguishes: From 6ec904f29d91f907c8df9245543c23b210310fde Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:02:37 +0900 Subject: [PATCH 32/69] docs(trd): register audited implementation gaps --- docs/automation/TRD.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/automation/TRD.md b/docs/automation/TRD.md index cf58af56d..edc97e7f9 100644 --- a/docs/automation/TRD.md +++ b/docs/automation/TRD.md @@ -225,6 +225,26 @@ comment is user feedback, not the authoritative receipt. Raw PR source, logs, and artifacts MUST use the shortest retention compatible with review and incident needs. Evidence stores SHOULD retain hashes, classifications, decisions, and bounded diagnostic excerpts rather than unlimited raw content. Access to unmasked business PII MUST be purpose-bound and auditable. Credentials MUST never be retained as evidence. -## 12. Verification +## 12. Audited implementation state and gaps + +The stronger contracts above are not all protected-main behavior. These gaps +stay non-passing wherever their target contract is required; their live object +type and maturity are mapped in [TRACEABILITY.md](TRACEABILITY.md). + +| ID | Audited protected-main behavior | Required closure | +|---|---|---| +| `IG-001` | Mention wrappers bind a claim, but the complete downstream review path does not yet preserve every versioned field, live-base observation, and review-only policy through authoritative publication. | Integrate and accept [PR #840](https://github.com/ContextualWisdomLab/.github/pull/840). | +| `IG-002` | Bot/model identities remain advisory; a counted qualifying independent non-author human approval is still externally required. | Establish the governance path in [Issue #772](https://github.com/ContextualWisdomLab/.github/issues/772). | +| `IG-003` | Privileged targeted scheduler, OpenCode, and Strix paths reject external heads while older policy prose described them as reviewable. | Align every entrypoint under [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889). | +| `IG-004` | Per-workflow concurrency, invocation claims, and expected-head guards exist, but there is no shared owner/TTL/heartbeat/fencing record across branch-mutating workflows. | Implement [Issue #890](https://github.com/ContextualWisdomLab/.github/issues/890). | +| `IG-005` | A provider/backend outage can leave Strix without authoritative exact-head scan evidence while a transport/check shape may be skipped or neutral. | Add the fail-closed gate in [Issue #891](https://github.com/ContextualWisdomLab/.github/issues/891). | +| `IG-006` | Workflow-token preference coexists with App/token fallbacks, and merge-mode/credential authority is not one executable table. | Align authority under [Issue #892](https://github.com/ContextualWisdomLab/.github/issues/892). | +| `IG-007` | A post-claim pre-forward mention failure is an at-most-once dead-letter during retention and needs a new trusted comment. | Add recoverable states/fencing under [Issue #893](https://github.com/ContextualWisdomLab/.github/issues/893). | +| `IG-008` | The scheduler records PR-local mutation failure as `action_error` and continues, but the CLI can still return success after the scan. | Preserve the summary and return non-passing under [Issue #894](https://github.com/ContextualWisdomLab/.github/issues/894). | + +An accepted ADR, diagram, prompt, issue, or this register cannot promote these +gaps to `implemented_on_protected_main`. + +## 13. Verification The minimum verification classes are defined in [TEST_STRATEGY.md](TEST_STRATEGY.md). Requirement-to-implementation status is maintained in [TRACEABILITY.md](TRACEABILITY.md); documentation presence is not proof of implementation. From 5074de4f069abbe867fbee464c806e45ea81ebfb Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:02 +0900 Subject: [PATCH 33/69] docs(mentions): disclose snapshot and dead-letter limits --- docs/automation/review-agent-comment-invocation.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/automation/review-agent-comment-invocation.md b/docs/automation/review-agent-comment-invocation.md index 51c84dcde..b7cb5a71e 100644 --- a/docs/automation/review-agent-comment-invocation.md +++ b/docs/automation/review-agent-comment-invocation.md @@ -11,6 +11,16 @@ Trusted ContextualWisdomLab maintainers can invoke the existing review planes fr The router never checks out or executes pull-request-controlled code. It reads live PR metadata, binds the request to the current head SHA and base branch, and dispatches the already deployed central workflows in `ContextualWisdomLab/.github`. +### Audited end-to-end limitation + +The current protected router/wrapper claim binds dispatch identity, but it is +not yet proof that every field reached the final review publisher or that the +authoritative downstream execution remained review-only. The strict versioned +snapshot/live-base/review-only route is proposed by [PR #840](https://github.com/ContextualWisdomLab/.github/pull/840). +Until it integrates and passes protected-main consumer acceptance, an +acknowledgement or immutable claim is **dispatch evidence**, not final-review +snapshot evidence. + ## Architecture GitHub organization ruleset workflows support `pull_request`, `pull_request_target`, and `merge_group`, but not `issue_comment`. Separately, an `issue_comment` workflow runs only when that workflow file exists on the commented repository's default branch. Therefore, a workflow stored only in the central `.github` repository cannot directly receive comments created in sibling repositories. @@ -24,7 +34,7 @@ Each requested agent receives a deterministic invocation key containing the targ The exact-name Actions artifact ledger uses `cwl-agent-invocation-` as the artifact name. The router queries GitHub's repository artifact endpoint with the server-side exact `name` filter, validates the complete response, and treats any live exact-name artifact as durable dispatch evidence. This avoids depending on filtered workflow-run enumeration, which GitHub caps at 1,000 results even when pagination is requested. -Wrapper workflows use the verified key in their non-cancelling concurrency group, inspect the exact artifact name, and upload a 30-day immutable claim before forwarding to the authoritative review plane. Exact-key concurrency serializes duplicate wrapper runs. If a prior live claim exists, the wrapper performs no forward. If artifact visibility is delayed and a duplicate upload collides, the upload fails before the forwarding step, so the control plane fails closed rather than forwarding twice. Completed or failed authoritative work remains claimed for the retention window; a maintainer who needs a new attempt creates a new trusted source comment, which produces a distinct key. +Wrapper workflows use the verified key in their non-cancelling concurrency group, inspect the exact artifact name, and upload a 30-day immutable claim before forwarding to the authoritative review plane. Exact-key concurrency serializes duplicate wrapper runs. If a prior live claim exists, the wrapper performs no forward. If artifact visibility is delayed and a duplicate upload collides, the upload fails before the forwarding step, so the control plane fails closed rather than forwarding twice. Completed or failed authoritative work remains claimed for the retention window. A post-claim failure before successful forward is currently an **at-most-once dead-letter**: a maintainer must create a new trusted source comment. Recoverable reserved/forwarded/completed/failed state and fencing remain planned in [Issue #893](https://github.com/ContextualWisdomLab/.github/issues/893). Target-repository acknowledgement comments and reactions are user-experience signals only. They are not dispatch authority because repository writers, bot identities, or credential rotation could otherwise forge or invalidate a marker. A failed acknowledgement cannot cause completed agent work to be redispatched. From ce597fdcb2fda5db44f2f261d20a7b8c45f61b0a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:03 +0900 Subject: [PATCH 34/69] docs(sbom): mark zero inventory as unmaterialized --- docs/sbom/inventory.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/sbom/inventory.md b/docs/sbom/inventory.md index 8892aa3b3..ebf48cc87 100644 --- a/docs/sbom/inventory.md +++ b/docs/sbom/inventory.md @@ -1,6 +1,11 @@ # Organization SBOM inventory -Generated: pending first scheduled run +Generated: no successful scheduled inventory persisted at this revision + +The zero rows below are an **unmaterialized generated-artifact state**, not +proof that the organization has zero repositories/components or no policy +findings. Require an attributable successful `SBOM Inventory Scheduler` run +before making any fleet claim. One central view of every managed repository's software components, versions, and licenses. Feeds license and vulnerability governance @@ -20,6 +25,6 @@ alongside the central Security Scan. ## Flagged components (policy violations) -No copyleft or NOASSERTION components detected. +No rows have been materialized. A license-compliance conclusion requires a successful attributable inventory run. ## Per-repository components From 45b04019da4065c9cf1e271fa6d1ff48bce0a704 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:33 +0900 Subject: [PATCH 35/69] docs(policy): align reviewer tool boundary --- README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 120f1ac8a..6c401fa46 100644 --- a/README.md +++ b/README.md @@ -132,10 +132,11 @@ that discovers runtime matrices, package manifests, test, coverage, docstring, E2E, lint, security, Docker, and unpackaged-source gaps before the agent chooses commands. The configured `code-reviewer` subagent is reviewer-only: it may read, grep, -glob, and run safe local verification commands, but it must not edit files, -stage changes, commit, push, install dependencies, mutate branches, or touch -production state. Blocking findings must be source-backed, severity-labeled, -impactful, remediable, and include suggested verification. +and glob, but it cannot invoke Bash, task/subagents, or webfetch under +`opencode.jsonc`. It must not edit files, stage changes, commit, push, install +dependencies, mutate branches, or touch production state. Blocking findings +must be source-backed, severity-labeled, impactful, remediable, and include +suggested verification for an authorized execution plane. Failed GitHub Checks are not reviewed as URL lists. OpenCode must explain the failed check name, failing step, source-backed file and line when available, From 0df7483de266a13748b56d48dbb7eb2c46b87ebc Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:35 +0900 Subject: [PATCH 36/69] docs(rollout): add current executable corrections --- docs/org-required-workflow-rollout.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/org-required-workflow-rollout.md b/docs/org-required-workflow-rollout.md index c1c88cdb6..7953df22c 100644 --- a/docs/org-required-workflow-rollout.md +++ b/docs/org-required-workflow-rollout.md @@ -9,6 +9,16 @@ Updated: 2026-07-23 06:35 KST > In particular, later entries supersede the early `~ALL` narrative with the > explicit `.github`, `IRT-bibliography-set`, and `noema` exclusions. +> **Current contract correction (2026-08-09).** Protected-main targeted +> scheduler, OpenCode, and Strix dispatch reject external heads; one safe +> review-or-reject contract is tracked by +> [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889). +> Strix uses PR/event-or-ref concurrency with `cancel-in-progress: true`, not a +> head-SHA non-cancelling group. The central autofix worker is the shared +> default; product-specific application/security workflows stay local. The +> configured `code-reviewer` may read/search only and cannot invoke Bash, +> task/subagents, or webfetch. These corrections govern current behavior. + ## Decision at the recorded rollout snapshot Use an organization repository ruleset instead of copying workflow files into each repository. @@ -50,7 +60,7 @@ The central `.github/workflows/opencode-review.yml` is now part of the active or - Model token posture: use the organization `STRIX_GITHUB_MODELS_TOKEN` secret for GitHub Models calls, with `github.token` as the fallback; live workflow evidence showed `github.token` alone can return 403 from `models.github.ai/inference` - Write posture: OpenCode may create review/comment side effects through the OpenCode app token when available; the workflow token is limited to the same-repository PR context and publication failures remain visible - Coverage execution posture: PR-controlled package, test, build, R, Rust, and Docker inputs are never executed from `pull_request_target`; the dispatch workflow runs bounded low-privilege coverage only after exact live metadata and scheduler identity validation -- Fork posture: PR heads are fetched through `refs/pull//head` when direct head-SHA fetch is not available, so review can inspect fork PR source as data without executing it in the trusted workflow context +- External-head posture: current privileged targeted dispatch rejects a head repository that differs from the base repository. Safe read-only support versus consistent rejection remains tracked by [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889). - Runtime posture: pre-model failed-check evidence waits are capped at about five minutes; the later approval gate rechecks current-head peer checks and extends its bounded wait only while image-validation checks remain pending, logging the reason before approval - Model-exhaustion posture: command exit codes and deterministic checks cannot synthesize an approval. Exhaustion remains `MODEL_OUTPUT_UNAVAILABLE`; only a prior real-model approval bound to the exact current head can satisfy the review gate after all checks, alerts, and threads are revalidated. - Adversarial-evidence posture: every probe must cite its exact changed path and positive in-range line in the materialized current-head source tree. Unrelated paths, nonexistent lines, circular claims, and missing observed results fail closed with a concrete rejection reason. @@ -134,7 +144,7 @@ The central `.github/workflows/pr-review-merge-scheduler.yml` is now part of the - Token posture: the workflow passes the first available mutation credential in this order: `PR_REVIEW_MERGE_TOKEN`, `OPENCODE_APPROVE_TOKEN`, exchanged OpenCode GitHub App token, then the target repository workflow token. The scheduler reports the non-secret token source and expected actor class in every mutation decision. - Flow posture: default branches named `main` or `master` are treated as GitHub Flow; default branches named `develop` are treated as Git Flow unless a repository explicitly sets `PROJECT_FLOW` - Merge posture: the default merge mode is `direct_or_auto`. When a current-head approved PR is same-repository and the scheduler has no failed-check, action-required, unresolved-thread, or conflict blocker, it requests an immediate guarded squash merge with `--match-head-commit`. This includes PRs where native GitHub auto-merge is already enabled; native auto-merge is a fallback queue, not the scheduler's first stop when direct merge is possible. -- Fork posture: fork or external-head PRs remain reviewable, but the scheduler does not direct-merge them and does not enable auto-merge for them. A maintainer must make the final merge decision after same-head OpenCode approval, same-head Strix evidence, required checks, and unresolved-thread checks are clean. +- External-head posture: fork or external-head PRs remain observable, but current privileged targeted scheduler/OpenCode/Strix routes reject them rather than promising central review. [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889) owns convergence. - Branch freshness posture: the scheduler also runs after protected base-branch pushes to `main`, `develop`, or `master`, because those pushes can create the GitHub UI state where reviews are satisfied, auto-merge is enabled, checks are stale or failed, and the PR shows `Update branch` without a PR `synchronize` event. - Auto-merge posture: `auto_merge_enabled` PR events trigger the scheduler so an already stale branch is refreshed immediately after native auto-merge is turned on instead of waiting for the periodic schedule. If the same PR is already mergeable, the scheduler attempts the guarded direct merge immediately. - Automation boundary: current-head failed checks and `ACTION_REQUIRED` checks are reported before branch updates, so an update attempt does not hide the concrete reason a PR cannot merge. `update-branch` handles approved `BEHIND` PRs and already queued auto-merge PRs only when there is no current-head failed or action-required check to diagnose first. `DIRTY` or `CONFLICTING` PRs still require author or maintainer conflict resolution guidance; current-head approved conflicts may keep or queue native GitHub auto-merge as a wait state while the conflict is repaired, but the scheduler must not treat queued auto-merge as a conflict resolver. @@ -308,7 +318,7 @@ non-fork inventory snapshot and rollout ledger, not the ruleset target list. - Existing open PRs may need a new push or base update before the latest required workflow SHA appears on their current head. - The central OpenCode workflow now retries DeepSeek R1, DeepSeek V3, GPT-5, and a catalog fallback pool. Keep model/tooling failures out of PR comments unless there is a source-backed failed-check diagnosis. -- The central OpenCode config includes a read-only `code-reviewer` subagent for focused review passes. The subagent may read, grep, glob, and run safe local verification commands, but it must not edit files, stage changes, commit, push, install dependencies, mutate branches, or touch production state. +- The central OpenCode config includes a read-only `code-reviewer` subagent for focused review passes. The subagent may read, grep, and glob; `opencode.jsonc` denies Bash, task/subagents, and webfetch. It must not edit files, stage changes, commit, push, install dependencies, mutate branches, or touch production state. - OpenCode execution evidence must be sandboxed in the CI workspace or an isolated temporary directory, with a credential-scrubbed environment by default and no persistent mutation outside test caches or scratch files. Prefer `python3 scripts/ci/sandboxed_verify.py --repo-root -- ` when the central helper is available, and cite its `SANDBOXED_VERIFY_RESULT` line. When repo-native verification legitimately needs network access or GitHub Secrets, pass only the needed names with `--allow-env`, record `--network required`, and explain it with `--evidence-note` without printing secret values. The helper does not replace existing bash, task, webfetch, websearch, lsp, CodeGraph, DeepWiki, Context7, or web_search review policy. If a verification cannot be sandboxed without changing the result, the review must say so instead of presenting an unsafe run as evidence. - Web application reviews should run backend, frontend, and repository-native E2E checks together through `python3 scripts/ci/sandboxed_web_e2e.py --repo-root --backend-cmd --frontend-cmd --e2e-cmd ` when those contracts exist, then cite `SANDBOXED_WEB_E2E_RESULT`. If backend/frontend/E2E/readiness contracts are missing, the review must name the gap instead of treating unit or lint evidence as full E2E proof. - Bounded OpenCode evidence includes `Review execution contracts`, which inventories runtime matrices, package manifests, test, coverage, docstring, E2E, lint, security, Docker, and unpackaged-source gaps before the model chooses verification commands. From 6b8578a356b30b513429672fdf397e9e24d7c479 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:37 +0900 Subject: [PATCH 37/69] docs(audit): separate current corrections from history --- PR_GOVERNANCE_AUDIT.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/PR_GOVERNANCE_AUDIT.md b/PR_GOVERNANCE_AUDIT.md index 78738ca89..99c9b717e 100644 --- a/PR_GOVERNANCE_AUDIT.md +++ b/PR_GOVERNANCE_AUDIT.md @@ -7,6 +7,16 @@ > [`docs/automation/README.md`](docs/automation/README.md); this file preserves > the dated fleet observation and decision history for auditability. +> **Current contract correction (2026-08-09).** Strix now uses +> PR/event-or-ref concurrency with `cancel-in-progress: true`, not a head-SHA +> non-cancelling group. Protected-main targeted scheduler, OpenCode, and Strix +> dispatch reject external heads while the contract is resolved in +> [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889). +> The central autofix worker is the shared default for `pg-erd-cloud`; the old +> repo-local recommendation is superseded. Mutation authority still includes +> an exchanged repository-scoped App-token fallback after configured tokens. +> The canonical automation documents and executable source govern. + Snapshot check: 2026-06-26 17:53 KST, GitHub API via `gh` as `seonghobae`. - Snapshot status: historical and stale for mutation decisions From f8f6edf1cdd1757800d9bf63b112d92708f2e1da Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:57 +0900 Subject: [PATCH 38/69] docs(nvidia): remove stale bypass and secret alias --- docs/nvidia-nim-opencode-hotfix.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/nvidia-nim-opencode-hotfix.md b/docs/nvidia-nim-opencode-hotfix.md index df8c193b2..858fe9160 100644 --- a/docs/nvidia-nim-opencode-hotfix.md +++ b/docs/nvidia-nim-opencode-hotfix.md @@ -1,4 +1,10 @@ -# NVIDIA NIM OpenCode model priority (hotfix) +# NVIDIA NIM OpenCode model priority (historical hotfix) + +> **Current contract correction (2026-08-09).** The only workflow secret name +> is `NVIDIA_NIM_API_KEY`; approved jobs bind it to process environment +> `NVIDIA_API_KEY` because `opencode.jsonc` reads that variable. There is no +> legacy secret-name fallback. No branch-protection or ruleset bypass is +> authorized. Protected workflow source and rulesets govern. ## Why @@ -17,22 +23,17 @@ still emit APPROVE / REQUEST_CHANGES when GitHub Models / free tiers stall. with `apiKey: {env:NVIDIA_API_KEY}` 2. `.github/workflows/opencode-review-dispatch.yml` - `OPENCODE_MODEL_CANDIDATES` prefixes six NIM models before existing pool - - binds `NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}` + - binds process `NVIDIA_API_KEY` from `${{ secrets.NVIDIA_NIM_API_KEY }}` 3. `scripts/ci/run_opencode_review_model_pool.sh` - skips `nvidia-nim/*` when `NVIDIA_API_KEY` is unset (same pattern as OpenRouter) -## Temporary permission bypass (hotfix only) - -For this merge-aid hotfix only: +## Permission and merge boundary -- Branch-protection / ruleset admin override may be used to land the central - `.github` change if required checks conflict during the hotfix window. -- **Do not** permanently weaken Security Scan, trivy-fs, osv-scan, or - CodeQL gates. -- **Do not** flip OpenCode agent `permission.edit` / `bash` from `deny` to - `allow` permanently; review agents remain read-only. -- Org secret `NVIDIA_API_KEY` must be set on ContextualWisdomLab for NIM pool - entries to execute; without it the pool falls through to prior candidates. +- No branch-protection or ruleset bypass is authorized for this provider configuration. +- Do not weaken Security Scan, trivy-fs, osv-scan, or CodeQL gates. +- Do not change reviewer edit/Bash permissions from `deny`. +- The organization secret is `NVIDIA_NIM_API_KEY`; without it, the bounded + pool continues only through independently approved configured candidates. ## Rollback @@ -42,8 +43,7 @@ catalog reliability is restored. ## Secret name -Org secret is **`NVIDIA_NIM_API_KEY`**. Workflows bind it to process env `NVIDIA_API_KEY` -(fallback: `secrets.NVIDIA_API_KEY` if present) so `opencode.jsonc` `{env:NVIDIA_API_KEY}` resolves. +Org secret is **`NVIDIA_NIM_API_KEY`**. Workflows bind it to process env `NVIDIA_API_KEY` so `opencode.jsonc` `{env:NVIDIA_API_KEY}` resolves. No legacy secret alias is accepted. ## Large-repo OpenCode timeouts (~1 hour) From d4e05696bdcc3acadde06e2b6b9dc63201ea21f6 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:03:59 +0900 Subject: [PATCH 39/69] docs(scorecard): pin exact approval count --- docs/scorecard-governance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/scorecard-governance.md b/docs/scorecard-governance.md index 6e4db65f3..0a427a8ab 100644 --- a/docs/scorecard-governance.md +++ b/docs/scorecard-governance.md @@ -16,8 +16,8 @@ or the inherited organization ruleset must require all of the following: - stale approvals to be dismissed after a push; - current-head OpenCode review evidence from the central required workflow; - code owner review coverage through CODEOWNERS-owned workflow and CI paths, - with the live organization ruleset carrying the enforceable exactly-two- - approval gate and last-pusher separation; + with the live organization ruleset requiring exactly two eligible approvals + and last-pusher separation; - review thread resolution before merge; - last-pusher approval protection; - force-push and branch deletion protection. From 01a988383e158293406b98d7c5c8b6e73de47f7c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:04:10 +0900 Subject: [PATCH 40/69] docs(agents): enforce same-invocation continuation --- AGENTS.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index c7e02404b..83819c31f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,3 +7,8 @@ For central review, security, merge, fix, sandbox, or autonomous-maintenance wor also read the authoritative [automation documentation graph](docs/automation/README.md) and update its traceability/ADR entries whenever an authority, trust, failure, secret, evidence, compatibility, or operational-acceptance boundary changes. +Treat a prompt or documentation update, Draft/Ready transition, auto-merge or +review request, defer decision, merge, and status report as intermediate work: +return to live queue selection in the same invocation. Ordinary termination +requires two fresh whole-queue no-work sweeps; real budget exhaustion records a +non-clean `budget_exhausted` continuation handoff. From b700ff0a8264824bc12a8efbd6e0d32d3a6db8f1 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:04:36 +0900 Subject: [PATCH 41/69] docs(audit): reconcile donors and executable drift --- docs/automation/DOCUMENTATION_AUDIT.md | 36 ++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md index 09588a971..3c3ba9155 100644 --- a/docs/automation/DOCUMENTATION_AUDIT.md +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -62,6 +62,42 @@ This table assesses the **active PR documentation baseline**, not protected-main | Traceability | ADEQUATE | Controlled maturity, current redaction lineage, whole-conversation governance, standards, implementation/test/gate ownership and explicit product/control-plane debt are mapped. | | Standards doctoring | ADEQUATE | Current final baselines include SLSA 1.2, ISO/IEC/IEEE 42010:2022, ISO/IEC/IEEE 29148:2018, ISO/IEC 25010:2023 and NIST SSDF 1.1; newer drafts are labelled informative/non-normative until final. | +## Concurrent documentation-line reconciliation + +- [PR #886](https://github.com/ContextualWisdomLab/.github/pull/886) is closed + unmerged and `superseded` as a change line. Its controlled maturity, + same-invocation continuation, double-sweep, and conversation ownership + contracts are retained here. +- [PR #898](https://github.com/ContextualWisdomLab/.github/pull/898) is closed + unmerged and `superseded` as a change line. Its value-free secret registry, + executable-invariant checks, implementation-gap register, mention-claim + honesty, and SBOM zero-state warning are retained here. +- [PR #896](https://github.com/ContextualWisdomLab/.github/pull/896) is the + active canonical candidate because it contains the broader indexed graph, + versioned event contract, logical model, runbooks, standards, and sixteen + ADRs. Closed donor histories remain preserved; force-push or deletion is not + reconciliation. + +## Current executable-invariant correction ledger + +A file's existence alone is not adequate documentation. These reproduced +source/prose contradictions are corrected or explicitly classified as debt: + +| Invariant | Executable observation | Canonical correction | +|---|---|---| +| Reviewer tools | `opencode.jsonc` denies Bash, task/subagents, and webfetch to `code-reviewer`. | Root README and rollout no longer promise local command execution. | +| External heads | Privileged targeted scheduler, OpenCode, and Strix routes reject a different head repository. | Current correction banners name [Issue #889](https://github.com/ContextualWisdomLab/.github/issues/889). | +| Strix concurrency | PR/event-or-ref grouping uses `cancel-in-progress: true`. | Current README/rollout/audit corrections match executable YAML. | +| Autofix owner | `pg-erd-cloud` uses the central autofix worker as shared default. | Historical repo-local recommendations are marked superseded. | +| NVIDIA secret | Workflow secret `NVIDIA_NIM_API_KEY` maps to process `NVIDIA_API_KEY`; there is no legacy alias or routine ruleset bypass. | The historical hotfix note now names the current fail-closed contract. | +| Reusable secrets | Four protected-main workflows consume undeclared reusable secrets. | `SECURITY.md` registers the debt; deploy repair is active in PR #901 and the other three remain planned. | +| SBOM zero state | No successful inventory receipt means no fleet population was materialized. | Zero rows cannot prove zero components or no license findings. | + +Runtime gaps `IG-001` through `IG-008` are registered in +[TRD.md](TRD.md) and mapped to the exact active PR or planned issue in +[TRACEABILITY.md](TRACEABILITY.md). Their presence is documented debt, not +implementation. + ## No-soft-timeout continuation invariant The hourly recurrence is a continuation mechanism after genuine practical execution/tool-budget exhaustion. It is not a voluntary wall-clock timeout. From b2ac52c1f0f7822d91a56dd993c5380022e2d284 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:09:40 +0900 Subject: [PATCH 42/69] test(docs): enforce current automation contracts --- .../test_automation_documentation_contract.py | 230 +++++++++++++++++- 1 file changed, 226 insertions(+), 4 deletions(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 5dfd52567..1927507c2 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -195,7 +195,10 @@ def test_conceptual_erd_uses_explicit_two_word_snake_case_entities() -> None: assert "automation_run }o--o| source_revision" in data_model assert "merge_revision ||--o{ operational_acceptance" in data_model assert "source_revision ||--o{ operational_acceptance" not in data_model - assert "orchestration_run ||--o{ automation_run" in data_model + assert "orchestration_run o|--o{ automation_run" in data_model + assert "source_revision o|--o{ workflow_evidence" in data_model + assert "standalone automation_run" in data_model + assert "Runtime receipt and issuer aliases" in data_model assert "string source_kind" in data_model @@ -375,11 +378,11 @@ def test_traceability_has_exact_identifiers_and_accountability_columns() -> None ): assert requirement_id in traceability for heading in ( - "Exact implementation source", - "Exact tests", + "Implementation/source locator", + "Test/evidence locator", "Required gate / receipt authority", "Owner / closure target", - "Status", + "Maturity", ): assert heading in traceability @@ -435,3 +438,222 @@ def test_timeless_architecture_does_not_embed_transient_run_identity() -> None: ) assert re.search(r"\b[0-9a-f]{40}\b", stable_corpus) is None assert re.search(r"\brun(?:_id)?[ =`]\d{8,}\b", stable_corpus, re.IGNORECASE) is None + + +def test_workflow_secret_registry_is_exact_and_debt_is_explicit() -> None: + """The value-free registry equals workflow references and names known debt.""" + + security = read_document("SECURITY.md") + registry = security.split("### 3.2", 1)[1].split("## 4", 1)[0] + registered = set( + re.findall(r"^| `([A-Z][A-Z0-9_]*)` |", registry, re.MULTILINE) + ) + # The delimiters are escaped explicitly below because Markdown pipes are + # regex alternation operators. + registered = set( + re.findall(r"^\| `([A-Z][A-Z0-9_]*)` \|", registry, re.MULTILINE) + ) + reference_pattern = re.compile( + r"""secrets(?:\.([A-Z][A-Z0-9_]*)|\[\s*["']([A-Z][A-Z0-9_]*)["']\s*\])""" + ) + observed: set[str] = set() + workflow_root = REPOSITORY_ROOT / ".github" / "workflows" + workflow_paths = tuple(workflow_root.rglob("*.yml")) + tuple( + workflow_root.rglob("*.yaml") + ) + for workflow_path in workflow_paths: + workflow = workflow_path.read_text(encoding="utf-8") + for dot_name, bracket_name in reference_pattern.findall(workflow): + observed.add(dot_name or bracket_name) + + expected_registry = { + "CLOUDFLARE_ACCOUNT_ID", + "CLOUDFLARE_API_TOKEN", + "GCP_SA_KEY", + "NOEMA_GITHUB_APP_PRIVATE_KEY", + "NOEMA_LLM_API_KEY", + "NOEMA_REVIEW_TOKEN", + "NVIDIA_NIM_API_KEY", + "OPENAI_API_KEY", + "OPENCODE_APPROVE_TOKEN", + "OPENCODE_ZEN_API_KEY", + "OPENROUTER_API_KEY", + "PR_REVIEW_MERGE_TOKEN", + "SBOM_INVENTORY_TOKEN", + "STRIX_GITHUB_MODELS_TOKEN", + "STRIX_OPENAI_API_KEY", + "VERTEX_LOCATION", + } + assert registered == expected_registry + assert observed == registered + + for known_debt in ( + ".github/workflows/pr-review-fix-scheduler.yml", + ".github/workflows/pr-review-merge-scheduler.yml", + ".github/workflows/pr-auto-rebase.yml", + ".github/workflows/deploy-pages.yml", + ): + assert known_debt in security + assert "registry proves names" in security + assert "not that every caller mapping is already explicit" in security + + +def test_current_lineage_threat_maturity_and_gap_objects_are_exact() -> None: + """Lineage and maturity stay bound to current PRs/issues, not stale claims.""" + + threat_model = read_document("THREAT_MODEL.md") + redaction_adr = read_document( + "adr/0009-sandbox-evidence-redaction-boundary.md" + ) + for document in (threat_model, redaction_adr): + assert "#842" not in document + assert "/pull/842" not in document + assert "https://github.com/ContextualWisdomLab/.github/pull/888" in document + + traceability_section = threat_model.split( + "## 7. Exact source and test traceability", 1 + )[1] + observed_threat_maturity = dict( + re.findall( + r"^\| (TM-\d{2}) \|[^\n]*\| `([a-z_]+)` \|$", + traceability_section, + re.MULTILINE, + ) + ) + assert observed_threat_maturity == { + "TM-01": "active_pr", + "TM-02": "accepted_architecture", + "TM-03": "active_pr", + "TM-04": "implemented_on_protected_main", + "TM-05": "implemented_on_protected_main", + "TM-06": "implemented_on_protected_main", + "TM-07": "implemented_on_protected_main", + "TM-08": "implemented_on_protected_main", + "TM-09": "accepted_architecture", + "TM-10": "active_pr", + "TM-11": "planned", + "TM-12": "accepted_architecture", + "TM-13": "accepted_architecture", + "TM-14": "implemented_on_protected_main", + "TM-15": "accepted_architecture", + } + + trd = read_document("TRD.md") + traceability = read_document("TRACEABILITY.md") + gap_objects = { + "IG-001": ( + "https://github.com/ContextualWisdomLab/.github/pull/840", + "active_pr", + ), + "IG-002": ( + "https://github.com/ContextualWisdomLab/.github/issues/772", + "planned", + ), + "IG-003": ( + "https://github.com/ContextualWisdomLab/.github/issues/889", + "planned", + ), + "IG-004": ( + "https://github.com/ContextualWisdomLab/.github/issues/890", + "planned", + ), + "IG-005": ( + "https://github.com/ContextualWisdomLab/.github/issues/891", + "planned", + ), + "IG-006": ( + "https://github.com/ContextualWisdomLab/.github/issues/892", + "planned", + ), + "IG-007": ( + "https://github.com/ContextualWisdomLab/.github/issues/893", + "planned", + ), + "IG-008": ( + "https://github.com/ContextualWisdomLab/.github/issues/894", + "planned", + ), + } + traceability_lines = traceability.splitlines() + for gap_id, (object_url, maturity) in gap_objects.items(): + assert gap_id in trd + assert object_url in trd + gap_line = next( + line for line in traceability_lines if line.startswith(f"| `{gap_id}`") + ) + assert object_url in gap_line + assert gap_line.endswith(f"| `{maturity}` |") + + +def test_current_executable_policy_corrections_are_machine_checked() -> None: + """Historical ledgers cannot silently override current executable policy.""" + + root_readme = (REPOSITORY_ROOT / "README.md").read_text(encoding="utf-8") + rollout = (REPOSITORY_ROOT / "docs/org-required-workflow-rollout.md").read_text( + encoding="utf-8" + ) + historical_audit = (REPOSITORY_ROOT / "PR_GOVERNANCE_AUDIT.md").read_text( + encoding="utf-8" + ) + nvidia_note = ( + REPOSITORY_ROOT / "docs/nvidia-nim-opencode-hotfix.md" + ).read_text(encoding="utf-8") + scorecard = (REPOSITORY_ROOT / "docs/scorecard-governance.md").read_text( + encoding="utf-8" + ) + ruleset_audit = ( + REPOSITORY_ROOT / "scripts/ci/audit_central_required_workflows.py" + ).read_text(encoding="utf-8") + + assert "exactly two approving reviews" in root_readme + assert "cannot invoke Bash, task/subagents, or webfetch" in root_readme + for correction_ledger in (rollout, historical_audit): + assert "cancel-in-progress: true" in correction_ledger + assert "https://github.com/ContextualWisdomLab/.github/issues/889" in ( + correction_ledger + ) + assert "central autofix worker" in correction_ledger + + assert "NVIDIA_NIM_API_KEY" in nvidia_note + assert "No legacy secret alias is accepted" in nvidia_note + assert "No branch-protection or ruleset bypass is authorized" in nvidia_note + assert "exactly two eligible approvals" in scorecard + assert 'EXPECTED_EXCLUSIONS = {".github", "IRT-bibliography-set", "noema"}' in ( + ruleset_audit + ) + assert "if approving_reviews != 2:" in ruleset_audit + + mention_contract = read_document("review-agent-comment-invocation.md") + assert "dispatch evidence" in mention_contract + assert "at-most-once dead-letter" in mention_contract + assert "https://github.com/ContextualWisdomLab/.github/issues/893" in ( + mention_contract + ) + sbom_inventory = (REPOSITORY_ROOT / "docs/sbom/inventory.md").read_text( + encoding="utf-8" + ) + assert "unmaterialized generated-artifact state" in sbom_inventory + assert "not proof that the organization has zero" in sbom_inventory + + +def test_documentation_changes_trigger_full_quality_ci() -> None: + """The documentation graph has a permanent exact-head full-suite gate.""" + + workflow = ( + REPOSITORY_ROOT + / ".github/workflows/automation-documentation-quality-ci.yml" + ).read_text(encoding="utf-8") + assert "name: Automation Documentation Quality CI" in workflow + for watched_path in ( + '"docs/**"', + '"*.md"', + '"tests/test_automation_documentation_contract.py"', + '"tests/test_pr_governance_audit_contract.py"', + '"requirements-opencode-review-ci-hashes.txt"', + ): + assert watched_path in workflow + assert "github.event.pull_request.head.sha || github.sha" in workflow + assert "persist-credentials: false" in workflow + assert "python -m coverage run -m pytest tests -q" in workflow + assert "python -m coverage report --fail-under=100" in workflow + assert "python -m compileall -q tests" in workflow From cde2a0cd437723e0a0a8f5f31ea01758fd5da91f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:10:13 +0900 Subject: [PATCH 43/69] test(docs): align secret debt assertions --- tests/test_automation_documentation_contract.py | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 1927507c2..98e1522e9 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -445,11 +445,6 @@ def test_workflow_secret_registry_is_exact_and_debt_is_explicit() -> None: security = read_document("SECURITY.md") registry = security.split("### 3.2", 1)[1].split("## 4", 1)[0] - registered = set( - re.findall(r"^| `([A-Z][A-Z0-9_]*)` |", registry, re.MULTILINE) - ) - # The delimiters are escaped explicitly below because Markdown pipes are - # regex alternation operators. registered = set( re.findall(r"^\| `([A-Z][A-Z0-9_]*)` \|", registry, re.MULTILINE) ) @@ -494,8 +489,8 @@ def test_workflow_secret_registry_is_exact_and_debt_is_explicit() -> None: ".github/workflows/deploy-pages.yml", ): assert known_debt in security - assert "registry proves names" in security - assert "not that every caller mapping is already explicit" in security + assert "registry proves name inventory, not reusable-call mapping" in security + assert "target contract, not a claim that every protected-main path" in security def test_current_lineage_threat_maturity_and_gap_objects_are_exact() -> None: From 51dbf9cc89f33c0118a40e74bd1f6da2f7ee5ffd Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:10:37 +0900 Subject: [PATCH 44/69] ci(docs): add permanent full-quality gate --- .../automation-documentation-quality-ci.yml | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 .github/workflows/automation-documentation-quality-ci.yml diff --git a/.github/workflows/automation-documentation-quality-ci.yml b/.github/workflows/automation-documentation-quality-ci.yml new file mode 100644 index 000000000..4dbedbc2b --- /dev/null +++ b/.github/workflows/automation-documentation-quality-ci.yml @@ -0,0 +1,74 @@ +name: Automation Documentation Quality CI + +on: + pull_request: + branches: [main] + paths: + - ".github/workflows/**" + - "docs/**" + - "*.md" + - "scripts/ci/**" + - "tests/**" + - "requirements-opencode-review-ci-hashes.txt" + - "pyproject.toml" + push: + branches: [main] + paths: + - ".github/workflows/**" + - "docs/**" + - "*.md" + - "scripts/ci/**" + - "tests/**" + - "requirements-opencode-review-ci-hashes.txt" + - "pyproject.toml" + +concurrency: + group: automation-documentation-quality-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true + PYTHONDONTWRITEBYTECODE: "1" + +jobs: + docs-quality: + name: Full suite and documentation contracts + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - name: Harden runner + uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 + with: + egress-policy: audit + + - name: Checkout exact event revision + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 1 + persist-credentials: false + + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.14" + cache: pip + cache-dependency-path: requirements-opencode-review-ci-hashes.txt + + - name: Install hash-pinned test dependencies + run: >- + python -m pip install --disable-pip-version-check + --require-hashes + -r requirements-opencode-review-ci-hashes.txt + + - name: Run full repository suite with permanent coverage gate + run: | + python -m coverage erase + python -m coverage run -m pytest tests -q + python -m coverage report --fail-under=100 + + - name: Compile test contracts + run: python -m compileall -q tests From f7ac19478553fde5f1215478030b479b95456018 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:13:38 +0900 Subject: [PATCH 45/69] docs(standards): state certification boundary explicitly --- docs/doctoring/automation-control-plane-standards.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/doctoring/automation-control-plane-standards.md b/docs/doctoring/automation-control-plane-standards.md index c17466533..a63120d29 100644 --- a/docs/doctoring/automation-control-plane-standards.md +++ b/docs/doctoring/automation-control-plane-standards.md @@ -42,6 +42,8 @@ Scope: architecture, requirements engineering, product quality, secure developme ## Alignment and certification limits +Reference alignment does not establish certification, formal conformance, or independent assurance. + - NIST publications and SLSA concepts are engineering guidance here. Their citation does not establish regulatory compliance, a SLSA level, or independent assurance. - ISO/IEC/IEEE 42010, ISO/IEC/IEEE 29148, and ISO/IEC 25010 provide architecture, requirements, and quality-model baselines. Repository documentation may be aligned to their concepts, but no formal conformance assessment is claimed. - ISO/IEC 27002 provides control guidance and cannot itself be certified. ISO/IEC 27001 or ISO/IEC 42001 certification would require a defined management-system scope, implemented controls, evidence over time, and an authorized independent certification process. From 641ef4ef12751471813b38a93296d543ee281ec9 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:13:39 +0900 Subject: [PATCH 46/69] test(docs): accept semantic coverage paths and wrapped prose --- tests/test_automation_documentation_contract.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 98e1522e9..1ee6128a8 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -628,7 +628,7 @@ def test_current_executable_policy_corrections_are_machine_checked() -> None: encoding="utf-8" ) assert "unmaterialized generated-artifact state" in sbom_inventory - assert "not proof that the organization has zero" in sbom_inventory + assert re.search(r"not\\s+proof that the organization has zero", sbom_inventory) def test_documentation_changes_trigger_full_quality_ci() -> None: @@ -642,8 +642,7 @@ def test_documentation_changes_trigger_full_quality_ci() -> None: for watched_path in ( '"docs/**"', '"*.md"', - '"tests/test_automation_documentation_contract.py"', - '"tests/test_pr_governance_audit_contract.py"', + '"tests/**"', '"requirements-opencode-review-ci-hashes.txt"', ): assert watched_path in workflow From aed69a1f2051ae10fde3e39cf7a1957d94233676 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 9 Aug 2026 23:14:09 +0900 Subject: [PATCH 47/69] test(docs): normalize wrapped SBOM prose --- tests/test_automation_documentation_contract.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 1ee6128a8..7b8eb49e6 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -628,7 +628,7 @@ def test_current_executable_policy_corrections_are_machine_checked() -> None: encoding="utf-8" ) assert "unmaterialized generated-artifact state" in sbom_inventory - assert re.search(r"not\\s+proof that the organization has zero", sbom_inventory) + assert "not proof that the organization has zero" in " ".join(sbom_inventory.split()) def test_documentation_changes_trigger_full_quality_ci() -> None: From e18e88a264c7a37a801f6011cb14c2ac48c2c144 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:09:31 +0900 Subject: [PATCH 48/69] docs(automation): follow redaction successor lineage --- docs/automation/THREAT_MODEL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/automation/THREAT_MODEL.md b/docs/automation/THREAT_MODEL.md index ce95f981c..e55f9ea25 100644 --- a/docs/automation/THREAT_MODEL.md +++ b/docs/automation/THREAT_MODEL.md @@ -170,7 +170,7 @@ Paths below are the current concrete control and regression locations. A listed | Threat | Exact control source | Exact regression evidence | Closure boundary | Current maturity | |---|---|---|---|---| -| TM-01 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | Clean-history integration and consumer proof are proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888). | `active_pr` | +| TM-01 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py`; `tests/test_opencode_security_boundaries.py` | [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) is closed unmerged as `superseded` incident evidence after reachable-history re-contamination; clean-history integration and consumer proof are proposed by Draft [PR #906](https://github.com/ContextualWisdomLab/.github/pull/906). | `active_pr` | | TM-02 | `.github/workflows/deploy-pages.yml`; `docs/automation/SECURITY.md` | `tests/test_automation_documentation_contract.py` | Documentation detects known debt; explicit workflow-call secret declarations and a runtime negative contract remain open. | `accepted_architecture` | | TM-03 | `.github/workflows/agent-mention-router.yml`; `.github/workflows/opencode-review-dispatch.yml`; `scripts/ci/agent_mention_router.py`; `scripts/ci/agent_mention_sweep.py` | `tests/test_agent_mention_complete_payload_binding.py`; `tests/test_agent_mention_idempotency.py`; `tests/test_agent_mention_workflow_contract.py`; `tests/test_opencode_security_boundaries.py` | Named router/wrapper controls exist; end-to-end strict snapshot preservation remains proposed by PR #840. | `active_pr` | | TM-04 | `scripts/ci/pr_head_replay_guard.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_pr_head_replay_guard.py`; `tests/test_pr_review_merge_scheduler.py` | Enforcement still depends on current live ruleset configuration. | `implemented_on_protected_main` | From c89aaf828938955e21d4b0811b7ba32ad4e87478 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:10:17 +0900 Subject: [PATCH 49/69] docs(traceability): supersede contaminated redaction history --- docs/automation/TRACEABILITY.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md index 49a93cc78..da97e7d04 100644 --- a/docs/automation/TRACEABILITY.md +++ b/docs/automation/TRACEABILITY.md @@ -19,7 +19,7 @@ Repository locators below are literal paths with no globs. External canaries and | `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | `accepted_architecture` | | `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; `AGENTS.md`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; external continuation receipt | 15-minute queue sweep plus double fresh whole-queue exit sweep | automation operator / before run exit | `accepted_architecture` | | `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_automation_documentation_contract.py` | source-backed finding/RCA, `remediation_candidate` evidence, exact-head repair and protected recovery receipt | incident owner / before closure | `accepted_architecture` | -| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / clean-history replacement #888 + protected consumer canary | `active_pr` | +| `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / Draft clean-history successor #906 + protected consumer canary | `active_pr` | | `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/DOCUMENTATION_AUDIT.md`; `docs/automation/README.md`; `docs/automation/EVENT_CONTRACTS.md`; `ARCHITECTURE.md` | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | | `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/opencode-review.yml`; `.github/workflows/pr-review-merge-scheduler.yml`; `docs/automation/RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_pr_review_merge_scheduler.py`; real-consumer canary receipt | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | `accepted_architecture` | | `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | `.github/workflows/opencode-review.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml`; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_automation_documentation_contract.py` | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | `accepted_architecture` | @@ -47,9 +47,10 @@ Repository locators below are literal paths with no globs. External canaries and - `ContextualWisdomLab/.github#841` is closed/unmerged historical origin evidence. It established the disclosure concern but mixed unrelated scope; it is `superseded` as an integration path. - `ContextualWisdomLab/.github#842` is closed/unmerged historical RED→GREEN and exhaustive-boundary evidence. Its final blobs removed current-source defects but its reachable PR history retained secret-shaped test fixtures that kept Secret Scan red; it is `superseded` as an integration path. -- `ContextualWisdomLab/.github#888` is the `active_pr` clean-history replacement from protected main. It carries the final ten #842 blobs without the secret-shaped predecessor history. Its exact-head checks, reviews, and threads must be re-fetched at every decision; predecessor or queued evidence is non-authorizing. +- `ContextualWisdomLab/.github#888` is closed/unmerged `superseded` incident evidence. It initially replayed the final ten #842 blobs without predecessor fixture history, but a later committed credential-shaped fixture made its reachable range fail Secret Scan again. +- `ContextualWisdomLab/.github#906` is the Draft `active_pr` clean-history successor from protected main. It carries the corrected final ten blobs, includes the direct Docker/Podman login repair, and keeps bounded wrapper-recursion plus atomic multiline/duplicate-key JSON work explicit. Its exact-head checks, reviews, and threads must be re-fetched at every decision; predecessor or queued evidence is non-authorizing. - Output-memory and service-file quotas remain a separate `planned` resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. -- Every #888 head change invalidates predecessor test counts, coverage counts, reviews, and run IDs. Protected-main/consumer synthetic credential canaries are required before operational closure. +- Every #906 head change invalidates predecessor test counts, coverage counts, reviews, and run IDs. Protected-main/consumer synthetic credential canaries are required before operational closure. ## 4. Whole-conversation and documentation governance traceability From 4b3f4e30fc6ae36e7c7be0d1657290efa1d76c84 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:10:19 +0900 Subject: [PATCH 50/69] docs(adr): record redaction successor supersession --- .../adr/0009-sandbox-evidence-redaction-boundary.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md index c679e0b58..66308b533 100644 --- a/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md +++ b/docs/automation/adr/0009-sandbox-evidence-redaction-boundary.md @@ -1,6 +1,6 @@ # ADR-0009: Redact the complete sandbox evidence boundary while preserving diagnosis -Status: Accepted; protected integration and consumer proof proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) +Status: Accepted; Draft [PR #906](https://github.com/ContextualWisdomLab/.github/pull/906) is the active integration path; [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) is closed unmerged as superseded incident evidence Date: 2026-08-09 Decision owners: CWL security and CI maintainers @@ -8,7 +8,7 @@ Decision owners: CWL security and CI maintainers `sandboxed_verify.py` and `sandboxed_web_e2e.py` execute proof commands and publish completed output, timeout payloads, exceptions, backend/frontend service tails, command metadata, and structured result JSON. Applying redaction to only some print calls leaves credentials reachable through JSON, terminal/Unicode evasion, nested commands, explicit allowed environment values, or stream/result boundaries. Over-broad pattern matching can instead erase benign keys and the diagnostics needed to fix CI. -`ContextualWisdomLab/.github#841` identified the disclosure class but mixed an unrelated SSRF slice and missed publication paths. It was closed unmerged. Its first broad successor was also closed unmerged because secret-shaped fixtures remained in reachable history. [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) is the clean-history active replacement and preserves both predecessors as historical evidence. +`ContextualWisdomLab/.github#841` identified the disclosure class but mixed an unrelated SSRF slice and missed publication paths. It was closed unmerged. Its first broad successor was also closed unmerged because secret-shaped fixtures remained in reachable history. [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) was the clean-history replacement, but a later committed credential-shaped fixture recontaminated its reachable history; it was closed unmerged and remains superseded incident evidence. Draft [PR #906](https://github.com/ContextualWisdomLab/.github/pull/906) is the current clean-history active replacement and preserves all predecessors as historical evidence. ## Decision drivers @@ -62,7 +62,7 @@ This reduces credential disclosure and log-injection risk while preserving audit ## Migration and rollback -Integrate [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) through normal protection, then run a synthetic credential fixture from protected main and an affected consumer. Update the PR body, doctoring, and CHANGELOG with only final exact-head numbers. If regression requires rollback, disable the affected evidence publication or revert to a reviewed fail-closed no-output path; do not restore a known disclosure path. +Integrate Draft [PR #906](https://github.com/ContextualWisdomLab/.github/pull/906) through normal protection after it becomes review-ready, then run a synthetic credential fixture from protected main and an affected consumer. Update the PR body, doctoring, and CHANGELOG with only final exact-head numbers. If regression requires rollback, disable the affected evidence publication or revert to a reviewed fail-closed no-output path; do not restore a known disclosure path. ## Supersession conditions From 034d4a01e5b21e2596ce3827be10a3ea649f36ba Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:10:22 +0900 Subject: [PATCH 51/69] docs(adr): index current redaction successor --- docs/automation/adr/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/automation/adr/README.md b/docs/automation/adr/README.md index 4de778384..5f9c4fc5c 100644 --- a/docs/automation/adr/README.md +++ b/docs/automation/adr/README.md @@ -14,7 +14,7 @@ ADRs record durable decisions, not transient run status. A changed exact head, r | [ADR-0006](0006-protected-main-operational-acceptance.md) | Protected-main/consumer evidence closes operational incidents | Accepted | | [ADR-0007](0007-work-conserving-maintenance.md) | Work-conserving automation; reporting is not completion | Accepted | | [ADR-0008](0008-central-control-plane-and-thin-consumers.md) | Central control-plane ownership and thin product consumers | Accepted | -| [ADR-0009](0009-sandbox-evidence-redaction-boundary.md) | One complete sandbox evidence-redaction boundary with diagnostic preservation | Accepted; clean-history integration proposed by [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) | +| [ADR-0009](0009-sandbox-evidence-redaction-boundary.md) | One complete sandbox evidence-redaction boundary with diagnostic preservation | Accepted; Draft clean-history integration proposed by [PR #906](https://github.com/ContextualWisdomLab/.github/pull/906); [PR #888](https://github.com/ContextualWisdomLab/.github/pull/888) closed unmerged as superseded evidence | | [ADR-0010](0010-agent-mention-routing-and-idempotency-ledger.md) | Authenticated agent-mention routing with an idempotency ledger | Accepted | | [ADR-0011](0011-provider-routing-and-credential-isolation.md) | Ordered provider routing with per-purpose credential isolation | Accepted | | [ADR-0012](0012-hash-pinned-toolchains-and-exact-base-materialization.md) | Hash-pinned toolchains and independently resolved live-base identity | Accepted | From c25e7660e1cf3565428e8d863ae4c524493b7ebe Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:10:24 +0900 Subject: [PATCH 52/69] test(docs): require superseded and active redaction lineage --- tests/test_automation_documentation_contract.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 7b8eb49e6..3a03d7ffb 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -362,6 +362,9 @@ def test_traceability_has_exact_identifiers_and_accountability_columns() -> None """The matrix carries stable IDs and exact implementation accountability.""" traceability = read_document("TRACEABILITY.md") + assert "#888` is closed/unmerged `superseded`" in traceability + assert "#906` is the Draft `active_pr`" in traceability + assert "Every #906 head change invalidates" in traceability for number in range(1, 15): assert f"`PRD-{number:02d}`" in traceability for requirement_id in ( @@ -504,6 +507,10 @@ def test_current_lineage_threat_maturity_and_gap_objects_are_exact() -> None: assert "#842" not in document assert "/pull/842" not in document assert "https://github.com/ContextualWisdomLab/.github/pull/888" in document + assert "https://github.com/ContextualWisdomLab/.github/pull/906" in document + assert "closed unmerged" in document + assert "superseded" in document + assert "Draft" in document traceability_section = threat_model.split( "## 7. Exact source and test traceability", 1 From d096d6ddd3b8dca502b8ab56bf113f9b6030e8a2 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:14:54 +0900 Subject: [PATCH 53/69] docs(traceability): link residual redaction work --- docs/automation/TRACEABILITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md index da97e7d04..b93ed6625 100644 --- a/docs/automation/TRACEABILITY.md +++ b/docs/automation/TRACEABILITY.md @@ -48,7 +48,7 @@ Repository locators below are literal paths with no globs. External canaries and - `ContextualWisdomLab/.github#841` is closed/unmerged historical origin evidence. It established the disclosure concern but mixed unrelated scope; it is `superseded` as an integration path. - `ContextualWisdomLab/.github#842` is closed/unmerged historical RED→GREEN and exhaustive-boundary evidence. Its final blobs removed current-source defects but its reachable PR history retained secret-shaped test fixtures that kept Secret Scan red; it is `superseded` as an integration path. - `ContextualWisdomLab/.github#888` is closed/unmerged `superseded` incident evidence. It initially replayed the final ten #842 blobs without predecessor fixture history, but a later committed credential-shaped fixture made its reachable range fail Secret Scan again. -- `ContextualWisdomLab/.github#906` is the Draft `active_pr` clean-history successor from protected main. It carries the corrected final ten blobs, includes the direct Docker/Podman login repair, and keeps bounded wrapper-recursion plus atomic multiline/duplicate-key JSON work explicit. Its exact-head checks, reviews, and threads must be re-fetched at every decision; predecessor or queued evidence is non-authorizing. +- `ContextualWisdomLab/.github#906` is the Draft `active_pr` clean-history successor from protected main. It carries the corrected final ten blobs and includes the direct Docker/Podman login repair; bounded wrapper recursion is tracked by [Issue #907](https://github.com/ContextualWisdomLab/.github/issues/907), while atomic multiline/duplicate-key JSON redaction is tracked by [Issue #908](https://github.com/ContextualWisdomLab/.github/issues/908). Its exact-head checks, reviews, and threads must be re-fetched at every decision; predecessor or queued evidence is non-authorizing. - Output-memory and service-file quotas remain a separate `planned` resource-safety requirement. Redaction completion must not be reported as resource-exhaustion closure. - Every #906 head change invalidates predecessor test counts, coverage counts, reviews, and run IDs. Protected-main/consumer synthetic credential canaries are required before operational closure. From a6e22300195a95244929c75c716dc5906edc99d7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:38:24 +0900 Subject: [PATCH 54/69] docs(automation): define continuation receipt sink --- docs/automation/CONTINUATION_RUNBOOK.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/automation/CONTINUATION_RUNBOOK.md b/docs/automation/CONTINUATION_RUNBOOK.md index 0471e5d3f..4ed962b21 100644 --- a/docs/automation/CONTINUATION_RUNBOOK.md +++ b/docs/automation/CONTINUATION_RUNBOOK.md @@ -79,6 +79,14 @@ Only `RUN_BUDGET_EXHAUSTED` needs a `continuation_handoff` when executable work The receipt is not persisted in a new database by this document. A GitHub artifact, automation task state, or other existing bounded continuation store may represent it. Persistence changes require the data-model ADR gate. +Use this sink and acknowledgement order: + +1. Persist the exact JSON in an existing bounded automation task-state or run-artifact field when the execution platform exposes one. +2. If no durable structured field is available, emit the receipt as the **sole** terminal user-visible output and label it `NON_CLEAN_CONTINUATION`; it is not `SWEEP2_EMPTY` or completion. +3. Do not create a database, credential, branch, workflow, or GitHub issue solely as a handoff store without the data-model ADR gate. +4. The next recurrence loads the most recent available receipt, acknowledges its `run_identity`, re-fetches every recorded head/base/external identity, discards stale defer keys, and resumes queue selection. +5. If the receipt is missing or unreadable, perform a full fresh sweep. Missing handoff state never authorizes a clean exit. + ## 5. Double exit sweep ### Sweep 1 From 8344fe42514457143b93ee2ef9cf19d00dec7b33 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 00:38:28 +0900 Subject: [PATCH 55/69] test(docs): enforce continuation handoff acknowledgement --- tests/test_automation_documentation_contract.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 3a03d7ffb..8f04576f7 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -277,6 +277,13 @@ def test_autonomy_threats_and_continuation_reason_codes_are_explicit() -> None: assert "Elapsed time exceeds N minutes" in runbook assert "source_head_sha" in runbook assert "live_base_tip_sha" in runbook + for handoff_phrase in ( + "task-state or run-artifact", + "NON_CLEAN_CONTINUATION", + "acknowledges its `run_identity`", + "Missing handoff state never authorizes a clean exit", + ): + assert handoff_phrase in runbook def test_remediation_continuation_and_documentation_entities_are_explicit() -> None: From 439426d7093b9b9019fddd15eeed960b0b893400 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 07:28:02 +0900 Subject: [PATCH 56/69] test(docs): preserve historical loop supersession contracts --- ...n_historical_loop_supersession_contract.py | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 tests/test_automation_historical_loop_supersession_contract.py diff --git a/tests/test_automation_historical_loop_supersession_contract.py b/tests/test_automation_historical_loop_supersession_contract.py new file mode 100644 index 000000000..96839c7b3 --- /dev/null +++ b/tests/test_automation_historical_loop_supersession_contract.py @@ -0,0 +1,45 @@ +"""Regression contracts for superseded historical autonomous-loop guidance.""" + +from __future__ import annotations + +from pathlib import Path + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[1] +AUTOMATION_DOCS = REPOSITORY_ROOT / "docs" / "automation" + + +def _read(relative_path: str) -> str: + """Return one canonical automation document as UTF-8 text.""" + + return (AUTOMATION_DOCS / relative_path).read_text(encoding="utf-8") + + +def test_historical_fixed_wall_clock_cutoffs_are_explicitly_superseded() -> None: + """Old 45-minute/minute-35 caps cannot override work-conserving exit proof.""" + + audit = _read("DOCUMENTATION_AUDIT.md") + decision = _read("adr/0007-work-conserving-maintenance.md") + corpus = f"{audit}\n{decision}" + + assert "fixed 45-minute execution budget" in audit + assert "minute-35 write cutoff" in audit + assert "superseded" in audit + assert "fixed wall-clock cutoff" in decision + assert "practical execution/tool-budget exhaustion" in decision + assert "second fresh sweep" in decision + + +def test_historical_copilot_agent_task_alias_is_explicitly_superseded() -> None: + """The old Agent Tasks token alias cannot re-enter development credentials.""" + + audit = _read("DOCUMENTATION_AUDIT.md") + secret_decision = _read("adr/0004-explicit-secret-contracts.md") + corpus = f"{audit}\n{secret_decision}" + + assert "historical Agent Tasks guidance" in audit + assert "`COPILOT_GITHUB_TOKEN`" in audit + assert "superseded" in audit + assert "must not be reused as a GitHub API credential alias" in secret_decision + assert "purpose-bound explicit secret" in secret_decision + assert "NVIDIA_NIM_API_KEY" in corpus From e978f24c32218096f61743a9e49bed7736f0e6fc Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 07:31:38 +0900 Subject: [PATCH 57/69] docs(automation): supersede fixed wall-clock exit cutoffs --- .../adr/0007-work-conserving-maintenance.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/automation/adr/0007-work-conserving-maintenance.md b/docs/automation/adr/0007-work-conserving-maintenance.md index 438e0e80b..da5bdeba5 100644 --- a/docs/automation/adr/0007-work-conserving-maintenance.md +++ b/docs/automation/adr/0007-work-conserving-maintenance.md @@ -8,6 +8,8 @@ Decision owners: CWL product and automation maintainers Long model reviews, human approval, queued checks, provider cooldowns, and one blocked PR repeatedly caused finite maintenance invocations to stop after reporting status. An hourly recurrence then became an excuse to defer work despite safe tasks in other branches, docs, issues, operations, or product gaps. A second failure mode appeared when prompt edits, documentation audits, review requests, CI dispatches, Draft/Ready transitions, auto-merge enablement, or one successful commit/merge were treated as convenient endpoints even though safe work remained. +Historical scheduler guidance also encoded a fixed 45-minute execution budget and a minute-35 prohibition on beginning new repository writes. That policy was useful as an early collision-avoidance heuristic but is now superseded because it turned elapsed wall-clock time into an implicit completion signal even when another bounded, non-conflicting action was executable. A fixed wall-clock cutoff therefore cannot authorize a clean exit from the current maintenance architecture. + ## Decision drivers - Maximize validated repository progress within each finite invocation. @@ -20,7 +22,7 @@ Long model reviews, human approval, queued checks, provider cooldowns, and one b 1. **Stop after one named request or blocker.** Rejected because it strands executable work. 2. **Poll until the active PR completes.** Rejected because it wastes budget and can starve the queue. -3. **Treat the hourly cadence as a soft wall-clock budget and resume later.** Rejected because recurrence is a continuation mechanism, not evidence that the current finite invocation is work-complete. +3. **Treat the hourly cadence or a fixed wall-clock cutoff as a soft budget and resume later.** Rejected because recurrence is a continuation mechanism, not evidence that the current finite invocation is work-complete. 4. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, and require a double exit sweep.** Selected. ## Decision @@ -29,6 +31,8 @@ Each run maintains live lanes for mergeable PRs, defects/reviews/checks, protect The hourly recurrence is **continuation after genuine practical execution/tool-budget exhaustion**, never a voluntary soft timeout. A prompt update, documentation assessment/update, inventory, RCA without remediation, status or blocker comment, review request, workflow dispatch/rerun, queued/running CI or model review, Draft/Ready transition, auto-merge enablement, commit, merge, completed document, protected-main proof of one scenario, external-governance wait, or completion of one buyer-visible slice is an intermediate event with zero terminal credit while another safe lane exists. +The superseded fixed 45-minute execution budget and minute-35 write cutoff are not current exit criteria. Component jobs and external model calls may still have bounded technical timeouts for safety, cost, or failure isolation, but orchestration must use remaining practical tool/runtime capacity rather than voluntarily stopping at a predeclared minute. When the platform or tool boundary actually prevents another safe action, record the continuation state and resume on the next recurrence. + Before termination, perform a **fresh whole-queue sweep** that includes PRs, issues, protected-main acceptance, docs versus implementation, quality/security/release debt, active writers, and buyer/control-plane gaps. If it finds work, execute the highest-value safe item and sweep again. Only actual practical execution/tool-budget exhaustion or a **second fresh sweep** proving every lane non-actionable permits termination. ## Consequences @@ -37,7 +41,7 @@ Positive: higher throughput, less queue starvation, fewer report-only runs, and ## Failure and recovery -If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If a tool/run budget is genuinely exhausted mid-item, leave exact continuation evidence and resume on the next recurrence; do not claim product completion. If a meta action accidentally becomes the last action, the next run treats that as unfinished continuation rather than completed work. +If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If a tool/run budget is genuinely exhausted mid-item, leave exact continuation evidence and resume on the next recurrence; do not claim product completion. If a meta action accidentally becomes the last action, the next run treats that as unfinished continuation rather than completed work. If a user points out that executable work was left behind, treat that redirection as evidence that the prior exit sweep failed, RCA the missed lane, correct the scheduling/control contract when needed, and resume execution rather than replying with status alone. ## Security and governance impact @@ -51,6 +55,7 @@ Work conservation never permits bypassing approval, checks, writer leases, permi - docs drift appears as executable debt; - prompt/document/status/review/dispatch/Draft/Ready/auto-merge actions cannot satisfy the run exit condition by themselves; - one successful commit or merge cannot satisfy the exit condition while another safe lane exists; +- historical fixed wall-clock cutoffs are explicitly superseded and cannot replace practical execution/tool-budget exhaustion; - elapsed time below a real platform/tool-budget boundary cannot substitute for an exit sweep; - the first exit sweep finding work forces another action; - only practical budget exhaustion or the second fresh all-lanes-non-actionable sweep permits termination; and @@ -58,7 +63,7 @@ Work conservation never permits bypassing approval, checks, writer leases, permi ## Migration and rollback -Update maintainer prompts and scheduler policies with the continuation ledger and double exit sweeps. Remove duplicative inactive loops and prompt wording that encourages status-first termination. Rollback may reduce concurrency but must preserve no-report-as-completion, no-soft-timeout semantics, branch rotation, and the second-sweep termination proof. +Update maintainer prompts and scheduler policies with the continuation ledger and double exit sweeps. Remove duplicative inactive loops and prompt wording that encourages status-first termination or fixed-minute voluntary exits. Rollback may reduce concurrency but must preserve no-report-as-completion, no-soft-timeout semantics, branch rotation, and the second-sweep termination proof. ## Supersession conditions From 1b129e7bd542316accc54e9f27da8d4214e6f102 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 07:32:01 +0900 Subject: [PATCH 58/69] docs(security): supersede Copilot-named API credential alias --- .../adr/0004-explicit-secret-contracts.md | 24 ++++++++++++------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/docs/automation/adr/0004-explicit-secret-contracts.md b/docs/automation/adr/0004-explicit-secret-contracts.md index e41ca3a55..e2193ea38 100644 --- a/docs/automation/adr/0004-explicit-secret-contracts.md +++ b/docs/automation/adr/0004-explicit-secret-contracts.md @@ -8,6 +8,8 @@ Decision owners: CWL security and workflow maintainers Reusable workflows can receive explicitly named secrets or inherit a caller's full secret set. Inheritance is convenient but violates purpose limitation, complicates review, increases disclosure impact, and may supply model/deploy/write credentials to deterministic jobs that never need them. +Historical Agent Tasks guidance also used the name `COPILOT_GITHUB_TOKEN` for a fine-grained GitHub API token. That naming conflated an API authority with Copilot/model identity and now directly conflicts with the organization policy that autonomous development must not use `COPILOT_GITHUB_TOKEN`. The historical alias is superseded rather than grandfathered into the current credential registry. + ## Decision drivers - Least privilege and clear data-flow review. @@ -15,43 +17,49 @@ Reusable workflows can receive explicitly named secrets or inherit a caller's fu - Short-lived OIDC/App authority where possible. - Prevent secrets from reaching free/anonymous providers or untrusted code. - Preserve compatible deployments while legacy callers migrate. +- Keep GitHub API authority, reviewer identity, and model-provider credentials distinct by name and purpose. ## Alternatives considered 1. **Use `secrets: inherit` everywhere.** Rejected as over-broad and unauditable. -2. **Use one organization super-token.** Rejected due blast radius and authority conflation. +2. **Use one organization super-token or reuse a model/reviewer-looking token name for API mutation.** Rejected due blast radius and authority conflation. 3. **Explicit named secrets plus scoped OIDC/App exchange; narrow documented legacy exceptions.** Selected. ## Decision -Every reusable workflow declares the minimum named secrets and documents purpose, required/optional status, consumer job, provider, and failure behavior. Deterministic jobs receive no model secret. `NVIDIA_NIM_API_KEY` is used only for actual approved model calls; `COPILOT_GITHUB_TOKEN` is prohibited for autonomous development. Anonymous/free provider execution receives no unrelated credential. +Every reusable workflow declares the minimum named secrets and documents purpose, required/optional status, consumer job, provider, and failure behavior. Deterministic jobs receive no model secret. `NVIDIA_NIM_API_KEY` is used only for actual approved model calls; `COPILOT_GITHUB_TOKEN` is prohibited for autonomous development and must not be reused as a GitHub API credential alias. Anonymous/free provider execution receives no unrelated credential. + +If a future GitHub API integration cannot use the workflow's minimum `GITHUB_TOKEN` permissions or a short-lived OIDC/App exchange, it requires a separately reviewed, purpose-bound explicit secret whose name describes the authority it grants. That secret must remain separate from model credentials and independent reviewer identities. Existing reviewer credential names/scopes remain unchanged until a separately reviewed migration. -Cross-repository privileges prefer short-lived OIDC/App exchange constrained by trusted workflow source and target. Existing credential names/scopes remain unchanged until a separately reviewed migration. The current inherited-secret deploy pattern is a legacy exception to remove, not precedent. +Cross-repository privileges prefer short-lived OIDC/App exchange constrained by trusted workflow source and target. The current inherited-secret deploy pattern is a legacy exception to remove, not precedent. ## Consequences -Positive: smaller blast radius, readable contracts, provider isolation, and easier audit. Negative: caller updates and compatibility bridges are required; OIDC/App availability becomes an explicit dependency. +Positive: smaller blast radius, readable contracts, provider isolation, and easier audit. Negative: caller updates and compatibility bridges are required; OIDC/App availability becomes an explicit dependency, and legacy token aliases require explicit migration rather than silent reuse. ## Failure and recovery -Missing or failed exchange causes only the privileged/model operation to fail closed. Do not fall back to a broader undocumented token. Continue deterministic/read-only work. Rotate/revoke any exposed credential and inspect every job that could have inherited it. +Missing or failed exchange causes only the privileged/model operation to fail closed. Do not fall back to a broader undocumented token or repurpose `COPILOT_GITHUB_TOKEN`. Continue deterministic/read-only work. Rotate/revoke any exposed credential and inspect every job that could have inherited it. ## Security and governance impact -This implements least privilege, separation of duties, purpose limitation, and auditable credential flow. Credential availability never grants review or merge authority beyond GitHub policy. +This implements least privilege, separation of duties, purpose limitation, and auditable credential flow. Credential availability never grants review or merge authority beyond GitHub policy. A token name is part of the authority contract: misleading aliases are treated as security debt because they make reviewer, model, and repository-mutation boundaries harder to audit. ## Tests and acceptance - workflow contracts enumerate allowed secrets and job permissions; - deterministic and anonymous paths assert model/write secrets absent; +- autonomous development contracts reject `COPILOT_GITHUB_TOKEN`; +- historical Agent Tasks guidance is explicitly marked superseded rather than accepted as a credential exception; +- future non-default GitHub API authority uses a purpose-bound explicit secret or reviewed short-lived App/OIDC exchange, not a model/reviewer alias; - OIDC audience/target/actor negative tests; - redaction covers all secret transport and error paths; - real positive/negative consumer canary after migration. ## Migration and rollback -Inventory inherited callers, add explicit callee declarations, update one low-risk caller, run positive and missing-secret controls, then migrate the fleet. Rollback restores the prior caller only if it does not re-expose a known credential; otherwise disable the optional deployment/review path. +Inventory inherited callers and legacy credential aliases, add explicit callee declarations, update one low-risk caller, run positive and missing-secret controls, then migrate the fleet. Remove or rename any historical `COPILOT_GITHUB_TOKEN` API alias through a reviewed consumer migration rather than retaining two meanings for one name. Rollback restores a prior caller only if it does not re-expose a known credential or resurrect the superseded alias; otherwise disable the optional deployment/review path. ## Supersession conditions -Supersede when GitHub enforces callee-declared per-secret purpose and automatic least-privilege token attenuation across reusable workflows with equivalent audit evidence. +Supersede when GitHub enforces callee-declared per-secret purpose and automatic least-privilege token attenuation across reusable workflows with equivalent audit evidence, while preserving distinct reviewer, model, and repository-mutation authorities. \ No newline at end of file From 6de58d9beaac3300674a9df1ab36cfbf8ba45294 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 07:32:46 +0900 Subject: [PATCH 59/69] docs(automation): reconcile superseded loop guidance --- docs/automation/DOCUMENTATION_AUDIT.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md index 3c3ba9155..3792cb724 100644 --- a/docs/automation/DOCUMENTATION_AUDIT.md +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -1,7 +1,7 @@ # Whole-conversation documentation fitness audit Status: active_pr -Last reviewed: 2026-08-09 +Last reviewed: 2026-08-10 Scope: `ContextualWisdomLab/.github` automation control plane and its contracts with thin repository consumers ## Purpose @@ -44,6 +44,15 @@ Every durable claim in this documentation graph MUST map to one of these states: | BandScope / Inkspan / pg-erd-cloud / naruon / AppGuardrail and other leaf products | Thin caller, review, security, release and evidence interfaces | `out_of_scope` product semantics; central docs specify only the reusable contract and consumer acceptance requirements. | | Organization ecosystem composition through naruon/contextual-orchestrator/connectors | Interface-level concern | `accepted_architecture` only for central interface ownership, credentials, provenance and thin-consumer rules; business composition is leaf-owned. | +## Historical autonomous-loop contradictions explicitly superseded + +The whole-conversation review found two old instructions that are useful incident evidence but cannot coexist with the current accepted automation contract. + +1. **Fixed time cutoffs.** Historical scheduler guidance used a **fixed 45-minute execution budget** and a **minute-35 write cutoff**. Those constraints are `superseded`. They were an early attempt to avoid overlapping writers, but they can leave executable work behind solely because the clock reached an arbitrary minute. Current authority comes from ADR-0007: branch-local writer leases, exact deferred identities, work-conserving rotation, practical execution/tool-budget exhaustion, and a second fresh all-lanes-nonactionable sweep. Individual jobs or model calls may still have bounded technical timeouts, but elapsed wall-clock time alone is not a clean-exit proof. +2. **Copilot-named Agent Tasks credential.** Historical Agent Tasks guidance instructed operators to store a fine-grained GitHub API token under `COPILOT_GITHUB_TOKEN`. That **historical Agent Tasks guidance** is `superseded`. Current policy prohibits `COPILOT_GITHUB_TOKEN` for autonomous development and keeps model, reviewer, and repository-mutation authorities separate. Model-backed development uses `NVIDIA_NIM_API_KEY`; GitHub API authority uses the minimum `GITHUB_TOKEN`, reviewed short-lived OIDC/App exchange, or—only where strictly necessary—a separately named purpose-bound explicit secret. The old alias is not a compatibility exception. + +These corrections do not rewrite historical artifacts. They make their maturity explicit so an old prompt/file cannot silently override the current canonical graph. + ## Documentation family fitness This table assesses the **active PR documentation baseline**, not protected-main runtime implementation. An `ADEQUATE` documentation family may still describe `accepted_architecture`, `planned`, or `active_pr` behavior whose runtime evidence is incomplete. @@ -56,9 +65,9 @@ This table assesses the **active PR documentation baseline**, not protected-main | UML | ADEQUATE | Machine-check component, PR-maintenance/product-development sequences, evidence state, authority, deployment, retry, mention/sandbox, writer rotation and documentation-continuation flows. The FigJam companion remains supplemental only. | | ERD / data model | ADEQUATE | `remediation_candidate`, `continuation_handoff`, `documentation_artifact`, `traceability_record`, PR-base/live-base identity and evidence-authority relationships are explicit; the model remains conceptual until a persistence ADR is accepted. | | Security / threat model | ADEQUATE | Keep credential, confused-deputy, untrusted-source, prompt-injection, stale-evidence, PII and supply-chain cases current. | -| Test strategy | ADEQUATE | Documentation-fitness regression coverage now binds this audit, maturity mapping, logical entities and standards freshness; semantic review remains required. | +| Test strategy | ADEQUATE | Documentation-fitness regression coverage binds this audit, maturity mapping, historical supersession, logical entities and standards freshness; semantic review remains required. | | Operability / incident runbook | ADEQUATE | Preserve protected-main/consumer closure and queue/provider/runner/DNS recovery evidence. | -| ADR set | ADEQUATE | ADR-0007 now explicitly classifies prompt/docs/status/review/dispatch/Draft/Ready/auto-merge/commit/merge as intermediate and forbids a voluntary soft timeout. | +| ADR set | ADEQUATE | ADR-0007 explicitly supersedes fixed wall-clock exit cutoffs and classifies prompt/docs/status/review/dispatch/Draft/Ready/auto-merge/commit/merge as intermediate; ADR-0004 supersedes the old Copilot-named Agent Tasks token alias. | | Traceability | ADEQUATE | Controlled maturity, current redaction lineage, whole-conversation governance, standards, implementation/test/gate ownership and explicit product/control-plane debt are mapped. | | Standards doctoring | ADEQUATE | Current final baselines include SLSA 1.2, ISO/IEC/IEEE 42010:2022, ISO/IEC/IEEE 29148:2018, ISO/IEC 25010:2023 and NIST SSDF 1.1; newer drafts are labelled informative/non-normative until final. | @@ -90,6 +99,8 @@ source/prose contradictions are corrected or explicitly classified as debt: | Strix concurrency | PR/event-or-ref grouping uses `cancel-in-progress: true`. | Current README/rollout/audit corrections match executable YAML. | | Autofix owner | `pg-erd-cloud` uses the central autofix worker as shared default. | Historical repo-local recommendations are marked superseded. | | NVIDIA secret | Workflow secret `NVIDIA_NIM_API_KEY` maps to process `NVIDIA_API_KEY`; there is no legacy alias or routine ruleset bypass. | The historical hotfix note now names the current fail-closed contract. | +| Historical scheduler cutoff | Older planning material imposed a 45-minute run and minute-35 no-new-write boundary. | `superseded`; ADR-0007 and the current automation prompt require practical-budget plus double-sweep exit proof instead. | +| Historical Agent Tasks token alias | Older guidance used `COPILOT_GITHUB_TOKEN` as a GitHub API token name. | `superseded`; ADR-0004 requires purpose-bound explicit GitHub authority and prohibits reuse of the Copilot-named alias. | | Reusable secrets | Four protected-main workflows consume undeclared reusable secrets. | `SECURITY.md` registers the debt; deploy repair is active in PR #901 and the other three remain planned. | | SBOM zero state | No successful inventory receipt means no fleet population was materialized. | Zero rows cannot prove zero components or no license findings. | @@ -127,7 +138,7 @@ flowchart LR The documentation family is review-ready only when: 1. `docs/automation/README.md` indexes the complete canonical graph and whole-conversation audit; -2. machine tests validate controlled maturity, whole-conversation ownership, double-exit/no-soft-timeout behavior, conceptual-vs-persisted boundaries, remediation/continuation/documentation/traceability entities, standards freshness, local links and Mermaid/code fences; +2. machine tests validate controlled maturity, whole-conversation ownership, historical-loop supersession, double-exit/no-soft-timeout behavior, conceptual-vs-persisted boundaries, remediation/continuation/documentation/traceability entities, standards freshness, local links and Mermaid code fences; 3. PRD/TRD/Architecture/UML/ERD/Data Model/Security/Test/Operability/ADR/Traceability/standards doctoring have no known family-level `PARTIAL` or `MISSING` gap on the same exact head; 4. exact-head documentation/security/supply-chain checks and current review are clean; and 5. after protected merge, the canonical index is refetched from protected `main` before any `implemented_on_protected_main` claim is made. From 4dbf099fc55d13e0af677a39c9221b462112af61 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 07:33:11 +0900 Subject: [PATCH 60/69] docs(changelog): record historical loop supersession --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ff8341881..c842d5c68 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Semantic Versioning where the repository publishes a release. ### Fixed +- Reconciled superseded autonomous-loop guidance so historical fixed 45-minute/minute-35 cutoffs cannot replace practical-budget plus double-sweep exit proof, and retired the historical `COPILOT_GITHUB_TOKEN` Agent Tasks API alias in favor of purpose-bound GitHub authority separated from `NVIDIA_NIM_API_KEY` model access. - Bounded the Strix quality self-test's deterministic timeout fixtures to 3-second process and 5-second fake-sleep budgets so exact-head policy evidence completes inside the existing job limit without changing production Strix scanner timeouts, providers, credentials, or review semantics. - Allowed commas and ASCII parentheses in the bounded Strix changed-file path policy so legal tracked Packrat fixtures can receive exact-head security analysis, while rejecting raw `..` components before normalization and keeping controls, backslashes, whitespace ambiguity, and shell punctuation fail-closed. - Bound each review-agent invocation key to the wrapper's complete canonical payload, including the base branch and requesting actor; altered fields with a valid-format key now fail before durable-leader election or forwarding, and wrapper write permission is job-scoped. From cc704323d59650165435a288584ba7b4b9ac9ed3 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:43:47 +0900 Subject: [PATCH 61/69] test(automation): require same-run redirection continuation --- ...ation_historical_loop_supersession_contract.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tests/test_automation_historical_loop_supersession_contract.py b/tests/test_automation_historical_loop_supersession_contract.py index 96839c7b3..9b43f2404 100644 --- a/tests/test_automation_historical_loop_supersession_contract.py +++ b/tests/test_automation_historical_loop_supersession_contract.py @@ -43,3 +43,18 @@ def test_historical_copilot_agent_task_alias_is_explicitly_superseded() -> None: assert "must not be reused as a GitHub API credential alias" in secret_decision assert "purpose-bound explicit secret" in secret_decision assert "NVIDIA_NIM_API_KEY" in corpus + + +def test_user_redirection_requires_same_invocation_multi_lane_continuation() -> None: + """A user-reported early stop must trigger work, not another report-only run.""" + + decision = _read("adr/0007-work-conserving-maintenance.md") + runbook = _read("CONTINUATION_RUNBOOK.md") + corpus = f"{decision}\n{runbook}".lower() + + assert "user_redirection_incident" in corpus + assert "same invocation" in corpus + assert "zero completion credit" in corpus + assert "at least two materially distinct" in corpus + assert "non-documentation" in corpus + assert "two fresh whole-queue sweeps" in corpus From d237505d5f7bc7c05a513875b0c9e73af1a5cb08 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:44:39 +0900 Subject: [PATCH 62/69] docs(automation): make user redirection a same-run execution gate --- .../adr/0007-work-conserving-maintenance.md | 28 +++++++++++++------ 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/docs/automation/adr/0007-work-conserving-maintenance.md b/docs/automation/adr/0007-work-conserving-maintenance.md index da5bdeba5..ac6fe9a1e 100644 --- a/docs/automation/adr/0007-work-conserving-maintenance.md +++ b/docs/automation/adr/0007-work-conserving-maintenance.md @@ -8,6 +8,8 @@ Decision owners: CWL product and automation maintainers Long model reviews, human approval, queued checks, provider cooldowns, and one blocked PR repeatedly caused finite maintenance invocations to stop after reporting status. An hourly recurrence then became an excuse to defer work despite safe tasks in other branches, docs, issues, operations, or product gaps. A second failure mode appeared when prompt edits, documentation audits, review requests, CI dispatches, Draft/Ready transitions, auto-merge enablement, or one successful commit/merge were treated as convenient endpoints even though safe work remained. +Repeated user redirection exposed a third control failure: after being told explicitly that executable work remained, the automation could repair its prompt or documentation and still terminate without proving that repository execution had actually resumed. A scheduler that treats the redirection itself as another reporting opportunity has not corrected the failed exit condition. The recovery therefore needs an observable same-invocation execution requirement rather than another prose-only reminder. + Historical scheduler guidance also encoded a fixed 45-minute execution budget and a minute-35 prohibition on beginning new repository writes. That policy was useful as an early collision-avoidance heuristic but is now superseded because it turned elapsed wall-clock time into an implicit completion signal even when another bounded, non-conflicting action was executable. A fixed wall-clock cutoff therefore cannot authorize a clean exit from the current maintenance architecture. ## Decision drivers @@ -15,15 +17,17 @@ Historical scheduler guidance also encoded a fixed 45-minute execution budget an - Maximize validated repository progress within each finite invocation. - Avoid polling and repetitive blocker narration. - Keep dependencies and writer safety without serializing the whole fleet. -- Make documentation and operational proof first-class work. +- Make documentation and operational proof first-class work without allowing them to substitute for implementation work. - Prevent elapsed wall-clock time or meta/control-plane activity from becoming an implicit voluntary timeout. +- Make recovery from a user-reported premature stop falsifiable in the same invocation. ## Alternatives considered 1. **Stop after one named request or blocker.** Rejected because it strands executable work. 2. **Poll until the active PR completes.** Rejected because it wastes budget and can starve the queue. 3. **Treat the hourly cadence or a fixed wall-clock cutoff as a soft budget and resume later.** Rejected because recurrence is a continuation mechanism, not evidence that the current finite invocation is work-complete. -4. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, and require a double exit sweep.** Selected. +4. **On user redirection, update the scheduler prompt and wait for the next hourly run.** Rejected because a prompt mutation is control-plane preparation, not evidence that missed repository work resumed. +5. **Maintain multiple ordered lanes, defer blocked identities, rotate after every action, require same-invocation recovery after redirection, and require a double exit sweep.** Selected. ## Decision @@ -31,21 +35,25 @@ Each run maintains live lanes for mergeable PRs, defects/reviews/checks, protect The hourly recurrence is **continuation after genuine practical execution/tool-budget exhaustion**, never a voluntary soft timeout. A prompt update, documentation assessment/update, inventory, RCA without remediation, status or blocker comment, review request, workflow dispatch/rerun, queued/running CI or model review, Draft/Ready transition, auto-merge enablement, commit, merge, completed document, protected-main proof of one scenario, external-governance wait, or completion of one buyer-visible slice is an intermediate event with zero terminal credit while another safe lane exists. +A user statement that work was left behind creates reason code `USER_REDIRECTION_INCIDENT`. Recovery occurs in the **same invocation**. Prompt editing, inventory, RCA prose, documentation assessment/mutation, one commit/PR/check/review request/merge/blocker, or one product slice receives **zero completion credit** by itself. The automation must rebuild the full live queue and resume repository execution. If at least two independent safe execute-now lanes exist, it must perform **at least two materially distinct** substantive repository actions before termination is eligible, with at least one **non-documentation** action whenever any safe non-documentation lane exists. A documentation action can satisfy at most one of those lanes. If fresh evidence proves only one execute-now lane exists, execute it and then require **two fresh whole-queue sweeps** to prove no second lane has become executable. A final response that knowingly leaves an execute-now lane is a recurrence of the scheduler defect. + The superseded fixed 45-minute execution budget and minute-35 write cutoff are not current exit criteria. Component jobs and external model calls may still have bounded technical timeouts for safety, cost, or failure isolation, but orchestration must use remaining practical tool/runtime capacity rather than voluntarily stopping at a predeclared minute. When the platform or tool boundary actually prevents another safe action, record the continuation state and resume on the next recurrence. -Before termination, perform a **fresh whole-queue sweep** that includes PRs, issues, protected-main acceptance, docs versus implementation, quality/security/release debt, active writers, and buyer/control-plane gaps. If it finds work, execute the highest-value safe item and sweep again. Only actual practical execution/tool-budget exhaustion or a **second fresh sweep** proving every lane non-actionable permits termination. +Before termination, perform a **fresh whole-queue sweep** that includes PRs, issues, protected-main acceptance, docs versus implementation, quality/security/release debt, active writers, and buyer/control-plane gaps. If it finds work, execute the highest-value safe item and sweep again. Only actual practical execution/tool-budget exhaustion or a **second fresh sweep** proving every lane non-actionable permits termination. A user-redirection recovery resets the sweep count after its required substantive actions. ## Consequences -Positive: higher throughput, less queue starvation, fewer report-only runs, and durable progress while providers or humans wait. Negative: the automation needs careful scope/lease management, exact defer identities, and a continuation ledger capable of distinguishing locally blocked work from globally empty work. +Positive: higher throughput, less queue starvation, fewer report-only runs, explicit recovery from premature-stop incidents, and durable progress while providers or humans wait. Negative: the automation needs careful scope/lease management, exact defer identities, an executable-queue view, and a continuation ledger capable of distinguishing locally blocked work from globally empty work. A redirection can require more than one branch or work family to be advanced in one finite invocation, so branch isolation and exact-head refetches remain mandatory. ## Failure and recovery -If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If a tool/run budget is genuinely exhausted mid-item, leave exact continuation evidence and resume on the next recurrence; do not claim product completion. If a meta action accidentally becomes the last action, the next run treats that as unfinished continuation rather than completed work. If a user points out that executable work was left behind, treat that redirection as evidence that the prior exit sweep failed, RCA the missed lane, correct the scheduling/control contract when needed, and resume execution rather than replying with status alone. +If queue selection creates conflicts or unsafe scope expansion, stop only the affected lane, record the boundary, and choose a disjoint item. If a tool/run budget is genuinely exhausted mid-item, leave exact continuation evidence and resume on the next recurrence; do not claim product completion. If a meta action accidentally becomes the last action, the next run treats that as unfinished continuation rather than completed work. + +If a user points out that executable work was left behind, classify `USER_REDIRECTION_INCIDENT`, identify the last voluntary terminal condition and at least one missed lane, correct the scheduling/control contract when needed, rebuild the whole queue, and resume execution in the same invocation. When two independent execute-now lanes exist, advancing only one is still an incomplete recovery. Documentation or prompt repair cannot be the final action when non-documentation work is executable. After recovery actions, restart the two-sweep exit proof from fresh evidence. ## Security and governance impact -Work conservation never permits bypassing approval, checks, writer leases, permissions, privacy controls, or fail-closed evidence boundaries. It changes scheduling, not authority. Routine output is suppressed so status narration cannot substitute for work. The no-soft-timeout rule does not authorize unbounded single-job execution; component/job timeouts remain bounded according to their own technical contracts. +Work conservation never permits bypassing approval, checks, writer leases, permissions, privacy controls, or fail-closed evidence boundaries. It changes scheduling, not authority. Routine output is suppressed so status narration cannot substitute for work. The no-soft-timeout rule does not authorize unbounded single-job execution; component/job timeouts remain bounded according to their own technical contracts. The two-lane redirection rule applies only to actions independently safe under current authority and lease; it never authorizes racing another writer or weakening a merge gate merely to satisfy an action count. ## Tests and acceptance @@ -55,6 +63,10 @@ Work conservation never permits bypassing approval, checks, writer leases, permi - docs drift appears as executable debt; - prompt/document/status/review/dispatch/Draft/Ready/auto-merge actions cannot satisfy the run exit condition by themselves; - one successful commit or merge cannot satisfy the exit condition while another safe lane exists; +- `USER_REDIRECTION_INCIDENT` requires same-invocation queue rebuild and substantive execution; +- user-redirection prompt/docs repair has zero completion credit by itself; +- when at least two independent safe lanes exist after redirection, at least two materially distinct substantive actions occur and one is non-documentation when available; +- a one-lane redirection recovery is followed by two fresh whole-queue sweeps proving no second execute-now lane exists; - historical fixed wall-clock cutoffs are explicitly superseded and cannot replace practical execution/tool-budget exhaustion; - elapsed time below a real platform/tool-budget boundary cannot substitute for an exit sweep; - the first exit sweep finding work forces another action; @@ -63,8 +75,8 @@ Work conservation never permits bypassing approval, checks, writer leases, permi ## Migration and rollback -Update maintainer prompts and scheduler policies with the continuation ledger and double exit sweeps. Remove duplicative inactive loops and prompt wording that encourages status-first termination or fixed-minute voluntary exits. Rollback may reduce concurrency but must preserve no-report-as-completion, no-soft-timeout semantics, branch rotation, and the second-sweep termination proof. +Update maintainer prompts and scheduler policies with the continuation ledger, `USER_REDIRECTION_INCIDENT`, same-invocation recovery, multi-lane proof, and double exit sweeps. Remove duplicative inactive loops and prompt wording that encourages status-first termination or fixed-minute voluntary exits. Rollback may reduce concurrency but must preserve no-report-as-completion, no-soft-timeout semantics, same-invocation redirection recovery, branch rotation, and the second-sweep termination proof. ## Supersession conditions -Supersede if a durable queue engine provides provably fair, dependency-aware, lease-safe work conservation and equivalent termination proofs across repositories, including explicit handling of meta actions, external waits, and practical run-budget exhaustion. \ No newline at end of file +Supersede if a durable queue engine provides provably fair, dependency-aware, lease-safe work conservation and equivalent termination proofs across repositories, including explicit handling of user redirection, meta actions, external waits, and practical run-budget exhaustion. \ No newline at end of file From 974bc5158ac5ebcc7f0fd542944edb5e74187076 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:45:36 +0900 Subject: [PATCH 63/69] docs(automation): operationalize same-run premature-stop recovery --- docs/automation/CONTINUATION_RUNBOOK.md | 39 +++++++++++++++++++------ 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/docs/automation/CONTINUATION_RUNBOOK.md b/docs/automation/CONTINUATION_RUNBOOK.md index 4ed962b21..c624b369f 100644 --- a/docs/automation/CONTINUATION_RUNBOOK.md +++ b/docs/automation/CONTINUATION_RUNBOOK.md @@ -1,20 +1,21 @@ # Autonomous continuation runbook Status: active_pr operability companion -Last reviewed: 2026-08-09 -Scope: work-conserving queue rotation, wait/defer semantics, real run-budget exhaustion, and observable termination evidence +Last reviewed: 2026-08-10 +Scope: work-conserving queue rotation, wait/defer semantics, user-redirection recovery, real run-budget exhaustion, and observable termination evidence This runbook operationalizes ADR-0007, [AUTONOMY_THREATS.md](AUTONOMY_THREATS.md), and the logical `continuation_handoff` model. It does not replace workflow-specific timeouts or GitHub rulesets. ## 1. Operator rule -A blocked or waiting item is **local state**, not run completion. The automation defers the exact identity and selects another safe lane. Hourly recurrence means continuation after a genuine practical run/tool limit, not “stop after about an hour.” +A blocked or waiting item is **local state**, not run completion. The automation defers the exact identity and selects another safe lane. Hourly recurrence means continuation after a genuine practical run/tool limit, not “stop after about an hour.” Prompt or documentation maintenance is control-plane preparation and cannot be the final voluntary action while safe repository work exists. ## 2. Observable lane states and reason codes | Reason code | Meaning | Allowed action | Run terminal? | |---|---|---|---| | `EXECUTABLE_NOW` | Safe mutation/test/merge/operational action exists and writer lease is available. | Execute highest-value item. | No | +| `USER_REDIRECTION_INCIDENT` | User reports that a prior invocation stopped while executable work remained. | Reconstruct missed terminal condition, rebuild whole queue, perform same-invocation substantive recovery, then restart exit sweeps. | No | | `WAIT_CHECK_PENDING` | Exact-head required check is queued/in-progress. | Defer exact PR/head/check identity; rotate. | No | | `WAIT_REVIEW_PENDING` | Current-head automated or human review is pending. | Defer exact PR/head/reviewer identity; rotate. | No | | `WAIT_EXTERNAL_APPROVAL` | Qualifying independent human approval is the local remaining gate. | Preserve expected-head-safe merge posture if policy allows; rotate. | No | @@ -87,7 +88,22 @@ Use this sink and acknowledgement order: 4. The next recurrence loads the most recent available receipt, acknowledges its `run_identity`, re-fetches every recorded head/base/external identity, discards stale defer keys, and resumes queue selection. 5. If the receipt is missing or unreadable, perform a full fresh sweep. Missing handoff state never authorizes a clean exit. -## 5. Double exit sweep +## 5. User-redirection recovery + +A user statement such as “work remained,” “why did you stop,” or an equivalent request to repair the prompt is direct evidence that the previous exit proof failed. Classify `USER_REDIRECTION_INCIDENT` and recover in the **same invocation**. + +Use this order: + +1. Reconstruct the last voluntary terminal condition from available evidence. Identify at least one missed safe lane; do not substitute a generic apology or status recap. +2. Refetch the full live queue, protected main, exact PR heads, independently resolved live base tips, current writer leases, reviews/checks/security evidence, canonical documentation, and protected-main acceptance debt. +3. Repair the scheduler prompt or canonical documentation only when the incident exposes an actual control-contract gap. Prompt editing, inventory, RCA prose, documentation assessment/mutation, one commit/PR/check/review request/merge/blocker, or one product slice receives **zero completion credit** by itself. +4. Resume substantive repository execution immediately. If **at least two materially distinct** independent safe execute-now lanes exist, advance at least two before termination is eligible. At least one recovery action must be **non-documentation** whenever a safe non-documentation lane exists. A documentation action may satisfy at most one lane. +5. If fresh evidence proves only one execute-now lane exists, execute it and then perform **two fresh whole-queue sweeps** from new live reads to prove no second lane is executable. +6. Reset the exit-sweep count after every recovery action. A final response that knowingly leaves an `EXECUTABLE_NOW` lane is another `USER_REDIRECTION_INCIDENT`. + +The action-count rule never changes authority. Two unsafe writes, two actions on the same waiting identity, duplicate review requests, or a raced writer do not satisfy recovery. Distinct actions must each be independently safe and materially advance repository state or exact acceptance evidence. + +## 6. Double exit sweep ### Sweep 1 @@ -109,9 +125,9 @@ If any safe action exists, classify `EXECUTABLE_NOW`, execute it, and the exit s Only after `SWEEP1_EMPTY`, perform the same inventory from **new live reads**. Do not reuse cached PR/check/reviewer state as proof. If still empty, `SWEEP2_EMPTY` permits termination. -A queued check or external approval on one PR does not make the sweep empty if any other safe lane exists. +A queued check or external approval on one PR does not make the sweep empty if any other safe lane exists. After `USER_REDIRECTION_INCIDENT`, the two fresh sweeps begin only after the required same-invocation recovery work has been performed. -## 6. Retry and timeout interaction +## 7. Retry and timeout interaction Workflow/component timeouts remain bounded and are separate from maintenance-run termination. @@ -121,7 +137,7 @@ Workflow/component timeouts remain bounded and are separate from maintenance-run - long OpenCode/Noema/Strix job → defer exact run identity and continue other work; - GitHub Actions queue saturation → do not poll; advance source/docs/issues that do not conflict. -## 7. Split authority operational check +## 8. Split authority operational check Before merge, mutation, or protected incident closure, operator evidence must answer separately: @@ -139,7 +155,7 @@ Before merge, mutation, or protected incident closure, operator evidence must an A missing answer cannot be inferred from another channel. -## 8. Monitoring +## 9. Monitoring Useful finite-cardinality measures: @@ -150,16 +166,21 @@ Useful finite-cardinality measures: - time from exact-head gate-clean to protected merge; - time from protected merge to operational acceptance; - count of `META_INTERMEDIATE` events followed by another substantive action; +- count of `USER_REDIRECTION_INCIDENT` recoveries that advanced one versus at least two distinct lanes; +- count of redirection recoveries where a safe non-documentation lane followed documentation repair; - count of first exit sweeps that discovered work; and - number of `RUN_BUDGET_EXHAUSTED` continuation handoffs with executable lanes remaining. Do not place repository source text, comment bodies, PII, model output, or credentials in metric labels. -## 9. Reopening and escalation +## 10. Reopening and escalation Reopen continuation incidents when: - a run terminates after a meta/control event while safe work existed; +- user redirection is answered only with prompt/document/report work while safe repository execution exists; +- two independent safe lanes exist after redirection but only one is advanced before termination; +- documentation is the last recovery action while a non-documentation lane is executable; - a waiting PR starves an unrelated lane; - a defer key suppresses a changed head/base or unrelated branch; - a second exit sweep reused stale evidence; From 562007ff78fb69fab08fdb511ab7e3761e85f41a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:46:55 +0900 Subject: [PATCH 64/69] docs(automation): record repeated premature-stop recovery contract --- docs/automation/DOCUMENTATION_AUDIT.md | 28 ++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/docs/automation/DOCUMENTATION_AUDIT.md b/docs/automation/DOCUMENTATION_AUDIT.md index 3792cb724..725ccb85a 100644 --- a/docs/automation/DOCUMENTATION_AUDIT.md +++ b/docs/automation/DOCUMENTATION_AUDIT.md @@ -10,6 +10,8 @@ This audit answers whether the durable PRD, TRD, Architecture, UML, ERD/data mod Conversation history, prompts, PR bodies, attached planning files, broad master-context documents, and model output are **candidate evidence**, not architectural authority. A decision enters the canonical graph only after it is revalidated against protected-main implementation, current GitHub state, or an explicit accepted-architecture decision. Product-specific detail remains owned by the product repository rather than being copied into the organization control plane. +The current family-level result is **ADEQUATE on this active PR only after the 2026-08-10 premature-stop reconciliation below**. The protected branch is still insufficient as durable canonical truth until this documentation line is integrated. `ADEQUATE` describes coverage and semantic coherence of the active documentation graph; it never means that runtime gaps, active PRs, operational acceptance, or releases are complete. + ## Controlled maturity vocabulary Every durable claim in this documentation graph MUST map to one of these states: @@ -30,6 +32,7 @@ Every durable claim in this documentation graph MUST map to one of these states: |---|---|---| | PR review → repair → current-head checks → merge → next work loops | Direct | `accepted_architecture`; PRD/TRD/ADR-0007 define work conservation and authority. | | Hourly autonomous continuation and no report-as-completion | Direct | `accepted_architecture`; every prompt edit, documentation assessment/update, inventory, status comment, review request, CI dispatch, Draft/Ready transition, auto-merge enablement, commit, merge, document update, or one buyer slice is an intermediate event. | +| User reports that the run stopped while work remained | Direct | `accepted_architecture`; classify `USER_REDIRECTION_INCIDENT`, repair the control contract only if needed, resume substantive work in the same invocation, and when two independent safe lanes exist advance at least two materially distinct actions with a non-documentation action whenever available before a new exit proof. | | Exact source-head versus PR-base snapshot versus independently resolved live-base tip | Direct | `accepted_architecture`; TRD revision identity and ADR-0002. | | Reviewer/check/status/model/merge/release authority separation | Direct | `accepted_architecture`; TRD evidence taxonomy and ADR-0005/0016. | | Central `.github` ownership with thin modular leaf consumers | Direct | `accepted_architecture`; Architecture and ADR-0008. | @@ -46,10 +49,11 @@ Every durable claim in this documentation graph MUST map to one of these states: ## Historical autonomous-loop contradictions explicitly superseded -The whole-conversation review found two old instructions that are useful incident evidence but cannot coexist with the current accepted automation contract. +The whole-conversation review found old instructions that are useful incident evidence but cannot coexist with the current accepted automation contract. 1. **Fixed time cutoffs.** Historical scheduler guidance used a **fixed 45-minute execution budget** and a **minute-35 write cutoff**. Those constraints are `superseded`. They were an early attempt to avoid overlapping writers, but they can leave executable work behind solely because the clock reached an arbitrary minute. Current authority comes from ADR-0007: branch-local writer leases, exact deferred identities, work-conserving rotation, practical execution/tool-budget exhaustion, and a second fresh all-lanes-nonactionable sweep. Individual jobs or model calls may still have bounded technical timeouts, but elapsed wall-clock time alone is not a clean-exit proof. 2. **Copilot-named Agent Tasks credential.** Historical Agent Tasks guidance instructed operators to store a fine-grained GitHub API token under `COPILOT_GITHUB_TOKEN`. That **historical Agent Tasks guidance** is `superseded`. Current policy prohibits `COPILOT_GITHUB_TOKEN` for autonomous development and keeps model, reviewer, and repository-mutation authorities separate. Model-backed development uses `NVIDIA_NIM_API_KEY`; GitHub API authority uses the minimum `GITHUB_TOKEN`, reviewed short-lived OIDC/App exchange, or—only where strictly necessary—a separately named purpose-bound explicit secret. The old alias is not a compatibility exception. +3. **Prompt repair as sufficient recovery.** Earlier loop text correctly said prompt/doc/status events were intermediate but did not make a repeated user-reported premature stop falsifiable in the current invocation. That weaker recovery interpretation is `superseded`. `USER_REDIRECTION_INCIDENT` now requires same-invocation substantive repository execution, at least two materially distinct safe actions when two lanes exist, a non-documentation action when available, and a new two-sweep exit proof. These corrections do not rewrite historical artifacts. They make their maturity explicit so an old prompt/file cannot silently override the current canonical graph. @@ -65,12 +69,14 @@ This table assesses the **active PR documentation baseline**, not protected-main | UML | ADEQUATE | Machine-check component, PR-maintenance/product-development sequences, evidence state, authority, deployment, retry, mention/sandbox, writer rotation and documentation-continuation flows. The FigJam companion remains supplemental only. | | ERD / data model | ADEQUATE | `remediation_candidate`, `continuation_handoff`, `documentation_artifact`, `traceability_record`, PR-base/live-base identity and evidence-authority relationships are explicit; the model remains conceptual until a persistence ADR is accepted. | | Security / threat model | ADEQUATE | Keep credential, confused-deputy, untrusted-source, prompt-injection, stale-evidence, PII and supply-chain cases current. | -| Test strategy | ADEQUATE | Documentation-fitness regression coverage binds this audit, maturity mapping, historical supersession, logical entities and standards freshness; semantic review remains required. | -| Operability / incident runbook | ADEQUATE | Preserve protected-main/consumer closure and queue/provider/runner/DNS recovery evidence. | -| ADR set | ADEQUATE | ADR-0007 explicitly supersedes fixed wall-clock exit cutoffs and classifies prompt/docs/status/review/dispatch/Draft/Ready/auto-merge/commit/merge as intermediate; ADR-0004 supersedes the old Copilot-named Agent Tasks token alias. | +| Test strategy | ADEQUATE | Documentation-fitness regression coverage binds this audit, maturity mapping, historical supersession, user-redirection recovery, logical entities and standards freshness; semantic review remains required. | +| Operability / incident runbook | ADEQUATE | `CONTINUATION_RUNBOOK.md` now gives `USER_REDIRECTION_INCIDENT` an executable same-invocation recovery path, plus protected-main/consumer closure and queue/provider/runner/DNS recovery evidence. | +| ADR set | ADEQUATE | ADR-0007 supersedes fixed wall-clock exits, makes prompt/docs/status/review/dispatch/Draft/Ready/auto-merge/commit/merge intermediate, and requires multi-lane same-invocation recovery after user redirection; ADR-0004 supersedes the old Copilot-named Agent Tasks token alias. | | Traceability | ADEQUATE | Controlled maturity, current redaction lineage, whole-conversation governance, standards, implementation/test/gate ownership and explicit product/control-plane debt are mapped. | | Standards doctoring | ADEQUATE | Current final baselines include SLSA 1.2, ISO/IEC/IEEE 42010:2022, ISO/IEC/IEEE 29148:2018, ISO/IEC 25010:2023 and NIST SSDF 1.1; newer drafts are labelled informative/non-normative until final. | +The family set is therefore **not missing ADR/PRD/TRD/Architecture/UML/ERD categories**. Its remaining risk is integration/currentness: this graph is `active_pr`, not protected-main truth, and every runtime capability named below keeps its independent source/review/check/operational acceptance gate. + ## Concurrent documentation-line reconciliation - [PR #886](https://github.com/ContextualWisdomLab/.github/pull/886) is closed @@ -101,7 +107,9 @@ source/prose contradictions are corrected or explicitly classified as debt: | NVIDIA secret | Workflow secret `NVIDIA_NIM_API_KEY` maps to process `NVIDIA_API_KEY`; there is no legacy alias or routine ruleset bypass. | The historical hotfix note now names the current fail-closed contract. | | Historical scheduler cutoff | Older planning material imposed a 45-minute run and minute-35 no-new-write boundary. | `superseded`; ADR-0007 and the current automation prompt require practical-budget plus double-sweep exit proof instead. | | Historical Agent Tasks token alias | Older guidance used `COPILOT_GITHUB_TOKEN` as a GitHub API token name. | `superseded`; ADR-0004 requires purpose-bound explicit GitHub authority and prohibits reuse of the Copilot-named alias. | -| Reusable secrets | Four protected-main workflows consume undeclared reusable secrets. | `SECURITY.md` registers the debt; deploy repair is active in PR #901 and the other three remain planned. | +| Premature-stop recovery | Repeated user redirection proved that prompt/docs repair could still become the last voluntary action while other PR lanes remained executable. | ADR-0007, `CONTINUATION_RUNBOOK.md`, the hourly prompt, and `test_user_redirection_requires_same_invocation_multi_lane_continuation` require same-invocation multi-lane recovery plus a non-documentation handoff when available. | +| Reusable secrets and Pages input trust | Protected-main Pages reusable workflow lacks the accepted explicit-secret/input-safety repair. | PR #901 is `active_pr`; its current line owns explicit secret mappings plus bounded deployment-input validation and must independently satisfy exact-head reviews/checks before merge. | +| Operational SLI receipt | Finite-cardinality continuation/queue measures were previously documentation-only/distributed. | PR #905 is `active_pr` for a read-only bounded `control_plane_sli_receipt` implementation; it remains non-shipped until its own exact-head gates and protected integration pass. | | SBOM zero state | No successful inventory receipt means no fleet population was materialized. | Zero rows cannot prove zero components or no license findings. | Runtime gaps `IG-001` through `IG-008` are registered in @@ -109,13 +117,15 @@ Runtime gaps `IG-001` through `IG-008` are registered in [TRACEABILITY.md](TRACEABILITY.md). Their presence is documented debt, not implementation. -## No-soft-timeout continuation invariant +## No-soft-timeout and user-redirection continuation invariant The hourly recurrence is a continuation mechanism after genuine practical execution/tool-budget exhaustion. It is not a voluntary wall-clock timeout. The following events have zero terminal credit while any safe lane remains: prompt update, documentation assessment or update, inventory, RCA without remediation, status/report comment, review request, workflow dispatch/rerun, queued or running CI/model evidence, Draft/Ready transition, auto-merge enablement, commit, merge, document completion, protected-main acceptance of one scenario, or completion of one buyer-visible slice. -Before any voluntary termination the automation performs a fresh whole-queue sweep. If it finds an executable mutation, test, thread resolution, documentation repair, operational acceptance, issue action, merge, or bounded product/control-plane action, it executes that work and sweeps again. Only actual practical run/tool-budget exhaustion or a **second** fresh sweep proving all lanes non-actionable permits termination. +When the user reports that work was left behind, classify `USER_REDIRECTION_INCIDENT`. Prompt or documentation repair has zero completion credit. Recovery must happen in the same invocation: rebuild the entire live queue, execute substantive repository work, and if at least two independent safe lanes exist perform at least two materially distinct actions before termination is eligible. At least one must be non-documentation whenever such a lane exists. If only one lane exists, execute it and then perform two fresh whole-queue sweeps to prove no second lane is executable. + +Before any voluntary termination the automation performs a fresh whole-queue sweep. If it finds an executable mutation, test, thread resolution, documentation repair, operational acceptance, issue action, merge, or bounded product/control-plane action, it executes that work and sweeps again. Only actual practical run/tool-budget exhaustion or a **second** fresh sweep proving all lanes non-actionable permits termination. User-redirection recovery resets the sweep count. ## Evidence-to-canonicalization flow @@ -133,12 +143,14 @@ flowchart LR C -->|obsolete| K[superseded historical evidence] ``` +A documentation mutation itself is not the end of this flow. When safe non-documentation work is visible, the control plane returns from `F` to the executable queue before considering termination. + ## Exit condition for this documentation repair The documentation family is review-ready only when: 1. `docs/automation/README.md` indexes the complete canonical graph and whole-conversation audit; -2. machine tests validate controlled maturity, whole-conversation ownership, historical-loop supersession, double-exit/no-soft-timeout behavior, conceptual-vs-persisted boundaries, remediation/continuation/documentation/traceability entities, standards freshness, local links and Mermaid code fences; +2. machine tests validate controlled maturity, whole-conversation ownership, historical-loop supersession, same-invocation user-redirection/multi-lane recovery, double-exit/no-soft-timeout behavior, conceptual-vs-persisted boundaries, remediation/continuation/documentation/traceability entities, standards freshness, local links and Mermaid code fences; 3. PRD/TRD/Architecture/UML/ERD/Data Model/Security/Test/Operability/ADR/Traceability/standards doctoring have no known family-level `PARTIAL` or `MISSING` gap on the same exact head; 4. exact-head documentation/security/supply-chain checks and current review are clean; and 5. after protected merge, the canonical index is refetched from protected `main` before any `implemented_on_protected_main` claim is made. From cb43ed9d083e6df2aca5a640b861b1e159cd4826 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:54:54 +0900 Subject: [PATCH 65/69] test(automation): trace premature-stop recovery across canonical graph --- ...n_historical_loop_supersession_contract.py | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tests/test_automation_historical_loop_supersession_contract.py b/tests/test_automation_historical_loop_supersession_contract.py index 9b43f2404..d281780d2 100644 --- a/tests/test_automation_historical_loop_supersession_contract.py +++ b/tests/test_automation_historical_loop_supersession_contract.py @@ -58,3 +58,24 @@ def test_user_redirection_requires_same_invocation_multi_lane_continuation() -> assert "at least two materially distinct" in corpus assert "non-documentation" in corpus assert "two fresh whole-queue sweeps" in corpus + + +def test_user_redirection_is_visible_in_behavior_and_traceability_graphs() -> None: + """Keep premature-stop recovery represented beyond prose runbook guidance.""" + + uml = _read("UML.md") + traceability = _read("TRACEABILITY.md") + + for required in ( + "USER_REDIRECTION_INCIDENT", + "same invocation", + "non-documentation", + "two fresh whole-queue sweeps", + ): + assert required in uml + assert required in traceability + + assert "PRD-06" in traceability + assert "tests/test_automation_historical_loop_supersession_contract.py" in traceability + assert "PR #905" in traceability + assert "active_pr" in traceability From e981054ad9ac8270251c651591000d9c789147f8 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:56:07 +0900 Subject: [PATCH 66/69] docs(automation): diagram same-run premature-stop recovery --- docs/automation/UML.md | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/docs/automation/UML.md b/docs/automation/UML.md index 177f42c35..dfe652f13 100644 --- a/docs/automation/UML.md +++ b/docs/automation/UML.md @@ -1,7 +1,7 @@ # UML and behavior diagrams Status: accepted baseline -Last reviewed: 2026-08-09 +Last reviewed: 2026-08-10 These diagrams are normative at the boundary level. Detailed step names remain in workflow source and are checked through [TRACEABILITY.md](TRACEABILITY.md). @@ -272,4 +272,39 @@ flowchart LR Leaf --> Continue ``` -A documentation assessment or update is never completion by itself. Candidate conversation/planning evidence becomes canonical only after revalidation, maturity classification, Git-tracked mutation, and machine/reviewer checks. Product-specific architecture remains with its leaf repository; central documentation owns only shared automation and interface contracts. \ No newline at end of file +A documentation assessment or update is never completion by itself. Candidate conversation/planning evidence becomes canonical only after revalidation, maturity classification, Git-tracked mutation, and machine/reviewer checks. Product-specific architecture remains with its leaf repository; central documentation owns only shared automation and interface contracts. + +## 13. User-redirection incident recovery + +```mermaid +sequenceDiagram + participant U as User/operator + participant A as Maintainer automation + participant G as GitHub live queue + participant D as Documentation control plane + + U->>A: Work remained; prior invocation stopped early + A->>A: Classify USER_REDIRECTION_INCIDENT + A->>G: Re-fetch protected main, all PRs/issues, heads, live bases, writers, gates + G-->>A: Fresh executable and deferred lanes + opt Control contract was incomplete + A->>D: Repair prompt/docs/test contract + D-->>A: META_INTERMEDIATE only; zero completion credit + end + alt At least two independent safe lanes exist + A->>G: Execute first substantive safe action + A->>G: Execute second materially distinct safe action + Note over A,G: At least one action is non-documentation when available + else Exactly one safe lane exists + A->>G: Execute the sole safe action + end + A->>G: Fresh whole-queue sweep 1 + G-->>A: Current queue + alt Any executable lane exists + A->>G: Execute and reset sweep count + else No executable lane + A->>G: Fresh whole-queue sweep 2 from new reads + end +``` + +`USER_REDIRECTION_INCIDENT` is never terminal. Recovery must occur in the **same invocation**. Prompt, documentation, RCA, status, one commit, or one merge has zero completion credit while another safe lane exists. If two independent safe lanes exist, at least two materially distinct actions are required and a safe **non-documentation** lane must be included when available. When only one lane exists, two fresh whole-queue sweeps must prove no second executable lane before voluntary termination. \ No newline at end of file From c443f6439571142e62cc5552498bb794a8232c71 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 12:57:20 +0900 Subject: [PATCH 67/69] docs(automation): trace live continuation and operability repair lines --- docs/automation/TRACEABILITY.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/automation/TRACEABILITY.md b/docs/automation/TRACEABILITY.md index b93ed6625..fe45b9bf1 100644 --- a/docs/automation/TRACEABILITY.md +++ b/docs/automation/TRACEABILITY.md @@ -1,7 +1,7 @@ # Requirements and evidence traceability Status: living authoritative matrix -Last reviewed: 2026-08-09 +Last reviewed: 2026-08-10 Canonical maturity states are defined in [DOCUMENTATION_AUDIT.md](DOCUMENTATION_AUDIT.md): `implemented_on_protected_main`, `active_pr`, `accepted_architecture`, `planned`, `research_only`, `superseded`, and `out_of_scope`. Every maturity cell contains exactly one canonical state; explanatory qualifiers belong in the evidence or closure columns. A row marked `implemented_on_protected_main` still requires fresh exact-head evidence for each later change; a row marked `active_pr` is never shipped truth. @@ -17,10 +17,10 @@ Repository locators below are literal paths with no globs. External canaries and | `PRD-03` authority separation | `TRD-AUTH-01`, ADR-0005/0015/0016 | `scripts/ci/opencode_review_normalize_output.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_opencode_review_normalize_output.py`; `tests/test_pr_review_merge_scheduler.py` | distinct Check/Status/Review/thread/ruleset objects plus scheduler v2 receipt | governance maintainer / every merge | `implemented_on_protected_main` | | `PRD-04` independent approval | `TRD-AUTH-01`, ADR-0005/0016 | `scripts/ci/audit_central_required_workflows.py`; `scripts/ci/pr_review_merge_scheduler.py` | `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_scheduler_independent_approval_gate.py` | live review decision, qualifying non-author formal review and ruleset | repository admins / before merge | `active_pr` | | `PRD-05` writer lease | `TRD-WRITE-01`, ADR-0001 | `scripts/ci/pr_review_merge_scheduler.py`; `.github/workflows/pr-review-autofix.yml`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_pr_review_merge_scheduler.py`; `tests/test_pr_review_fix_scheduler.py` | live branch/head observation plus expected-head GitHub response | active branch writer / immediately before mutation | `accepted_architecture` | -| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; `AGENTS.md`; `docs/automation/CONTINUATION_RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; external continuation receipt | 15-minute queue sweep plus double fresh whole-queue exit sweep | automation operator / before run exit | `accepted_architecture` | +| `PRD-06` work conservation | `TRD-RUN-01`, ADR-0007 | `.github/workflows/pr-review-merge-scheduler.yml`; `AGENTS.md`; `docs/automation/CONTINUATION_RUNBOOK.md`; `docs/automation/UML.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_automation_documentation_contract.py`; `tests/test_automation_historical_loop_supersession_contract.py`; external continuation receipt | queue sweep plus `USER_REDIRECTION_INCIDENT` same invocation recovery; when two safe lanes exist at least two materially distinct actions with a non-documentation action when available; otherwise two fresh whole-queue sweeps before exit | automation operator / after every meta action and before run exit | `accepted_architecture` | | `PRD-07` RCA + feasible remedy | `TRD-RETRY-01`, ADR-0003 | `ci-review-prompt.md`; `code-reviewer-prompt.md`; `docs/automation/RUNBOOK.md` | `tests/test_adversarial_evidence.py`; `tests/test_opencode_adversarial_receipts.py`; `tests/test_automation_documentation_contract.py` | source-backed finding/RCA, `remediation_candidate` evidence, exact-head repair and protected recovery receipt | incident owner / before closure | `accepted_architecture` | | `PRD-08` safe diagnostic redaction | `TRD-LOG-01`, ADR-0009 | `scripts/ci/redact_sensitive_log.py`; `scripts/ci/sandboxed_verify.py`; `scripts/ci/sandboxed_web_e2e.py` | `tests/test_opencode_security_boundaries.py`; `tests/test_sandboxed_verify.py`; `tests/test_sandboxed_web_e2e.py` | exact-head redaction checks, then protected-main/consumer result markers | sandbox owners / Draft clean-history successor #906 + protected consumer canary | `active_pr` | -| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/DOCUMENTATION_AUDIT.md`; `docs/automation/README.md`; `docs/automation/EVENT_CONTRACTS.md`; `ARCHITECTURE.md` | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | +| `PRD-09` documentation graph | `TRD-IF-01`, ADR-0007/0008 | `docs/automation/DOCUMENTATION_AUDIT.md`; `docs/automation/README.md`; `docs/automation/EVENT_CONTRACTS.md`; `ARCHITECTURE.md` | `tests/test_automation_documentation_contract.py`; `tests/test_pr_governance_audit_contract.py`; `tests/test_automation_historical_loop_supersession_contract.py` | documentation contract, link/diagram/standards audit, `git diff --check`, reviewer approval | architecture maintainers / current canonical documentation PR | `active_pr` | | `PRD-10` protected-main acceptance | `TRD-REV-01`, ADR-0006 | `.github/workflows/opencode-review.yml`; `.github/workflows/pr-review-merge-scheduler.yml`; `docs/automation/RUNBOOK.md` | `tests/test_required_workflow_queue_contract.py`; `tests/test_pr_review_merge_scheduler.py`; real-consumer canary receipt | protected commit, workflow source, scenario, negative control, rollback receipt | service owner / after protected integration | `accepted_architecture` | | `PRD-11` thin modular consumers | `TRD-IF-01`, ADR-0008/0012/0014 | `.github/workflows/opencode-review.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml`; `.github/workflows/pr-review-fix-scheduler.yml`; `.github/workflows/deploy-pages.yml` | `tests/test_required_workflow_queue_contract.py`; `tests/test_central_required_workflow_ruleset_audit.py`; `tests/test_automation_documentation_contract.py` | ruleset source/ref plus positive/negative consumer run | interface owners / before breaking change | `accepted_architecture` | | `PRD-12` model credential policy | `TRD-SEC-01`, ADR-0004/0011 | `.github/workflows/opencode-review-dispatch.yml`; `.github/workflows/noema-review.yml`; `.github/workflows/strix.yml` | `tests/test_opencode_security_boundaries.py`; `tests/test_opencode_model_pool_runner.py`; `tests/test_strix_nvidia_nim_not_found_fallback.py` | per-job secret map, provider result, redacted log, deterministic gates | security/workflow owners / every provider path change | `implemented_on_protected_main` | @@ -35,11 +35,11 @@ Repository locators below are literal paths with no globs. External canaries and | TRD | events, revisions, evidence authority, permissions, retries, leases, redaction | file/index/source-name contract | `active_pr` | | Architecture | contexts, components, planes, trust/failure boundaries | Mermaid and source-name contract | `active_pr` | | Data model / ERD | conceptual vs persisted and evidence/governance/remediation/continuation/documentation entities | entity/naming/cardinality/link contract | `active_pr` | -| UML | component, two sequences, state, authority, topology, retry, sandbox, lease/continuation | diagram-section/fence contract | `active_pr` | +| UML | component, two sequences, state, authority, topology, retry, sandbox, lease/continuation, documentation→source handoff, and `USER_REDIRECTION_INCIDENT` same invocation recovery | diagram-section/fence + historical-loop contract | `active_pr` | | Security / threat model | required attack paths, privacy alternative, residual risk | term/link contract | `active_pr` | | Test strategy | realistic gate, security, performance, consumer proof, documentation fitness | term/link/standards contract | `active_pr` | -| Operability / runbook | SLI/SLO, queue/provider failures, exact queries, rollback, retention, receipt, closure/reopen | term/link contract | `active_pr` | -| Whole-conversation audit | maturity states, central/leaf ownership, no-soft-timeout, fitness findings | indexed file + documentation contract | `active_pr` | +| Operability / runbook | SLI/SLO, queue/provider failures, exact queries, rollback, retention, receipt, closure/reopen, same-invocation premature-stop recovery | term/link + historical-loop contract | `active_pr` | +| Whole-conversation audit | maturity states, central/leaf ownership, no-soft-timeout, user-redirection recovery, fitness findings | indexed file + documentation contract | `active_pr` | | ADR set | sixteen indexed decisions with alternatives, tests, rollback, supersession | index/file/section contract | `active_pr` | | Standards doctoring | final/draft version discipline and APA 7 references | standards-freshness contract | `active_pr` | @@ -56,9 +56,9 @@ Repository locators below are literal paths with no globs. External canaries and | Decision / evidence source | Canonical artifact | Maturity | Rule | |---|---|---|---| -| Repeated instruction to continue after merge/review/check/doc work | ADR-0007; `DOCUMENTATION_AUDIT.md` | `active_pr` | Meta/control actions and one substantive result are intermediate while a safe lane remains; hourly recurrence is not a soft timeout. | +| Repeated instruction to continue after merge/review/check/doc work | ADR-0007; `CONTINUATION_RUNBOOK.md`; `UML.md`; `DOCUMENTATION_AUDIT.md` | `active_pr` | Meta/control actions and one substantive result are intermediate while a safe lane remains; hourly recurrence is not a soft timeout. A user-reported premature stop is `USER_REDIRECTION_INCIDENT`: recovery occurs in the same invocation, includes a non-documentation lane when available, and ends only after the required work plus two fresh whole-queue sweeps or genuine tool/runtime exhaustion. | | Conversation/planning packs across TEPP, OriginWeave, EmbedRelay, MHTML ETL, LifeOS and leaf products | `DOCUMENTATION_AUDIT.md`; central/leaf Architecture | `active_pr` | Revalidate shared automation decisions; leaf product semantics remain `out_of_scope` here and are not copied into central architecture. | -| ADR/PRD/TRD/UML/ERD completeness request | documentation graph + this traceability matrix | `active_pr` | A prose assessment alone is insufficient; gaps become canonical GitHub mutations and machine contracts. | +| ADR/PRD/TRD/UML/ERD completeness request | documentation graph + this traceability matrix | `active_pr` | A prose assessment alone is insufficient; gaps become canonical GitHub mutations and machine contracts, and documentation repair returns to non-documentation execution whenever a safe lane exists. | | Architecture/requirements/product-quality standards | `docs/doctoring/automation-control-plane-standards.md` | `active_pr` | Final publications are normative baselines; drafts are informative only. | | Logical evidence-store relationships | `DATA_MODEL.md`; `ERD.md` | `active_pr` | Conceptual model only; material persistence requires a separate ADR and privacy/tenancy/DR design. | @@ -67,18 +67,18 @@ Repository locators below are literal paths with no globs. External canaries and | Gap | Risk | Next bounded evidence | Maturity | |---|---|---|---| | No persisted cross-repository continuation/writer ledger | collision and queue-starvation reconstruction | decide whether GitHub Project/artifact state is sufficient before proposing persistence | `planned` | -| SLI aggregation incomplete | buyer cannot quantify reliability | emit bounded run metrics keyed by repo/head/workflow/attempt | `planned` | +| SLI aggregation incomplete | buyer cannot quantify reliability | [PR #905](https://github.com/ContextualWisdomLab/.github/pull/905) implements the read-only finite-cardinality `control_plane_sli_receipt`; require exact-head 100% statement/branch/docstring gates, current review, protected integration, and real receipt canaries before claiming operational coverage | `active_pr` | | Sandbox total-output/service-file quota incomplete | memory/disk DoS | fail-first hostile-output quota tests and separate implementation PR | `planned` | -| Legacy `deploy-pages.yml` inherited-secret contract | excess secret exposure | [PR #901](https://github.com/ContextualWisdomLab/.github/pull/901) explicit interface plus real deployment positive/negative canary | `active_pr` | +| Legacy `deploy-pages.yml` secret/input trust boundary | excess secret exposure and unsafe reusable inputs | [PR #901](https://github.com/ContextualWisdomLab/.github/pull/901) explicit two-secret interface plus bounded `project_name`/`build_dir`/`custom_domain` validation and real deployment positive/negative canary | `active_pr` | | Undeclared scheduler/rebase reusable secrets | caller ambiguity and excess exposure | declare and map secrets in the three exact workflows registered by `SECURITY.md`; add negative caller tests and consumer receipts | `planned` | -| Operational receipts distributed | incident closure hard to audit | dated protected-main/consumer receipt index | `planned` | +| Operational receipts distributed | incident closure hard to audit even with a local SLI receipt | integrate accepted #905 receipt with dated protected-main/consumer acceptance evidence without inventing a new persistence authority | `planned` | | Project #1 requires GraphQL/`gh` project scope | agents without that capability cannot acquire visible project item | add supported connector or ensure native PR auto-add; never invent state | `planned` | | Master context current-state section is dated | stale operational narrative | separate timeless context from generated/daily live-state appendix | `planned` | | Explicit dispatch/result schema versions not universal | compatibility ambiguity | version all central dispatch/result contracts with migration/negative tests | `planned` | ## 6. Audited implementation-gap lineage -An issue records planned work; it does not implement the work it describes. +An issue records planned work; it does not implement the work it describes. An open PR moves a gap to `active_pr` only for that exact change line; it is not protected-main implementation. | Gap | Live object | Current boundary and required closure | Maturity | |---|---|---|---| @@ -89,7 +89,7 @@ An issue records planned work; it does not implement the work it describes. | `IG-005` authoritative Strix result | [Issue #891](https://github.com/ContextualWisdomLab/.github/issues/891) | Tracks planned implementation of a fail-closed terminal gate when authoritative scan evidence is absent. | `planned` | | `IG-006` merge mode and mutation authority | [Issue #892](https://github.com/ContextualWisdomLab/.github/issues/892) | Tracks planned implementation of one executable credential/mode authority table. | `planned` | | `IG-007` recoverable mention claim | [Issue #893](https://github.com/ContextualWisdomLab/.github/issues/893) | Tracks planned recoverable claim states and fencing without weakening completed-request idempotency. | `planned` | -| `IG-008` truthful scheduler terminal result | [Issue #894](https://github.com/ContextualWisdomLab/.github/issues/894) | Tracks a non-passing terminal result after preserving the queue summary when a material action is `action_error`. | `planned` | +| `IG-008` truthful scheduler terminal result | [PR #899](https://github.com/ContextualWisdomLab/.github/pull/899) | The active repair returns a non-zero process result only after the bounded scan and structured summary when a material decision is `action_error`; exact-head checks/review and protected integration remain required. | `active_pr` | ## 7. Standards traceability From 8dffbf428fddf52de14ccea3d085936a208434a9 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 13:05:05 +0900 Subject: [PATCH 68/69] test(automation): bind IG-008 to active scheduler repair --- tests/test_automation_documentation_contract.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test_automation_documentation_contract.py b/tests/test_automation_documentation_contract.py index 8f04576f7..2c6165066 100644 --- a/tests/test_automation_documentation_contract.py +++ b/tests/test_automation_documentation_contract.py @@ -579,8 +579,8 @@ def test_current_lineage_threat_maturity_and_gap_objects_are_exact() -> None: "planned", ), "IG-008": ( - "https://github.com/ContextualWisdomLab/.github/issues/894", - "planned", + "https://github.com/ContextualWisdomLab/.github/pull/899", + "active_pr", ), } traceability_lines = traceability.splitlines() From 26c7a8c4a6a7f593e1cbbb88f66fb2a604f8309b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 10 Aug 2026 13:06:12 +0900 Subject: [PATCH 69/69] docs(automation): bind scheduler terminal-gap closure to PR 899 --- docs/automation/TRD.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/automation/TRD.md b/docs/automation/TRD.md index edc97e7f9..100f74ec2 100644 --- a/docs/automation/TRD.md +++ b/docs/automation/TRD.md @@ -1,7 +1,7 @@ # Technical requirements — CWL automation control plane Status: accepted baseline -Last reviewed: 2026-08-09 +Last reviewed: 2026-08-10 ## 1. System boundary @@ -240,7 +240,7 @@ type and maturity are mapped in [TRACEABILITY.md](TRACEABILITY.md). | `IG-005` | A provider/backend outage can leave Strix without authoritative exact-head scan evidence while a transport/check shape may be skipped or neutral. | Add the fail-closed gate in [Issue #891](https://github.com/ContextualWisdomLab/.github/issues/891). | | `IG-006` | Workflow-token preference coexists with App/token fallbacks, and merge-mode/credential authority is not one executable table. | Align authority under [Issue #892](https://github.com/ContextualWisdomLab/.github/issues/892). | | `IG-007` | A post-claim pre-forward mention failure is an at-most-once dead-letter during retention and needs a new trusted comment. | Add recoverable states/fencing under [Issue #893](https://github.com/ContextualWisdomLab/.github/issues/893). | -| `IG-008` | The scheduler records PR-local mutation failure as `action_error` and continues, but the CLI can still return success after the scan. | Preserve the summary and return non-passing under [Issue #894](https://github.com/ContextualWisdomLab/.github/issues/894). | +| `IG-008` | The scheduler records PR-local mutation failure as `action_error` and continues; the accepted repair preserves the complete bounded scan and summary, then returns non-passing only when a material decision is `action_error`. | Integrate and accept [PR #899](https://github.com/ContextualWisdomLab/.github/pull/899), then verify the protected scheduler receipt and process exit contract. | An accepted ADR, diagram, prompt, issue, or this register cannot promote these gaps to `implemented_on_protected_main`.