feat(operations): audit external scheduler continuation evidence - #97
Open
seonghobae wants to merge 24 commits into
Open
feat(operations): audit external scheduler continuation evidence#97seonghobae wants to merge 24 commits into
seonghobae wants to merge 24 commits into
Conversation
📝 WalkthroughWalkthrough외부 스케줄러 증거 감사 문서와 평가기를 추가했다. CLI는 입력 파일을 fail-closed로 검증하고, PASS 또는 FAIL 보고서를 권한 제한 임시 파일에서 원자적으로 게시한다. 테스트와 커버리지 설정도 새 도구를 대상으로 갱신했다. Changes외부 스케줄러 감사
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
16 tasks
seonghobae
marked this pull request as ready for review
August 10, 2026 12:05
This was referenced Aug 10, 2026
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
docs/external-hourly-scheduler-evidence-audit.md (1)
1-137: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift운영 설명을 한국어로 작성하십시오.
이 파일은 운영 증거 계약과 운영자 명령을 설명합니다. 현재 설명 문단이 영어입니다. 코드 블록과 고정 식별자는 유지하고, 운영 설명은 한국어로 번역하십시오.
As per coding guidelines,
docs/**/*.{md,mdx}: Keep documentation in the language appropriate to the document; the operational and sales/acquisition documentation is largely Korean.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/external-hourly-scheduler-evidence-audit.md` around lines 1 - 137, Translate the prose in external-hourly-scheduler-evidence.md into Korean, including headings, explanatory paragraphs, contract descriptions, interpretation, sufficiency, and references where appropriate. Preserve all code blocks, JSON, shell commands, fixed identifiers, field names, enum values, URLs, and exact contract literals unchanged. Keep the document’s operational meaning and structure intact while ensuring the operational and acquisition-related narrative is Korean.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/external-scheduler-evidence-audit.mjs`:
- Around line 150-154: Update createValidationReport() so FAIL reports retain
only derived checks and failure codes, removing copied evidence fields such as
scheduler_task_identity, prompt_sha256, protected_main_sha, scheduled_at, and
started_at; add a regression test proving sensitive values in failed evidence
are absent from the generated report.
- Around line 81-82: Replace the direct JSON.parse(text) path after
fatalUtf8Decoder.decode(bytes) with a duplicate-key-aware parser or scanner that
detects decoded duplicate keys within each object before constructing the JSON
result. Reject the evidence when any object contains a duplicate key, including
nested objects, while preserving valid arrays and uniquely keyed objects.
In `@scripts/lib/external-scheduler-evidence-audit.mjs`:
- Line 8: Update FORBIDDEN_FIELD_PATTERN validation to normalize field names
from camelCase to snake_case before matching, so accessToken, clientSecret,
privateKey, and hiddenReasoning are rejected. Extend the forbidden patterns to
include api_key and common credential aliases, while preserving rejection of
existing sensitive fields. Add regression tests covering these camelCase and
alias cases.
- Around line 205-210: Update the generic_error_resumed_action_missing
validation to require resumed_action_identity to match an action_identity from
github_actions_performed when generic_error_observed is true, while preserving
the existing format validation. Add a regression test covering evidence with a
non-existent resumed action identity and assert that it is rejected.
- Around line 261-270: Update the materiallyDistinct validation in the external
scheduler evidence audit so the action-kind diversity requirement applies only
when at least two safe lanes are present. When there is one safe lane, allow
multiple actions with the same action_kind to pass; preserve the existing
requirement for two or more safe lanes and the current failure message behavior.
---
Nitpick comments:
In `@docs/external-hourly-scheduler-evidence-audit.md`:
- Around line 1-137: Translate the prose in
external-hourly-scheduler-evidence.md into Korean, including headings,
explanatory paragraphs, contract descriptions, interpretation, sufficiency, and
references where appropriate. Preserve all code blocks, JSON, shell commands,
fixed identifiers, field names, enum values, URLs, and exact contract literals
unchanged. Keep the document’s operational meaning and structure intact while
ensuring the operational and acquisition-related narrative is Korean.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1b942988-ed36-4f46-b7a4-2a202fe4276d
📒 Files selected for processing (11)
CHANGELOG.mddocs/external-hourly-scheduler-evidence-audit.mdpackage.jsonscripts/external-scheduler-evidence-audit.mjsscripts/lib/external-scheduler-evidence-audit.mjstest/external-scheduler-evidence-audit.test.tstest/external-scheduler-evidence-cli-defaults.test.tstest/external-scheduler-evidence-cli.test.tstest/external-scheduler-evidence-defensive-branches.test.tstest/production-coverage-policy.test.tsvitest.config.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Turn issue #96's external hourly-scheduler control-plane gap into repository-owned, fail-closed, executable evidence validation. This PR validates retained evidence bytes and exposes an operator command; it does not operate the provider-side scheduler.
Current exact identity
mainatc85d710804139c0697d7ef8fa47d02b1389e6d84;feat/external-scheduler-evidence-audit;a84d490c1f6596a49d4f29c41babf4996f9ad86f;Test-first and review-hardening lineage
The implementation was developed through explicit RED boundaries for schema/operator absence, production coverage, and CLI safety/testability. CodeRabbit subsequently identified valid current defects around duplicate decoded JSON keys, failed-report data retention, camelCase/credential aliases, resumed-action identity binding, and materially-distinct action semantics. Those inline findings are now addressed/resolved on the current lineage.
The latest exact-head correction,
a84d490c1f6596a49d4f29c41babf4996f9ad86f(fix(operations): fail closed without no-follow opens), removes the permissive0fallback forO_NOFOLLOW: platforms that cannot provide a real no-follow file-open flag now fail closed rather than silently weakening the evidence-file boundary.Current exact-head technical evidence
For unchanged head
a84d490c1f6596a49d4f29c41babf4996f9ad86f:cirun31389470222: terminal success;reviewer-cirun31389470263: terminal success;Security Scanrun31389470311: terminal success under its scanner/revision semantics;COMMENTEDand was submitted against predecessor head53df217b980616916b6f3d85d0aaf4b8942c5e36, not a qualifying independent approval;Technical CI/reviewer/scanner success remains separate from formal review, live ruleset authority, operational scheduler evidence, release/deployment acceptance, and acquisition evidence.
Implemented evidence boundary
npm run operations:external-scheduler-evidence -- <evidence.json>remains the repository-owned operator interface. The retained report is evidence only and cannot satisfy GitHub checks, formal approval, merge, release, deployment, or acquisition authority.Documentation / architecture boundary
PR #71 remains the single canonical whole-product PRD/TRD/Architecture/ADR/UML/ERD/Traceability authority. This PR adds the missing executable operational evidence contract rather than a parallel architecture graph. No physical relational ERD is asserted for this JSON evidence boundary.
Provider-side acceptance still owned by #96
This PR does not prove live provider configuration. Issue #96 still requires independently retained evidence for one enabled hourly task identity/configuration, prompt digest, duplicate disablement, generic-error recovery, work-conserving execution receipts, writer-safety refetches, and two clean fresh exit sweeps or a concrete practical invocation-budget boundary.
Acquisition / merge boundary
Noema still lacks real production KPI/provenance, immutable release-publication and deployment/attestation evidence, production environment governance, revenue/customer evidence, and legal/transfer/IP evidence required by the final acquisition gate. No outbound license is chosen here.
Merge remains fail closed on actual live #27 governance and a qualifying independent non-author formal-review path under #29 where required. No self-approval, protection weakening, synthetic status, repair/self-modifying workflow, version bump, release, deployment, licensing decision, or acquisition-ready claim is made.
Related: #27, #29, #71, #80, #96