-
Notifications
You must be signed in to change notification settings - Fork 0
docs(automation): establish canonical control-plane architecture baseline #886
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
aee3c85
b235c3d
8eff2ab
4b9e064
4405772
3772d48
c625eca
262d7cc
126b90c
ca868fc
5c47950
6671dc9
73408da
fd15f09
7fa7876
fbb8bca
8302e65
ad94279
767c8c1
e2beec7
6caa7b7
b96344e
fc1003b
2b226fa
5f3d864
142d90e
a2d77a0
bab4655
b099d5f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| name: Automation Documentation Quality CI | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: [main] | ||
| paths: | ||
| - "docs/automation/**" | ||
| - "tests/test_automation_documentation.py" | ||
| - ".github/workflows/automation-documentation-quality-ci.yml" | ||
| - "AGENTS.md" | ||
| - "CLAUDE.md" | ||
| - "CHANGELOG.md" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: automation-documentation-quality-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| documentation-contract: | ||
| name: documentation-contract | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - name: Checkout exact source revision | ||
| uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha || github.sha }} | ||
| persist-credentials: false | ||
| - name: Verify canonical automation documentation | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| test "$(git rev-parse HEAD)" = "${{ github.event.pull_request.head.sha || github.sha }}" | ||
| python tests/test_automation_documentation.py | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "Tracked workflow files:"
git ls-files '.github/workflows/**/*.yml' '.github/workflows/**/*.yaml' | sed -n '1,120p'
echo
echo "Target workflow:"
if [ -f .github/workflows/automation-documentation-quality-ci.yml ]; then
nl -ba .github/workflows/automation-documentation-quality-ci.yml | sed -n '1,120p'
else
echo "missing .github/workflows/automation-documentation-quality-ci.yml"
fi
echo
echo "Tests files:"
git ls-files tests | sed -n '1,80p'
echo
echo "Search tests/automation and test commands:"
rg -n "unit|unittest|pytest|tox|python (tests/|-m unittest)|test_automation|automation-documentation|document" . -g '!*.png' -g '!*.jpg' -g '!*.svg' -g '!package-lock.json' -g '!pnpm-lock.yaml' -g '!yarn.lock' | sed -n '1,240p'Repository: ContextualWisdomLab/.github Length of output: 248 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "Tracked workflow files:"
git ls-files '.github/workflows/**/*.yml' '.github/workflows/**/*.yaml' 2>/dev/null || git ls-files '.github/workflows/*' 2>/dev/null || true
echo
echo "Target workflow:"
if [ -f .github/workflows/automation-documentation-quality-ci.yml ]; then
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/automation-documentation-quality-ci.yml | sed -n '1,120p'
else
echo "missing .github/workflows/automation-documentation-quality-ci.yml"
fi
echo
echo "Tests files:"
git ls-files tests 2>/dev/null | sed -n '1,120p' || true
echo
echo "Search relevant keywords in workflows/tests:"
python3 - <<'PY'
import pathlib, re
for p in pathlib.Path('.github').rglob('*'):
if p.is_file() and p.suffix in {'.yml','.yaml'} and p.parts[:2] == ('.github','workflows'):
txt=p.read_text(errors='ignore')
if any(s in txt for s in ['automation-documentation','pull_request_target','python tests/test_automation_documentation.py','run: python']):
print(f'--- {p}')
for i,line in enumerate(txt.splitlines(),1):
if ('automation-documentation' in line or 'pull_request_target' in line or 'python tests/test_automation_documentation.py' in line or re.search(r'run:\s*python', line)):
print(f'{i}: {line}')
if pathlib.Path('tests').exists():
for p in pathlib.Path('tests').rglob('*.py'):
print(f'--- {p}')
txt=p.read_text(errors='ignore')
for i,line in enumerate(txt.splitlines(),1):
if 'unittest' in line or 'pytest' in line or 'def test_' in line or 'class ' in line or 'main' in line:
print(f'{i}: {line}')
PYRepository: ContextualWisdomLab/.github Length of output: 50384 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "Candidate dependency/config files:"
git ls-files | rg '(^|/)(pyproject\.toml|setup\.py|setup\.cfg|tox\.ini|pytest\.ini|requirements.*\.txt|Pipfile|uv\.lock|poetry\.lock)$' || true
echo
echo "Test command declarations:"
rg -n "pytest|unittest|tox|coverage|python -m|safe_pytest_command" .github tests pyproject.toml setup.cfg tox.ini pytest.ini 2>/dev/null || true
echo
echo "Target test entrypoint contents:"
if [ -f tests/test_automation_documentation.py ]; then
awk '{printf "%6d\t%s\n", NR, $0}' tests/test_automation_documentation.py | sed -n '1,160p'
fi
echo
echo "Programmatic check: does automation workflow execute only tests/test_automation_documentation.py in repository paths?"
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/automation-documentation-quality-ci.yml")
t = p.read_text()
lines = [i+1 for i,l in enumerate(t.splitlines()) if "python tests/test_automation_documentation.py" in l]
files_in_tests = sorted([str(p) for p in (Path("tests")).rglob("*.py")])
print("lines_invoking_target:", lines)
print("files_invoking_target_in_repo_paths:", [k for k,l in enumerate(t.splitlines(),1) if "python " in l or "pytest" in l or "unittest" in l])
print("tests_py_count:", len(files_in_tests))
print("target_py_exists:", Path("tests/test_automation_documentation.py").exists())
for k,l in enumerate(t.splitlines(),1):
if ("python " in l or "pytest" in l or "unittest" in l) and "python tests/test_automation_documentation.py" not in l:
print(f"OTHER_TEST_COMMAND_AT_{k}: {l.strip()}")
PYRepository: ContextualWisdomLab/.github Length of output: 50384 전체 테스트 스위트를 실행하세요.
🤖 Prompt for AI AgentsSource: Coding guidelines |
||
| git diff --exit-code | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,110 @@ | ||||||||||||||
| # CWL Automation Control Plane Architecture | ||||||||||||||
|
|
||||||||||||||
| Status: active_pr | ||||||||||||||
|
|
||||||||||||||
| ## Bounded contexts | ||||||||||||||
|
|
||||||||||||||
| ```mermaid | ||||||||||||||
| flowchart LR | ||||||||||||||
| X[External orchestration plane] --> O[Organization control plane] | ||||||||||||||
| O --> Q[Live executable queue] | ||||||||||||||
| Q --> E[Evidence collectors] | ||||||||||||||
| E --> R[RCA and feasibility] | ||||||||||||||
| R --> W[Repository writer lease] | ||||||||||||||
| W --> P[PR maintenance] | ||||||||||||||
| W --> D[Product development] | ||||||||||||||
| P --> G[Gate evaluator] | ||||||||||||||
| D --> G | ||||||||||||||
| G --> M[Merge authority] | ||||||||||||||
| M --> A[Protected-main acceptance] | ||||||||||||||
| O -. read-only .-> F[Fleet auditor] | ||||||||||||||
| O --> L[Thin leaf callers] | ||||||||||||||
| O --> C[Canonical documentation graph] | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| The External orchestration plane owns scheduled agent invocation, continuation policy, and external writer-lease state. The GitHub execution and evidence plane owns repositories, PRs, branches, workflows, checks, statuses, reviews, rulesets, artifacts, and merge/release evidence. These planes interact but are not interchangeable authorities. | ||||||||||||||
|
|
||||||||||||||
| The central repository owns reusable GitHub automation semantics and the canonical documentation graph. Product repositories own product code and thin local caller/contracts. Dedicated repository loops own writes to their repositories; the fleet auditor remains read-only. A conversational decision or automation prompt is an input to reconciliation, not a replacement for repository documentation. | ||||||||||||||
|
|
||||||||||||||
| ## Plane boundaries | ||||||||||||||
|
|
||||||||||||||
| ### External orchestration plane | ||||||||||||||
|
|
||||||||||||||
| Approved scheduled agent/orchestrator services may hold the authoritative writer lease for a repository and may choose the next execution lane. Their enabled state, schedule, prompt/configuration, and most recent run are external control records. They cannot turn a GitHub check, review, merge, or protected-main runtime result into success by declaration. | ||||||||||||||
|
|
||||||||||||||
| ### GitHub execution and evidence plane | ||||||||||||||
|
|
||||||||||||||
| GitHub-native scheduled/manual/event workflows, reusable workflows, repository-local thin callers, PRs, checks, review threads, artifacts, rulesets, and protected branches are the authoritative source of repository state and execution evidence. Cross-repository reusable behavior belongs centrally unless an accepted architecture decision assigns it elsewhere. | ||||||||||||||
|
|
||||||||||||||
| ### Canonical documentation plane | ||||||||||||||
|
|
||||||||||||||
| `docs/automation/**`, the ADR index, AGENTS/CLAUDE/CHANGELOG links, and their machine-checkable fitness gate are the durable design authority. Pull-request bodies, incident comments, conversation history, and downloadable planning artifacts supply evidence and candidate decisions, but material durable decisions must be reconciled into this graph with explicit maturity. | ||||||||||||||
|
|
||||||||||||||
|
Comment on lines
+39
to
+42
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 문서 권한의 단일 출처를 명확히 하세요. Line 41은 As per coding guidelines, 수정 예시- `docs/automation/**`, the ADR index, AGENTS/CLAUDE/CHANGELOG links, and their machine-checkable fitness gate are the durable design authority.
+ `docs/automation/**` and the ADR index are the durable design authority. AGENTS.md, CLAUDE.md, and CHANGELOG.md only link to, summarize, or trace that graph. Their machine-checkable fitness gate verifies the canonical graph.📝 Committable suggestion
Suggested change
🤖 Prompt for AI AgentsSource: Coding guidelines |
||||||||||||||
| ## Trust boundaries | ||||||||||||||
|
|
||||||||||||||
| 1. GitHub event and repository metadata are inputs, not proof of current state until refetched. | ||||||||||||||
| 2. External automation configuration is lease/control evidence, not source/check/review evidence. | ||||||||||||||
| 3. PR-controlled source, comments, logs, and model prompts are untrusted content. | ||||||||||||||
| 4. Workflow source is trusted only when immutably identified. | ||||||||||||||
| 5. Checks, statuses, formal reviews, model judgments, merge authority, and protected-main runtime evidence are separate channels. | ||||||||||||||
| 6. Model credentials are privileged secrets and are not needed for deterministic gates. | ||||||||||||||
| 7. A source merge and a protected-main runtime execution are separate acceptance boundaries. | ||||||||||||||
| 8. Conversation history and active-PR documentation cannot be presented as protected-main implementation. | ||||||||||||||
|
|
||||||||||||||
| ## Failure domains | ||||||||||||||
|
|
||||||||||||||
| - Repository-local product/test defect. | ||||||||||||||
| - Central reusable-workflow defect. | ||||||||||||||
| - External scheduler/orchestrator continuation defect. | ||||||||||||||
| - Reviewer methodology/provider failure. | ||||||||||||||
| - Runner/network/bootstrap infrastructure failure. | ||||||||||||||
| - Permission/governance configuration. | ||||||||||||||
| - Writer conflict or stale evidence. | ||||||||||||||
| - Documentation authority or traceability drift. | ||||||||||||||
|
|
||||||||||||||
| A failure freezes only the smallest affected domain/lane unless evidence proves a broader boundary. | ||||||||||||||
|
|
||||||||||||||
| ## Control flow | ||||||||||||||
|
|
||||||||||||||
| ```mermaid | ||||||||||||||
| sequenceDiagram | ||||||||||||||
| participant X as External Orchestrator | ||||||||||||||
| participant C as Control Plane | ||||||||||||||
| participant G as GitHub | ||||||||||||||
| participant R as Repository | ||||||||||||||
| participant V as Reviewer/Checks | ||||||||||||||
| X->>C: Start finite invocation with live continuation policy | ||||||||||||||
| C->>G: Refetch head, live base, policy, evidence | ||||||||||||||
| C->>C: RCA + distinct remedies + feasibility | ||||||||||||||
| alt safe repository mutation | ||||||||||||||
| C->>R: Exact-head/blob/ref-bound change | ||||||||||||||
| R->>V: Run exact-head verification | ||||||||||||||
| V-->>C: Evidence by authority channel | ||||||||||||||
| else lane blocked | ||||||||||||||
| C->>C: Defer exact identity and rotate | ||||||||||||||
| end | ||||||||||||||
| C->>G: Merge only if real gates pass | ||||||||||||||
| C->>G: Verify protected-main operation when required | ||||||||||||||
| C->>C: Continue next executable lane | ||||||||||||||
| C->>C: Double exit sweep before any terminal response | ||||||||||||||
| C-->>X: continuation_handoff or bounded termination | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| ## Documentation reconciliation flow | ||||||||||||||
|
|
||||||||||||||
| ```mermaid | ||||||||||||||
| flowchart TD | ||||||||||||||
| H[Conversation / prompt / incident / PR evidence] --> R[Refetch protected main and active PRs] | ||||||||||||||
| R --> K{Canonical documentation line exists?} | ||||||||||||||
| K -->|yes| U[Update existing canonical line] | ||||||||||||||
| K -->|no| N[Create one canonical line] | ||||||||||||||
| U --> S[Assign controlled maturity state] | ||||||||||||||
| N --> S | ||||||||||||||
| S --> T[Update PRD/TRD/Architecture/ADR/UML/Data Model/Security/Operations/Traceability as affected] | ||||||||||||||
| T --> V[Run documentation fitness contract] | ||||||||||||||
| V --> Q[Return to executable queue] | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| ## Deployment topology | ||||||||||||||
|
|
||||||||||||||
| The control plane is hybrid rather than purely GitHub-native. External scheduled agent/orchestrator services provide finite invocation and writer-lease coordination; GitHub provides repository execution, evidence, policy, collaboration, and protected integration; external model/reviewer providers sit behind explicit credential and network boundaries. No durable database is assumed by this architecture; the data model is conceptual unless a persistence implementation is separately accepted. | ||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,67 @@ | ||||||||||
| # CWL Automation Control Plane — Conceptual Data Model | ||||||||||
|
|
||||||||||
| Status: active_pr | ||||||||||
|
|
||||||||||
| This is a logical evidence/domain model. It does **not** assert that the central control plane currently persists these entities in a database. External scheduler state, GitHub repository state, and canonical documentation state are separate authorities even when one automation run observes all three. | ||||||||||
|
|
||||||||||
| ```mermaid | ||||||||||
| erDiagram | ||||||||||
| repository_target ||--o{ pull_request_snapshot : contains | ||||||||||
| pull_request_snapshot ||--|| source_revision : identifies | ||||||||||
| pull_request_snapshot ||--|| base_revision : targets | ||||||||||
|
Comment on lines
+10
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
동일한 head 또는 base commit은 여러 수정 예시- pull_request_snapshot ||--|| source_revision : identifies
- pull_request_snapshot ||--|| base_revision : targets
+ source_revision ||--o{ pull_request_snapshot : identifies
+ base_revision ||--o{ pull_request_snapshot : targets📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||
| pull_request_snapshot ||--o{ check_evidence : has | ||||||||||
| pull_request_snapshot ||--o{ review_evidence : has | ||||||||||
| pull_request_snapshot ||--o{ status_evidence : has | ||||||||||
| pull_request_snapshot ||--o{ workflow_evidence : has | ||||||||||
| pull_request_snapshot ||--o{ dependency_evidence : has | ||||||||||
| automation_run ||--o{ execution_lane : schedules | ||||||||||
| execution_lane ||--o{ deferred_item : defers | ||||||||||
| automation_run ||--o{ incident_hypothesis : evaluates | ||||||||||
| automation_run ||--o{ handoff_record : records | ||||||||||
| automation_run ||--o{ continuation_handoff : continues_with | ||||||||||
| automation_run ||--o{ automation_control_record : observes | ||||||||||
| automation_run ||--o{ operational_acceptance : verifies | ||||||||||
| automation_run ||--o{ secret_requirement : constrains | ||||||||||
| repository_target ||--o{ writer_lease : governed_by | ||||||||||
| automation_control_record ||--o{ writer_lease : may_grant | ||||||||||
| documentation_baseline ||--o{ documentation_fitness_result : evaluated_by | ||||||||||
| documentation_baseline ||--o{ decision_record : contains | ||||||||||
| ``` | ||||||||||
|
|
||||||||||
| ## Entities | ||||||||||
|
|
||||||||||
| - `automation_run`: one finite invocation of a maintenance, development, or audit loop. | ||||||||||
| - `automation_control_record`: observed external scheduler/orchestrator identity, enabled state, cadence, ownership scope, and configuration revision relevant to a run; it is not GitHub source evidence. | ||||||||||
| - `repository_target`: repository identity and control-plane ownership class. | ||||||||||
| - `execution_lane`: one independently executable unit such as a PR/head, issue, documentation line, operational acceptance probe, release task, or bounded product slice. | ||||||||||
| - `deferred_item`: an exact lane identity temporarily non-actionable because of queued evidence, approval, provider capacity, read-only dependency, writer conflict, or another bounded wait condition. | ||||||||||
| - `continuation_handoff`: records the preceding substantive action or defer decision and the next selected executable lane, or the bounded termination reason after the required exit sweeps. | ||||||||||
| - `pull_request_snapshot`: current PR metadata captured for a decision. | ||||||||||
| - `source_revision`: exact source/head commit identity. | ||||||||||
| - `base_revision`: independently resolved current base-ref tip identity. | ||||||||||
| - `check_evidence`: named GitHub Check evidence bound to a revision. | ||||||||||
| - `status_evidence`: commit-status context, creator, state, target URL where applicable, and revision. | ||||||||||
| - `review_evidence`: formal review/reviewer/thread evidence. | ||||||||||
| - `workflow_evidence`: workflow/run/job/checkout identity and outcome. | ||||||||||
| - `dependency_evidence`: state of a central or stacked prerequisite. | ||||||||||
| - `incident_hypothesis`: falsifiable causal hypothesis and disposition. | ||||||||||
| - `handoff_record`: read-only transfer to the authoritative owner when mutation is outside the lease. | ||||||||||
| - `operational_acceptance`: protected-main consumer execution evidence. | ||||||||||
| - `secret_requirement`: purpose, scope, materialization boundary, and least-privilege requirement for a secret. | ||||||||||
| - `writer_lease`: authoritative writer scope, source of authority, branch/repository boundary, and conflict evidence. | ||||||||||
| - `documentation_baseline`: one canonical documentation authority for a bounded control-plane/product scope. | ||||||||||
| - `documentation_fitness_result`: adequacy assessment by artifact class, including missing, stale, partial, adequate, conflicting, or not-applicable findings and required corrective actions. | ||||||||||
| - `decision_record`: material decision reconciled from protected-main source, active PRs, incidents, research, or conversation evidence with an explicit maturity state. | ||||||||||
|
|
||||||||||
| ## Invariants | ||||||||||
|
|
||||||||||
| - `source_revision` and `base_revision` are never collapsed into one identity. | ||||||||||
| - Review, check, status, workflow, model, merge, external-automation, and runtime evidence remain separate authorities. | ||||||||||
| - A stale snapshot cannot authorize a write. | ||||||||||
|
Comment on lines
+56
to
+60
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 모든 evidence authority를 ERD에 명시하거나 매핑을 정의해야 합니다. Line 59는 🤖 Prompt for AI Agents |
||||||||||
| - A `deferred_item` blocks only its exact `execution_lane` unless broader evidence proves shared scope. | ||||||||||
| - Every substantive action that does not exhaust the invocation produces a `continuation_handoff` to the next executable lane. | ||||||||||
| - A user-visible status or prompt update cannot satisfy `continuation_handoff` by itself. | ||||||||||
| - A `documentation_baseline` is singular for its declared scope; conversation or PR-body text does not silently become a second authority. | ||||||||||
| - A `documentation_fitness_result` marked missing, stale, partial, or conflicting requires a repository mutation or explicit non-actionable disposition when the current writer owns the documentation line. | ||||||||||
| - A conceptual entity is not evidence that persistence exists. | ||||||||||
| - Durable database object names, if later introduced, use descriptive two-or-more-word `snake_case` names. | ||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
라이브 워크플로 변경 시 문서 계약 검사를 실행하세요.
tests/test_automation_documentation.pyLine 113-115는 4개 라이브 워크플로의 존재를 검사합니다. 현재paths에는 이 파일들이 없습니다. 해당 워크플로를 삭제하거나 이름을 변경하는 PR은 이 품질 게이트를 실행하지 않고 병합될 수 있습니다.WORKFLOW_REFERENCES의 모든 경로를paths에 추가하세요.수정 예시
📝 Committable suggestion
🤖 Prompt for AI Agents