diff --git a/docs/design/support-boundary/README.md b/docs/design/support-boundary/README.md index 5c08547e..abecb1b0 100644 --- a/docs/design/support-boundary/README.md +++ b/docs/design/support-boundary/README.md @@ -13,9 +13,10 @@ | Topic | Docs | |---|---| | **The boundary** | [support-contract.md](support-contract.md) — the single statement · [kustomize-support-boundary.md](kustomize-support-boundary.md) — field taxonomy + layout allowlist · [gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md) — **the write boundary; the one home of fan-in = 1** | +| **Renderers & provenance** | [render-attribution.md](render-attribution.md) — attribution and verification · [render-root-scoping.md](render-root-scoping.md) — render roots and the oracle · [render-fidelity.md](render-fidelity.md) — **our render is not the orchestrator's; refuse where they diverge** · [kpt-and-krm-functions.md](kpt-and-krm-functions.md) — how Kpt packages, setters, and KRM functions may fit safely | | **Orchestrators & expansion** | [orchestrator-knowledge-boundary.md](orchestrator-knowledge-boundary.md) — renderability vs ownership; claims about paths · [expansion-boundary-and-corpus-organisation.md](expansion-boundary-and-corpus-organisation.md) — provenance; ApplicationSet vs ResourceSet; Helm · [`../../facts/expansion-provenance-markers.md`](../../facts/expansion-provenance-markers.md) — **the measured markers** · [argocd-bi-directional.md](argocd-bi-directional.md) — why `selfHeal` is incompatible | | **Documents & secrets** | [resource-capability-model.md](resource-capability-model.md) — what may I do to this document · [write-only-encrypted-secrets.md](write-only-encrypted-secrets.md) — SOPS · [sealed-secrets-and-external-secrets.md](sealed-secrets-and-external-secrets.md) | -| **Edits with no home** | [unreflectable-edits-and-write-gating.md](unreflectable-edits-and-write-gating.md) | +| **Edits with no home** | [unreflectable-edits-and-write-gating.md](unreflectable-edits-and-write-gating.md) — tier-1/2/3 accounting · [admission-consent.md](admission-consent.md) — say yes to a blast-radius refusal · [orchestrator-reconcile-trigger.md](orchestrator-reconcile-trigger.md) — revert a refusal / order around origin drift | | **Discovery (read-only)** | [repo-discovery-and-onboarding-scan.md](repo-discovery-and-onboarding-scan.md) | | **Shipped** | [finished/images-and-replicas-edit-through.md](finished/images-and-replicas-edit-through.md) · [finished/higher-level-krm-documents.md](finished/higher-level-krm-documents.md) | | **Evidence** | [`test/fixtures/gitops-layouts/`](../../../test/fixtures/gitops-layouts/) — the corpus of real-world repo shapes, and the generated behavioural baseline beside it | diff --git a/docs/design/support-boundary/admission-consent.md b/docs/design/support-boundary/admission-consent.md new file mode 100644 index 00000000..081f3c75 --- /dev/null +++ b/docs/design/support-boundary/admission-consent.md @@ -0,0 +1,211 @@ +# Admission consent: a blast-radius refusal you can say yes to + +> **design** — direction-setting; ships no code. Nothing it describes is supported today. +> Captured: 2026-07-15 +> Related: +> [README.md](README.md), +> [unreflectable-edits-and-write-gating.md](unreflectable-edits-and-write-gating.md) — **the tier-1/2/3 model this extends; tier 3 is the admission gate**, +> [gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md) — the write boundary; fan-in = 1; base read-only by L1, +> [render-attribution.md](render-attribution.md) §5 — attribution may be heuristic, verification may not, +> [orchestrator-reconcile-trigger.md](orchestrator-reconcile-trigger.md) — **the sibling half: what reverts a refusal that lands anyway**, +> [support-contract.md](support-contract.md) + +This is one half of a two-part design. This half is about turning a *refusal* into a *yes* at the +moment of the edit; the other half — [the reconcile trigger](orchestrator-reconcile-trigger.md) — is +about making the *outcome* of a "no" prompt and visible. They compose, but they are separate topics +and separate documents. + +Today a write the operator cannot place is refused, correctly, but the refusal is **binary and +mute**. The user who edits a base shared by prod and staging is told "no" — at flush time, on a +`GitTarget` condition they may not be watching — with no way to say *"yes, I know it changes both; +that is what I meant."* Some refusals should never be lift­able. But that one should, and this +document is about the line between them. + +--- + +## 1. The refusal is right, but there is no "yes" + +An edit to a shared base is refused because fan-in = 1 makes shared context read-only +([gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md)): +changing one file would change what more than one environment renders. That rule exists because the +user *usually* did not mean to change every environment at once. + +But sometimes they did. "Bump the base image for all environments" is an ordinary, legitimate +intent. Today it has no expression: the operator refuses it exactly as it refuses an accidental +cross-environment edit, because it cannot tell the two apart. The missing thing is not a weaker +rule — it is a way for the user to **declare which one this is.** + +That declaration has to happen where the user is, and synchronously, which is why this rides the +**tier-3 admission gate** already designed in +[unreflectable-edits-and-write-gating.md](unreflectable-edits-and-write-gating.md) (opt-in per +GitTarget, `failurePolicy: Ignore`, `--dry-run=server` preflight). Tier 3 as written only *rejects* +an unsavable write. This adds the "yes." + +--- + +## 2. Two kinds of "no", and only one is negotiable + +Everything turns on splitting refusals into two piles, because consent is safe for exactly one. + +```mermaid +flowchart TD + N["the operator says NO"] --> Q{which kind of no?} + Q -->|"the render does not reproduce
what you asked, or moves an
object you never touched"| C["CORRECTNESS
consent cannot lift it —
no annotation changes physics"] + Q -->|"the edit does exactly what you asked,
but the file is a base shared by
more than one environment (fan-in > 1)"| P["POLICY / blast radius
consent CAN lift it —
'yes, change every environment'"] + classDef bad fill:#fdd,stroke:#c33,color:#111 + classDef good fill:#dfd,stroke:#3a3,color:#111 + class C bad + class P good +``` + +| | **Correctness refusal** | **Policy refusal** | +|---|---|---| +| The operator is saying | "this does not reproduce what you asked, or corrupts an object you did not touch" | "this does what you asked, but the blast radius is bigger than you may realize" | +| Example | the field is owned by a patch/transformer; the render does not converge; an unpairable list | the file is a **base shared by prod and staging**; the edit changes *both* | +| Decided by | the render oracle (`VerifyBatchRenders`) | the write-boundary policy (L2 fan-in = 1) | +| Consent lifts it? | **Never.** | **Yes.** | + +The correctness pile is the render oracle, and it is absolute: a dyed render plus a real re-render +either reproduce the live object and disturb nothing unintended, or they do not +([render-attribution.md §5](render-attribution.md)). No annotation makes a non-converging write +converge. + +The policy pile is different — it is refused for the user's protection, not for physics. That is the +pile consent unlocks. + +--- + +## 3. Consent is declared intent, not a bypass + +The tempting model is a `force: true` that skips a check. That is exactly wrong, and the correct +model is already in the code. + +The oracle ([`render_verify.go`](../../../internal/manifestanalyzer/render_verify.go)) checks two +things against a set of **`WriteIntent`s**: every intended document renders to its live object, and +**every object the batch did *not* declare an intent for comes out byte-identical**. A fan-in +refusal is really the second clause firing: editing the shared base also moves `Deployment/web` in +staging, staging was never an intent, so the write "moves something it never set out to write" and +is refused. + +**Consent adds the sibling object as an intent.** When the user acknowledges "this changes prod and +staging," the operator promotes staging's object from the *must-be-untouched* set into the +*intended-to-change* set, and the **same oracle** runs, unchanged, over the larger intent — and it +still has to pass: the base edit must render to the acknowledged live state in *both* environments +and disturb nothing else. + +```mermaid +flowchart LR + E["edit a shared base"] --> O{"oracle: does anything
UNINTENDED move?"} + O -->|"staging/web moves,
and it is not an intent"| R["REFUSE — fan-in"] + R -->|"user acknowledges
the named consequence"| I["staging/web becomes
a DECLARED intent"] + I --> O2{"oracle: does the base edit render
to live in BOTH, and disturb
nothing else?"} + O2 -->|yes| W["write the base"] + O2 -->|no| R2["REFUSE — correctness
(consent cannot lift this)"] + + classDef bad fill:#fdd,stroke:#c33,color:#111 + classDef good fill:#dfd,stroke:#3a3,color:#111 + class R,R2 bad + class W,I good +``` + +So consent does not remove a check. It **re-labels collateral as intent**, and the correctness gate +verifies the whole expanded intent exactly as before. There is no second code path, no bypass, and +no way for consent to wave through a write that does not actually converge. + +### The token is scoped to a consequence, not to "off" + +The acknowledgement must name *what* is being consented to, so it can never become a standing "ignore +safety" flag: + +- The operator computes the consequence — the concrete set of `(object, environment)` pairs the + write would move — and reduces it to a **content hash**. +- Consent carries that hash. The operator honours it only when the *current* computed consequence + hashes to the same value. Change the base differently, or let the tree drift, and the old + acknowledgement no longer matches → the write is refused again, naming the *new* consequence to + acknowledge. + +This is the same discipline the dye uses for attribution: consent to a **named, verified** fact, not +to a mood. + +### Where the token lives + +Two surfaces, both already "near the actor" (the write-gating doc's fourth principle) and both +already understood by the pipeline: + +- **An annotation on the edited object** (`configbutler.ai/acknowledge-consequence: `) travels + with the `kubectl apply`, so consent is expressed in the same breath as the edit. It must be + **stripped before mirroring**, exactly as [`sanitize`](../../../internal/sanitize/types.go) already + strips orchestrator bookkeeping keys — a consent token is operator control data, never content + that belongs in Git. +- **A field on the `CommitRequest`** — the object a caller already polls for `Pushed` + `status.sha` + — is the natural home for *session-scoped* consent ("everything in this save window may touch the + base"), and it composes with the `FullyReflected` condition tier 2 puts there. + +### The edge consent must not cross: authorization + +Consent lifts *"did you realize,"* never *"are you allowed."* A base shared across environments in +**one** GitTarget's authorization scope is fine — the acking user already owns all of it. But a base +shared across **different** GitTargets (different RBAC, different tenants) is the case +[gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md) +forecloses on purpose: a user who can edit prod must not change staging by editing the base when they +may have no rights to staging. Consent from the prod editor cannot manufacture authority over +staging. So consent unlocks a shared-base edit **only within a single authorization scope**; across +scopes it stays refused, and the route remains Option C (base-as-variant with its own GitTarget and +RBAC). This boundary is the same class of "no" as correctness — not lift­able by annotation. + +--- + +## 4. The admission surface + +Consent needs a synchronous surface, and admission is the only one: it is the sole point where the +user's intent can be accepted or rejected **whole, before persistence** — the property the +write-gating doc names as the real argument for the gate. The infrastructure exists (the operator +already runs admission webhooks, +[`validate_operator_types_handler.go`](../../../internal/webhook/validate_operator_types_handler.go)), +and tier 3 already specifies it. Consent adds one branch: + +```mermaid +flowchart TD + A["kubectl apply (edit)"] --> W{"admission: single-object
source-form + oracle preview"} + W -->|"clean"| OK["allow — writes normally"] + W -->|"correctness refusal"| DENY["deny: name the field
(no token offered)"] + W -->|"policy refusal,
no matching ack"| ASK["deny: name the consequence
+ its ack token"] + W -->|"policy refusal,
ack matches"| OKC["allow — record the ack
as a declared intent"] + ASK -.->|"user re-applies
with the token"| W + + classDef bad fill:#fdd,stroke:#c33,color:#111 + classDef good fill:#dfd,stroke:#3a3,color:#111 + class DENY,ASK bad + class OK,OKC good +``` + +Two honest limits, both already answered by the tier-3 design: + +- **Admission sees one request; the oracle needs the batch.** The webhook can only run a + *single-object preview* of the oracle (project this one object to source form, re-render, read the + blast radius). It can be wrong — staleness, or cross-object batch effects it cannot see. That is + tolerable because the gate is **fail-open and advisory**: a wrong *deny* is a retry, and a wrong + *allow* is caught at flush. +- **Consent granted at admission does not bind the flush.** This is the load-bearing safety line. The + flush-time oracle re-verifies the *whole batch* against the declared intents — including the + consented ones — and still refuses if the consented set does not actually converge. So a stale or + mistaken admission-time "yes" cannot cause a bad write. Worst case it lands, the flush refuses it, + and the [reconcile trigger](orchestrator-reconcile-trigger.md) reverts it — the same safety net + that catches an edit made while the gate was disabled entirely. + +--- + +## 5. Open questions + +- **Surface and granularity.** Object annotation (per-edit, stripped like the sanitize deny-list) + vs. `CommitRequest` field (per-window, already polled) vs. both. Per-consequence-hash scoping is + the recommendation; a per-GitTarget "base edits allowed" mode is the blunt alternative, probably + too blunt. +- **Does consent-to-edit-a-base earn a place in the support contract?** It is a genuinely different + operation from per-environment patch authoring ("I mean the base" vs. "I mean this overlay"), and + [support-contract.md](support-contract.md) should say so explicitly rather than leave it implied by + the fan-in refusal being liftable. +- **What computes the consequence at admission time?** The single-object preview needs the same + source-form projection the writer uses, evaluated against a store snapshot. Its cost and staleness + are the tier-3 concerns; consent adds the requirement that the *hash* it produces is stable across + the preview and the eventual flush, or the token will spuriously fail to match. diff --git a/docs/design/support-boundary/kpt-and-krm-functions.md b/docs/design/support-boundary/kpt-and-krm-functions.md new file mode 100644 index 00000000..15a2a10e --- /dev/null +++ b/docs/design/support-boundary/kpt-and-krm-functions.md @@ -0,0 +1,331 @@ +# Kpt and KRM functions: a safe place in the reverse-GitOps model + +> **design** — an orientation note. Captured: 2026-07-14. +> +> Related: [render attribution](render-attribution.md), +> [render-root scoping](render-root-scoping.md), +> [the support contract](support-contract.md), and the +> `kustomize-tracer` plans (ConfigButler/kustomize-tracer, `plans/` — a local checkout under +> `external-sources/`, not tracked here). + +Kpt and KRM functions have a useful place in GitOps Reverser, but they do not make a +generic rendered object reversible. The product's central question remains: + +> Given a live field change, is there exactly one writable place in Git that can +> produce it again without changing anything else? + +Kustomize, Kpt, a CI hydration job, and an Argo or Flux controller are all possible +parts of the answer. They are not interchangeable. The renderer we use to attribute +and verify a write must be the renderer that produced the objects the delivery system +applies. + +## 1. The common model + +Every supported configuration system fits the same pipeline. The stages in green are +capabilities, not promises: an unfamiliar transform may stop at the red refusal. + +```mermaid +flowchart LR + G["Git intent\nfiles, Kustomization, Kptfile"] + R["actual renderer\nplain YAML / kustomize / kpt hydration"] + L["live KRM object\nuser changes a field"] + A["attribute governance\nsource field, override, setter, or unknown"] + P["propose the smallest\nsource edit"] + V["verify the whole\nrender scope"] + C["commit to Git"] + X["report refusal\nno write"] + + G --> R --> L --> A --> P --> V + V -->|target matches and\nother output is unchanged| C + A -->|no unique writable home| X + V -->|mismatch or blast radius| X + + classDef safe fill:#dfd,stroke:#3a3,color:#111 + classDef refuse fill:#fdd,stroke:#c33,color:#111 + class A,P,V,C safe + class X refuse +``` + +The current Kustomize work already implements the important safety half. A candidate +write is rendered against the exact post-write bytes; the intended object must match +the live object and every object outside the write batch must be unchanged. This is the +**oracle**. Attribution can improve over time, but it must never become the only proof +that a write is safe. + +This explains two rules that apply equally to Kustomize and Kpt: + +1. **Causation is not governance.** A transform that made no visible change can still + override a later source edit. An idempotent Kustomize `images:` entry and an + idempotent KRM function parameter are both "invisible and armed." +2. **A render result is not a source map.** It says what exists after rendering, not + which exact configuration field a user should edit to reproduce one value. + +## 2. Why Kustomize is the first renderer + +Kustomize is unusually tractable because its configuration language has named entries +such as `images[0]`, `replicas[0]`, and `patches[0]`. The tracer work observes the +before/after state around each transformer entry, so it can say both *which entry* and +*which field* changed. + +```yaml +# overlays/prod/kustomization.yaml +resources: + - ../../base + +images: + - name: ghcr.io/acme/web + newTag: "2.4.1" + +patches: + - path: production-tuning.yaml + target: + kind: Deployment + name: web +``` + +```yaml +# overlays/prod/production-tuning.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: web +spec: + replicas: 3 +``` + +For `containers[name=web].image`, an `images[0]` event can route the edit to +`kustomization.yaml`. For `spec.replicas`, a patch may be the governing input. The +important result is not "always edit the patch"; Kustomize ordering can make a patch +dead text when a later `replicas:` entry wins. The renderer, rather than a hand-written +ordering model, decides that. + +The two complementary attribution techniques are: + +| Question | Technique | Result | +|---|---|---| +| What actually changed this field in this build? | Observer around each transformer entry | exact causal field events and lineage | +| Which configured input will win even when it agrees with its source? | Counterfactual dye | governance of safe, dyeable values | + +Neither is permission to write. Both feed the oracle, which checks the proposed edit over +the whole render root and its read scope. Fields changed by unobserved mechanisms—such +as generator hashes and reference rewrites—are **unattributable**, not source-owned by +default. + +## 3. What Kpt adds + +Kpt is package-centred rather than Kustomization-centred. A `Kptfile` can declare an +upstream package, package metadata, inventory, and a function pipeline. A function takes +KRM resources plus configuration, then emits KRM resources. This produces three useful +ideas for this project. + +### Packages are render roots + +A Kpt package can be treated as another candidate unit in repo discovery, much as a +Kustomize render root is today. Its read scope includes the package and any declared +inputs that its actual hydration process reads. Its write scope must still remain inside +the `GitTarget` path. + +This is only meaningful when Kpt hydration is in the delivery path. There are two +materially different repository shapes: + +```mermaid +flowchart TB + subgraph authoring["Kpt used only while authoring"] + D1["dry Kpt package"] --> CI["developer or CI: kpt fn render"] --> W["wet YAML committed for Flux/Argo"] + W --> F1["Flux or Argo applies YAML"] + end + + subgraph delivery["Kpt hydration is the delivery renderer"] + D2["dry Kpt package"] --> K["delivery job/controller: kpt fn render"] --> F2["apply rendered KRM"] + end + + classDef note fill:#fff4cc,stroke:#b8860b,color:#111 + class CI,K note +``` + +In the first shape, GitOps Reverser sees the wet YAML as its ordinary Git source; it +cannot safely reverse through an unobserved CI step into the dry package. In the second, +Kpt is part of the renderer and needs a dedicated renderer adapter, provenance rules, and +the same post-write verification as Kustomize. + +### Setters are an explicit inverse contract + +The most promising Kpt concept is the setter. It marks a resource value with the name of +the parameter that controls it. That is much better than inferring ownership from a final +rendered value. + +```yaml +# deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: web +spec: + replicas: 2 # kpt-set: ${web-replicas} + template: + spec: + containers: + - name: web + image: ghcr.io/acme/web:2.4.1 # kpt-set: ghcr.io/acme/web:${web-tag} +``` + +```yaml +# setters.yaml -- function configuration, not an applied workload +apiVersion: v1 +kind: ConfigMap +metadata: + name: setters + annotations: + config.kubernetes.io/local-config: "true" +data: + web-replicas: "2" + web-tag: "2.4.1" +``` + +```yaml +# Kptfile +apiVersion: kpt.dev/v1 +kind: Kptfile +pipeline: + mutators: + - image: ghcr.io/kptdev/krm-functions-catalog/apply-setters:v0.2 + configPath: setters.yaml +``` + +An edit to the live Deployment's replicas can propose changing +`setters.yaml:data.web-replicas`, rather than overwriting the annotated Deployment +field. This is an **existing-setter** capability only. Creating comments, creating a +setter configuration, or guessing that equal values share a parameter is authoring and +must be a separate future feature. + +A setter is shared context. If `web-tag` controls ten resources, changing it for one +resource is valid only when the intended batch includes all ten resulting changes. The +whole-render oracle makes this check mechanical. + +### Function pipelines are a renderer extension point, not an inverse API + +Kpt's pipeline has useful structure: ordered mutators, validators, per-function +configuration, and selectors. It does **not** turn an arbitrary function into a reversible +operation. The function can be a container image or an executable, and selectors can fan +one configuration value out to many resources. + +```yaml +# This is renderable only under a deliberately approved runner policy. +apiVersion: kpt.dev/v1 +kind: Kptfile +pipeline: + mutators: + - image: ghcr.io/example/organisation-policy@sha256:REPLACE_WITH_PINNED_DIGEST + configMap: + team: payments + selectors: + - kind: Deployment + labels: + app.kubernetes.io/part-of: checkout +``` + +Even if the output has `metadata.labels.team: payments`, that does not identify a +source-field write location, prove that the function is deterministic, or prove it did +not change another object. Treat this form as **renderable read-only context** until a +specific function contract is supported. + +## 4. Support levels for Kpt and KRM functions + +The right boundary is additive and per capability, rather than "Kpt supported" or +"Kpt refused." + +```mermaid +flowchart TD + A["Kptfile or KRM function found"] --> B{"Does this process\nproduce deployed objects?"} + B -->|No / unknown| R1["Do not reverse through it\nedit deployed Git source only"] + B -->|Yes| C{"Runner is pinned, local/sandboxed,\nand reproducible?"} + C -->|No| R2["Read-only or refuse\narbitrary execution/network"] + C -->|Yes| D{"Existing explicit setter\nor supported function contract?"} + D -->|Existing setter| E["Propose setter-config edit\nthen verify whole render"] + D -->|Known function, proven sinks| F["Counterfactual attribution\nroute only proven fields"] + D -->|No| R3["Renderable, but field is\nunroutable: report/refuse edit"] + + classDef safe fill:#dfd,stroke:#3a3,color:#111 + classDef refuse fill:#fdd,stroke:#c33,color:#111 + class E,F safe + class R1,R2,R3 refuse +``` + +| Level | Example | Operator behaviour | +|---|---|---| +| 0 — metadata only | `Kptfile` without a pipeline in a raw-YAML delivery repo | Ignore it as a deployment transform; preserve it as package metadata. | +| 1 — observable but unroutable | A known pipeline renders the object, but no explicit ownership exists | Render for comparison; report the field as unreflectable; do not write. | +| 2 — explicit route | Existing `apply-setters` comment and config | Edit the named setter value; verify every render output. | +| 3 — narrow function contract | A pinned, allowlisted function with a tested sink-only configuration field | Dye or otherwise perturb the configuration, count fan-out, and write only after verification. | +| never generic | Arbitrary container/exec function, network access, unpinned tag, opaque generated output | No source routing. Refuse the edit or operate only on a separately committed rendered artifact. | + +The KRM Functions Catalog is therefore a source of **candidate contracts**, not a list to +enable. `apply-setters` is a strong candidate because it declares the inverse coordinate. +Bulk transforms such as `set-labels`, `set-namespace`, `set-image`, Starlark, Helm +rendering, or generic search-and-replace should begin at level 1. Each needs independent +evidence about determinism, selector fan-out, safe-to-dye values, and a comment-preserving +write location before it can move higher. + +## 5. Extending the render-plan idea + +The Kustomize tracer's proposed render plan is a useful *shape* that can eventually cover +more than Kustomize. It is an inverse-build artifact: a lookup hint for routing, never +permission to write. + +```yaml +apiVersion: gitopsreverser.io/v1alpha1 +kind: RenderPlan +renderer: + type: kpt + packageRoot: apps/web + pipelineFingerprint: sha256:REPLACE_WITH_REAL_CONTENT_HASH +inputs: + Kptfile: sha256:REPLACE_WITH_REAL_CONTENT_HASH + deployment.yaml: sha256:REPLACE_WITH_REAL_CONTENT_HASH + setters.yaml: sha256:REPLACE_WITH_REAL_CONTENT_HASH +objects: + - id: apps/v1/Deployment/default/web + origin: deployment.yaml + fields: + spec.replicas: + source: + kind: setter + file: setters.yaml + path: data.web-replicas + function: apply-setters +unattributable: + - object: apps/v1/Deployment/default/web + field: metadata.labels.team + reason: function-contract-not-supported +``` + +The fingerprint prevents a plan made for old inputs from posing as current knowledge. A +matching plan can propose the setter edit; a missing, stale, or incomplete plan causes a +fallback to live counterfactual attribution where that is safe, otherwise a refusal. In +all cases the post-write render remains decisive. + +This artifact could eventually serve three consumers without creating three analyses: + +- **writer:** field-to-source proposals and clear refusals; +- **repository map:** render roots/packages, inputs, and ownership edges; and +- **metrics:** counts of `unattributable` fields and the transforms that cause them. + +## 6. Practical sequencing + +Do not let Kpt exploration interrupt the current Kustomize support-boundary work. +`patches:` is the immediate, measured opportunity: make folders renderable first, move +refusals to individual unroutable fields, then add tightly bounded attribution and routing. + +After that, the smallest useful Kpt slice is: + +1. Detect `Kptfile` during repo discovery and report whether a pipeline is present. +2. Record whether the deployment path actually executes Kpt. Do not assume that it does. +3. Support **existing setters only** for a locally reproducible, pinned `apply-setters` + pipeline, with a fixture that proves single-field and fan-out behaviour. +4. Reuse the existing full-batch oracle over the Kpt render scope. +5. Add one function contract at a time, only when its safe inputs and writeback coordinate + are explicit and covered by counterfactual tests. + +The desired outcome is not universal transformation support. It is a growing set of +honest answers: *this live field belongs to this source coordinate and we can prove the +round-trip*, or *this renderer produced the field but we cannot safely write it back.* diff --git a/docs/design/support-boundary/orchestrator-reconcile-trigger.md b/docs/design/support-boundary/orchestrator-reconcile-trigger.md new file mode 100644 index 00000000..9f3f3da1 --- /dev/null +++ b/docs/design/support-boundary/orchestrator-reconcile-trigger.md @@ -0,0 +1,250 @@ +# The orchestrator reconcile trigger: revert a refusal, and order around origin drift + +> **design** — direction-setting; ships no code. Nothing it describes is supported today. +> Captured: 2026-07-15 +> Related: +> [README.md](README.md), +> [../../bi-directional.md](../../bi-directional.md) — **the user-facing model this expands: the reconciler as a *triggered applier***, +> [argocd-bi-directional.md](argocd-bi-directional.md) — why `selfHeal` must be off, and why that means nothing reverts a refused edit, +> [orchestrator-knowledge-boundary.md](orchestrator-knowledge-boundary.md) — **the ownership model this rides on; it is the first *write* action built on it**, +> [admission-consent.md](admission-consent.md) — the sibling half: deciding *whether* a write happens, +> [gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md) + +This is one half of a two-part design. [The other half](admission-consent.md) decides *whether* a +write happens, at admission time. This half is about the operator gaining **one new outward action — +asking the GitOps orchestrator (Flux/Argo) to reconcile now** — and the two places it is needed: +reverting a refused edit promptly, and ordering our processing behind an incoming origin change. + +It is not a correctness layer. The one correctness gate stays where it is — the flush-time render +oracle ([`VerifyBatchRenders`](../../../internal/manifestanalyzer/render_verify.go)). This is about +closing the loop *fast* and *visibly* once a "no" has been decided. + +--- + +## 1. The gap: a triggered applier that is never triggered on the two cases that need it + +[bi-directional.md](../../bi-directional.md) already establishes the model: the reconciler is not an +always-on loop, it is a **triggered applier**. After the operator commits, it triggers the reconciler +to apply *that exact commit* and waits for the SHA — and because the applied revision equals the +committed revision, the loop is closed. The guide draws it as a steady-state loop: + +```mermaid +flowchart LR + api(["API / operator edit"]) --> cluster[("Cluster")] + cluster -- watch --> gr["GitOps Reverser"] + gr -- "commit + push" --> git[("Git")] + git -- "push webhook" --> recon["Flux / Argo CD
(selfHeal OFF)"] + recon -- "apply fresh commit" --> cluster +``` + +That loop only closes on the **happy path**, where a commit is produced and a push fires the webhook. +Two cases fall straight through it, and they are exactly the ones where the operator must trigger the +reconciler *directly* rather than via a push that never happens: + +```mermaid +flowchart LR + api(["API / operator edit"]) --> cluster[("Cluster")] + cluster -- watch --> gr["GitOps Reverser"] + gr -- "commit + push (write placed)" --> git[("Git")] + git -- "push webhook" --> recon["Flux / Argo CD
(selfHeal OFF)"] + recon -- "apply fresh commit" --> cluster + + gr -. "REFUSED: no commit, no push, no webhook —
trigger reconcile to REVERT the live edit (§2)" .-> recon + other(["someone else pushes"]) --> git + git -. "ORIGIN DRIFT: trigger + wait, then process events (§3)" .-> recon + + classDef trig fill:#ffd,stroke:#cc3,color:#111 + class gr,git trig +``` + +The two dotted arrows are the whole of this document: the reconcile trigger, and where it sits. + +--- + +## 2. Use one: revert a refused edit + +### Why it is necessary, not merely nice + +With Argo `selfHeal: false` — *required* for bi-directional editing, because with it on Argo reverts +the live edit sub-second from cached Git and thrashes against us +([argocd-bi-directional.md](argocd-bi-directional.md)) — **nothing reverts a refused edit.** It is +`OutOfSync` until a human intervenes: the push webhook never fires (there was no commit, no push), and +the poll re-resolves an *unchanged* Git and does nothing. Under Flux it is milder but the same shape — +reverted only on the next interval reconcile. So triggering a reconcile is not a speed-up; for Argo it +is the *only* thing that ever reverts a refused edit. + +And the operator is uniquely entitled to do it. [argocd-bi-directional.md](argocd-bi-directional.md) +names the missing ingredient precisely: a system that can *distinguish authorized drift from +unauthorized*, which Argo cannot. **At refusal time the operator has exactly that signal** — the +refused edit is, by construction, the drift with no home in Git. So an operator-triggered reconcile of +a refused edit is the **targeted, per-write substitute for the blanket self-heal the operator had to +switch off.** + +```mermaid +sequenceDiagram + actor User + participant K8s as Cluster + participant Rev as GitOps Reverser + participant Git + participant Recon as Flux / Argo CD + + User->>K8s: edit a base-owned field + K8s-->>Rev: watch event + Note over Rev: flush oracle REFUSES —
no legal destination in Git + Rev-->>K8s: GitPathAccepted=False
(async — the User already got 200 OK) + Note over Git: nothing committed → no push → no webhook + Note over Recon: selfHeal off → it will NOT revert drift on its own + Rev->>Recon: TRIGGER reconcile of the governing object + Recon->>Git: fetch current revision (unchanged) + Recon->>K8s: re-apply desired state → the refused edit is reverted + Recon-->>Rev: reports revision applied + Note over Rev,K8s: loop closed in seconds, instead of never +``` + +Compare this to the guide's happy-path triggered-applier sequence: the shape is identical, but the +trigger fires on a **refusal** (nothing was committed) instead of on a commit, and its job is to +*revert* the drift rather than to *apply* a fresh commit. + +### What "trigger" means, per orchestrator + +The [orchestrator-knowledge-boundary](orchestrator-knowledge-boundary.md) rule is absolute: **never +depend on the `argoproj` or `fluxcd` Go modules.** The trigger is a patch on an object, matched by +group+kind over `unstructured`. But the two orchestrators differ, and the difference matters: + +| | **Flux** | **Argo CD** | +|---|---|---| +| Trigger | patch `reconcile.fluxcd.io/requestedAt` on the `Kustomization` | see below | +| Does it revert drift? | **Yes, cleanly** — Flux server-side-applies desired state, which reverts drift as a side effect | **Not with a plain refresh.** `argocd.argoproj.io/refresh` re-reads Git and re-compares, but with `selfHeal` off it only marks `OutOfSync` — it does **not** revert | +| To actually revert | (nothing more) | requires a **sync operation** — a deliberate, one-shot self-heal of the specific refused object | + +So Flux is a one-annotation, correct-outcome trigger. Argo needs the stronger action — a sync — which +writes to the cluster and must therefore be an explicitly granted authority, scoped so it can only +ever revert the specific refused object, never sync the whole app. + +--- + +## 3. Use two: reconcile-before-process, as an ordering barrier + +The same trigger answers a different, subtler problem — **origin moved under us** — and here it is a +*barrier*, not a revert. + +### What is, and isn't, already handled + +The commit direction is already safe against a moved remote. `PushAtomic` +([`git_atomic_push.go`](../../../internal/git/git_atomic_push.go)) is a compare-and-swap, never a +force-push; if the remote advanced, the push is rejected and `pushPendingCommits` +([`branch_worker.go`](../../../internal/git/branch_worker.go)) **rebases by replay** — hard-reset to +the new tip, re-plan and re-commit the retained pending writes on top, re-push with an updated CAS. So +we never clobber someone else's push, and our own intent survives. + +What is **not** handled is the *cluster* side. When origin gains new desired state, the orchestrator is +about to apply it, producing a flood of watch events. Two problems follow: + +1. **The reconcile echo.** Those events came *from* Git via the orchestrator. If the operator processes + them as user intent and mirrors them back, it is round-tripping the orchestrator's own apply into + Git — noise at best, a fight at worst. +2. **The stale baseline.** The operator's model of "what this folder renders to" was computed against + the *old* tree, so its refusal/attribution decisions for the transitional state can be wrong until + the cluster reflects the new origin. + +### The barrier + +The fix is an ordering rule: **on origin drift, trigger the orchestrator to reconcile Git → cluster, +wait for it, and only then resume processing live events.** After the reconcile, the operator's own +mark-and-sweep resync absorbs the orchestrator's apply as a **no-op against the new tree**, instead of +mirroring it as intent. + +```mermaid +sequenceDiagram + actor Other as Someone else + participant Git + participant Rev as GitOps Reverser + participant Recon as Flux / Argo CD + participant K8s as Cluster + + Other->>Git: push new desired state (origin drifts) + Note over Rev: detects the remote moved + Rev->>Git: land pending intent first
(rebase onto the new tip — §3, the hazard) + Rev->>Recon: TRIGGER reconcile Git → cluster, and WAIT + Recon->>Git: fetch the new revision + Recon->>K8s: apply the new desired state + Recon-->>Rev: reports the new revision applied + Note over Rev: only NOW resume live events —
the orchestrator's apply is absorbed
as a resync no-op, not mirrored back +``` + +This rides machinery that already exists. There is already a *reconcile-before-process* barrier: on +watch (re)establishment the operator enqueues a scoped mark-and-sweep ahead of live events, gated by +the `replaying` flag, all on one FIFO so order is preserved +([`target_watch.go`](../../../internal/watch/target_watch.go), +[`resync_flush.go`](../../../internal/git/resync_flush.go)). Two additions turn it into what we need: a +new **trigger** (origin-drift detection, from the existing cached remote-drift check `SyncAndGetMetadata`), +and a stronger **wait** (the barrier's "reconcile" step now waits for the *orchestrator* to apply +Git → cluster, not only the operator's own sweep). + +> **Terminology, because "reconcile" is overloaded and this doc would mislead without saying so.** +> There are two. The **orchestrator reconcile** (Flux/Argo applies Git → cluster) is what this +> document triggers and waits on. The operator's internal **resync** (a mark-and-sweep that rebuilds +> the Git-side model from the cluster, cluster → Git) is what runs *after* the barrier to absorb the +> apply. Where this doc means the internal one, it says "resync." + +### The hazard: pending intent vs. the reconcile that overwrites it + +There is a real ordering trap, drawn as the first `Rev->>Git` step above. When origin drifts, the +operator may hold **uncommitted** pending intent — live edits captured in the open commit window but +not yet pushed. Triggering the orchestrator reconcile *now* would apply the new origin over those live +edits on the cluster, erasing them before they reach Git. + +That is only safe because the intent is **durably captured** (the open window / `pendingWrites`) and +the commit side already rebases it onto a moved origin. So the ordering must be: **land pending intent +to Git first (rebased onto the new tip), *then* trigger the reconcile, *then* resume.** If capture +were not durable, the reconcile would eat the user's edit — so the barrier's safety rests on the +durability the pipeline already has, stated here as a precondition rather than discovered as a bug. + +--- + +## 4. The prerequisite: this is the first *write* on the ownership model + +The operator today **cannot name the Flux/Argo object that deploys a GitTarget's path.** A GitTarget +knows only `(provider, branch, path)`; there is no field, no lookup, and no code that reaches an +orchestrator object (confirmed across `internal/`, `api/`). That capability is designed but unbuilt: +[orchestrator-knowledge-boundary.md](orchestrator-knowledge-boundary.md) proposes per-orchestrator +*interpreters* (`internal/gitops/{flux,argocd}`) emitting **claims about paths** — e.g. +`RenderRootFor{path, by}` naming the object that renders a folder. + +The reconcile trigger is the **first write action** built on that model, which until now is a purely +read/claim vocabulary. It needs one new claim — *"object O reconciles path P"* — and the ability to +patch O. So this feature does not stand alone: **it is gated on the ownership interpreters landing +first.** Stated as a dependency, not smuggled as an assumption. + +--- + +## 5. The boundary: opt-in, and never on a mirror + +Patching another controller's object, and (on Argo) issuing a sync, are boundary crossings. They are +off by default and enabled per GitTarget, alongside the tier-3 write gate — e.g. a +`spec.reconcileTrigger: Off | OnRefusal | OnDrift` knob. Two hard rules: + +- **Never on a cluster the operator merely mirrors.** As with the write gate, we do not get to drive + someone's orchestrator because our mirror is lossy. Only where the cluster is an *editing surface* + whose changes are meant to flow to Git. +- **Absent orchestrator ⇒ no-op, not error.** If no interpreter claims the path, there is nothing to + trigger; the operator falls back to today's behavior (report and wait). The trigger is an + accelerator layered on top. + +--- + +## 6. Open questions + +- **The Argo sync decision.** Reverting a refused edit needs a *sync operation*, not just a refresh — + a deliberate one-shot self-heal. Is issuing it an authority the operator should hold, and how is it + scoped so it can only ever revert the specific refused object, never sync the whole app? +- **Wait semantics for the barrier.** How long to wait for the orchestrator to finish; timeout and + fallback (resume anyway, or stay blocked?); how to observe "done" without depending on the + orchestrator's Go types (its status conditions over `unstructured` — e.g. + `Kustomization.status.lastAppliedRevision`, `Application.status.sync.revision`). +- **How confident must the ownership claim be before we *write* on it?** A wrong "object O reconciles + path P" claim triggers the wrong controller. This is a higher bar than a claim used only to *read*. +- **Interaction with the happy path.** When a commit *does* land, `bi-directional.md`'s webhook already + triggers the apply. Does the operator still issue an explicit trigger (and wait for the SHA, closing + the handshake the guide describes), or defer to the webhook? Likely: trigger only where the webhook + cannot help — refusal and drift — and let the push webhook cover the happy path. diff --git a/docs/design/support-boundary/render-fidelity.md b/docs/design/support-boundary/render-fidelity.md new file mode 100644 index 00000000..36e61c8e --- /dev/null +++ b/docs/design/support-boundary/render-fidelity.md @@ -0,0 +1,360 @@ +# Render fidelity: our render is not the orchestrator's + +> **design** — direction-setting; ships no code. Nothing it describes is supported today. +> Captured: 2026-07-15 +> Related: +> [README.md](README.md), +> [render-root-scoping.md](render-root-scoping.md) §3 — the version-skew caveat this generalises, +> [render-attribution.md](render-attribution.md) §5 — *attribution may be heuristic, verification may not*, and the "shared blind spot" failure, +> [orchestrator-knowledge-boundary.md](orchestrator-knowledge-boundary.md) — reading the Flux/Argo object; the `TransformedOutOfBand` claim, +> [gittarget-granularity-and-cross-environment-edits.md](gittarget-granularity-and-cross-environment-edits.md), +> [finished/images-and-replicas-edit-through.md](finished/images-and-replicas-edit-through.md) + +We run `kustomize build` on the folder. Flux and Argo run kustomize on the folder **plus a layer +of context that is not in the folder** — so the object the cluster runs is not the object our +render produces, and every guarantee we make by rendering is only as good as that gap being empty. + +This document names the gap, records a fence that looked obvious and was **wrong** (and why), and +proposes the fence that is right: **measure our render against the live object, and refuse where +they disagree.** It is the same discipline as the rest of this workstream — do not reason about +what the renderer does, ask it — applied one level up: do not assume our render is the +orchestrator's, *check* it. + +--- + +## 1. The context the folder does not hold + +Grounded against the vendored trees, not assumed: + +- **Argo CD** keeps a *second kustomize layer in the Application object*, not the repo: + `spec.source.kustomize` overrides `images`, `replicas`, `patches`, `components`, `nameSuffix`, + `namePrefix`, `commonLabels`, `commonAnnotations`, `namespace`, and even the kustomize **`version`** + (measured in the vendored `argo-cd/pkg/apis/application/v1alpha1/types.go:723`, under + `external-sources/`). A folder we render to `X`, Argo can apply as `Y` — with the very features we + build edit-through around (`images`, `replicas`, `patches`) supplied from a place we never read. +- **Flux** runs `postBuild.substitute` / `substituteFrom` **after** the build, replacing `${var}` + tokens from cluster ConfigMaps/Secrets (measured in the vendored `flux2/internal/build/build.go:631`, + `kustomize.SubstituteVariables`), plus `targetNamespace` and object-level `patches`/`images` on the + Kustomization. +- **Both** run *their* kustomize version with *their* build flags (Argo's `kustomize.buildOptions`), + possibly through a config-management plugin. + +So `applied = f(repo, orchestrator-object, their-kustomize)`, while we compute `g(repo, our-defaults)`. +`g = f` only when the orchestrator adds no context. That is common — but not guaranteed, and **we +cannot tell from the repo which case we are in.** + +```mermaid +flowchart LR + repo[("Git folder")] --> ours["OUR kustomize build
pinned version, default flags"] + ours --> og["our render: g(repo)"] + repo --> orch["THE ORCHESTRATOR
kustomize + postBuild substitute
+ Application overrides + their version"] + ctx["context NOT in the folder:
Flux postBuild vars, Argo spec.source.kustomize,
build flags, kustomize version"] --> orch + orch --> live["live object: f(repo, context)"] + og -. "we ASSUME g = f
(and only sometimes it is)" .-> live + + classDef blind fill:#fdd,stroke:#c33,color:#111 + class ctx blind +``` + +This is not a new admission. [render-root-scoping.md §3](render-root-scoping.md) already concedes a +*version*-skew caveat — the guarantee is only *"this renders to what you edited, under the kustomize +we pinned."* This document generalises it: **it is not only the version that can differ, it is the +whole render context**, and version is the *least* likely axis to bite. + +--- + +## 2. Why it bites, and why the source-form fix does not close it + +The [source-form projection](finished/images-and-replicas-edit-through.md) stops **our** render's +output leaking into the source: where the live object and *our* render agree, the source keeps its +bytes. But context we do **not** render makes `live ≠ our-render` for a reason that is *not a user +edit* — and the projection has no way to tell the two apart. It reads the divergence as an edit and +writes the orchestrator's value into the source. + +Concretely, with Flux `postBuild`: + +```text +git source: env REGION = ${REGION} +our render: env REGION = ${REGION} (kustomize never touches a ${...} token) +live object: env REGION = us-east (Flux substituted it from a cluster ConfigMap) +``` + +The projection sees source == our-render (`${REGION}`) but live differs, concludes the user set +`us-east`, and **writes `us-east` into the source — destroying the `${REGION}` parameterisation.** +Next reconcile, Flux substitutes again (now a no-op, the value is already literal), so it *looks* +converged while the template is gone; change `REGION` later and the file no longer follows. + +And the oracle does not catch it. [`VerifyBatchRenders`](../../../internal/manifestanalyzer/render_verify.go) +re-renders the write with **our** kustomize, which also leaves `${REGION}` literal — so it agrees +with the corrupt write. This is precisely the failure +[render-attribution.md §5](render-attribution.md) warns about — *a verification that shares the blind +spot of the thing it verifies* — one level up: **our whole renderer shares the orchestrator's blind +spot.** + +--- + +## 3. What we learned: the structural `${...}` check is the wrong fence + +The obvious fence is structural and cheap, and it was tried (and reverted): refuse, at the +acceptance gate, any managed document whose values carry a `${...}` token. It is wrong, and the way +it is wrong is the reason the right fence looks the way it does — so it is recorded here rather than +quietly dropped. + +**`${...}` is ambiguous, and the repo cannot disambiguate it.** The same token is, with equal +frequency: + +- **literal, and safe to mirror** — a CRD schema `description` (the Flux Kustomization CRD documents + postBuild with `${var:=default}` *in its own schema text*), a KRO `${schema.spec.*}` template, an + nginx or envsubst ConfigMap. In every one of these the **live object carries the token verbatim + too** — nothing substitutes it — so `live == our-render` and there is no risk at all. +- **substituted, and dangerous** — the Flux `postBuild` case of §2, where live is `us-east` and our + render kept `${REGION}`. + +A structural check fires on both, and the measurement was unambiguous: **it broke CRD mirroring +outright.** The acceptance gate is all-or-nothing over a folder, so a single CRD whose description +merely *mentions* `${var:=default}` refused the entire folder — every unrelated write with it. A +folder of ordinary CRDs (Flux, cert-manager, prometheus-operator all ship `${}` in their schemas) +became unmanageable. This is not "over-refuse a little and be right"; it is breaking a core feature. + +**The lesson is not "narrow the regex."** No structural refinement helps, because the +discriminator — *was this token actually substituted?* — **is not in the repo.** A CRD's +`${var:=default}` and a Deployment's substituted `${REGION}` are identical on disk; they differ only +in whether the *cluster's* copy still holds the token. The fence therefore cannot be structural, and +cannot live at the structure-only acceptance gate. + +> A smaller lesson, recorded so it is not relearned: `task test-e2e 2>&1 | tail -N` reports `tail`'s +> exit status (0), not the suite's — a failing suite read as green. Capture the full log, or assert +> on the summary line. + +--- + +## 4. The right fence: measure against the live object + +The discriminator we lack on disk, we already hold at write time: **the live object *is* the +orchestrator's render.** It is what the orchestrator actually applied — postBuild, overrides, its +kustomize version, all of it. So we need not *predict* the orchestrator's context; we can *observe +its output*. + +> **Our edit-through is sound exactly where our render equals the live object at the fields we did +> not set out to change. Where it does not, the orchestrator did something we cannot see — refuse, +> do not guess.** + +This is the workstream's own method, one level up. The dye measures *which entry supplies a value*; +the oracle measures *whether a write reproduces the live object*; this measures *whether our render +is even the right baseline to reason from*. And it is **precise where the structural check was +blunt** — same tokens, opposite verdicts, each correct because it is read off the live object rather +than guessed from disk: + +```mermaid +flowchart TD + W["about to write a field back to source"] --> Q{"does our render equal
the LIVE object here?"} + Q -->|"yes"| K["SAFE — our render is what the cluster runs.
keep source / write as normal"] + Q -->|"no, and the source carries a substitution token here"| R["REFUSE — out-of-band substitution
would destroy the token on write"] + Q -->|"no, and no token (general case)"| D["context skew OR ordinary runtime drift —
the open discriminator (§5b, §8)"] + + classDef good fill:#dfd,stroke:#3a3,color:#111 + classDef bad fill:#fdd,stroke:#c33,color:#111 + classDef open fill:#ffd,stroke:#cc3,color:#111 + class K good + class R bad + class D open +``` + +| document | token | our render vs live | structural check | render-vs-live | +|---|---|---|---|---| +| Flux CRD, `${var:=default}` in a description | yes | **equal** (live has it too) | ❌ refused (wrong) | ✅ mirror | +| KRO RGD, `${schema.spec.*}` template | yes | **equal** | ❌ refused (wrong) | ✅ mirror | +| nginx ConfigMap, `${host}`, no postBuild | yes | **equal** | ❌ refused (wrong) | ✅ mirror | +| Deployment env `${REGION}`, Flux postBuild | yes | **differ** (`us-east`) | ✅ refused | ✅ refuse | + +--- + +## 5. Two shapes of the fence, and which to build first + +### 5a. The precise instance — refuse a write over a diverged token + +The cheapest correct fence, and the one with **no false positives**: *when a write would replace a +source value that still carries a `${...}` token with a different live value, refuse it.* This is the +render-vs-live rule scoped to the one case we can be certain of. kustomize provably never touches a +`${...}` token, so **our render always keeps it**; if the live object has a *different* value there, +something out of band changed it — not the user, not our build. It catches the Flux +postBuild / envsubst class, per field, and it fires on none of the literal-token documents in §4's +table, because there the live value still *is* the token. + +It applies to **any** mirrored document, not only kustomize-governed ones: for a plain folder our +"render" is the source itself, so the rule reduces to *"live differs from a source token ⇒ refuse,"* +which is exactly right for a plain folder Flux deploys with postBuild. + +### 5b. The general version — our render must reproduce live for the untouched set + +The broader fence catches more than tokens — Argo `spec.source.kustomize` overrides, version skew, +anything that makes the applied object differ: *before trusting our render as the baseline, require +it to reproduce the live object for every field the write does not deliberately change.* + +It is strictly more powerful and strictly harder, for one honest reason: **a live object legitimately +drifts from Git for reasons that are not out-of-band render context.** An HPA changed `replicas`; a +defaulting webhook filled a field; another controller populated something. A naive *"live ≠ render ⇒ +refuse"* would abort a flush because an HPA scaled a Deployment — which is not ours to police. +Distinguishing *context skew* from *ordinary runtime drift* is the open problem of the general fence +(§8), and it is why **5a is the right thing to build first**: it sidesteps the problem entirely by +keying on a token kustomize is *guaranteed* never to produce. + +--- + +## 6. Two surfaces of the one measurement + +The measurement — *does our render equal the live object?* — needs the live object, so it runs at +**reconcile time**, where the operator holds both the Git content and the watched live state. That +rules out only the **structure-only** acceptance gate (the CLI scan, the initial dry validation), +which has no cluster to look at — the trap the reverted structural check fell into. It does **not** +rule out the operator, which has live state in hand on every reconcile. And once the measurement runs +there, the answer is worth exposing two different ways. + +### 6a. A per-write refusal (§5) + +Point-in-time, per field: a write that would overwrite a source token whose live value diverged is +refused, in the family of `WriteBoundaryRefused`, naming the file, field, and token. This is the guard +that stops the corruption at the moment it would happen. It sits beside the source-form projection — +that decides *keep-source vs write-live* per field; this turns a *write-live* into a refusal when the +field it would overwrite holds a token the live object no longer has. It is per field, per object, so +one diverged token refuses one write, never a whole folder (the failure that broke CRD mirroring). + +### 6b. A GitTarget status you can read *before* you edit + +The same measurement, aggregated to the folder and surfaced as a standing **GitTarget condition** — +e.g. `RenderFaithful` — answers a more fundamental question than any single write does: + +> **Do we even have a chance of tracking this folder?** + +Because if our render does not match what the cluster runs, *nothing* we do on the folder is +trustworthy — not the mirror, not edit-through, not the refusal decisions themselves — since all of +them reason from a baseline that is wrong. A per-write refusal tells you *this edit* could not land; a +`RenderFaithful=False` condition tells you *this whole folder* is deployed with context we cannot +reproduce (Flux postBuild, Argo `spec.source.kustomize`, a divergent version), so you learn it **up +front, from status, before you waste an edit** — rather than one refusal at a time. + +It carries a bounded sample of the diverging `(file, field)` pairs, in the style of the +`FullyReflected` condition in +[unreflectable-edits-and-write-gating.md](unreflectable-edits-and-write-gating.md), and it is a +sibling of that condition: `FullyReflected` says *everything you edited was expressed*; +`RenderFaithful` says *our render matches what is running, so we can be trusted at all* — the more +fundamental of the two. It is recomputable: the mark-and-sweep resync rebuilds it from scratch, +steady-state events keep it current. + +This is exactly what the reverted structural check was reaching for and could not have — a +folder-level *"can we track this?"* verdict. It failed because it tried to answer from the **disk**; +the same question, answered from the **live object**, is both correct and precisely the up-front +signal a user wants. + +**And the condition blocks.** A `GitTarget` *is* the claim that a folder can be reverse-GitOps'd — +live changes captured faithfully back to Git. If our render is not equal to what the cluster runs +that claim is void: we cannot reverse a state we cannot reproduce. So `RenderFaithful=False` gates +adoption exactly as a structural refusal does — the folder is **not tracked**, `Ready=False`, with a +reason naming the diverging fields — not a soft warning beside a folder we quietly mishandle. This is +deliberately the strict choice, and it can be loosened later (mirror for audit, refuse only the +writes) *if* a demand for tracking read-only-but-unfaithful folders proves out. Strict first, because +the failure it prevents is silent corruption, and loosening a gate is reversible where a shipped +corruption is not. + +### How it composes with the oracle + +`VerifyBatchRenders` checks our render reproduces live **after** a write, sharing our render's blind +spot. The fidelity measurement checks our render reproduces live **before** it, at the fields we are +not touching — catching the blind spot the oracle cannot. + +--- + +## 7. Implementation: where the comparison runs + +The blocking decision forces the timing: `RenderFaithful` must be computed where the operator holds +both the Git content and the live objects, and **before any write** — because the first mirror of an +unfaithful folder *is* the corruption. That point already exists, which is why the shape you +suggested is the right one. + +### The one predicate, computed once, read by both surfaces + +Everything reduces to a single per-document test: + +> **unfaithful(doc) := our render of the Git document carries a `${...}` token at a field where the +> live object holds a resolved value.** + +kustomize never emits a `${...}` token, so our render preserves the ones the source carries; if the +live object has a *different, resolved* value there, something out of band produced it. The per-write +refusal (§6a) is this predicate on the one document a write touches; the folder condition (§6b) is the +same predicate ORed across the folder. Build it once and read it twice. + +It is the **token** form (5a) we block on, not the general render≠live form (5b), and that is a +*requirement* of blocking rather than a shortcut: 5b would flag an HPA that scaled a Deployment, and +blocking a folder for ordinary runtime drift would refuse to track a folder that is perfectly fine. A +`${...}` token — which kustomize provably never produces and an HPA never introduces — has no such +false positive. 5b, and the `managedFields` discriminator it needs, is the follow-on (§8). + +### Where to run it — three ways + +**(A) Fold it into the reconcile that runs on acceptance — recommended, and the shape you described.** +When a folder is accepted, the watch opens with `SendInitialEvents` and enqueues a scoped +**mark-and-sweep resync ahead of any live event** (the `replaying` barrier; +[`target_watch.go`](../../../internal/watch/target_watch.go), +[`resync_flush.go`](../../../internal/git/resync_flush.go)). That resync already scans the whole +subtree *and* replays the live objects — the one moment both halves are in hand and nothing has been +written. Add the predicate as a **resync precondition**, beside the write-boundary ones: render the +roots (already done for the oracle), walk each rendered object against its live counterpart for a +token divergence, and if any document is unfaithful, **abort the resync's writes and set +`RenderFaithful=False`**. That same abort is what stops the corruption — an unfaithful resync would +otherwise mirror `us-east` over `${REGION}` on the spot. *Cost:* one field-walk on top of a render we +already do — milliseconds. + +**(B) A distinct live-aware acceptance layer.** Keep structure-only `Accept` as it is (fast, no +cluster), and add a *second* gate — `AcceptRenderFaithful(store, liveObjects)` — that runs at the same +resync moment but is named and tested as its own function. Behaviourally this is (A); the difference is +packaging. Worth it only if the seam buys clarity: structure-only acceptance answers *"can we parse +and route this?"*, the fidelity gate answers *"does our render match reality?"*, and separate pure +functions keep each independently testable. + +**(C) Derive the folder verdict purely from per-write refusals.** Don't compute a folder pass at all — +refuse each unfaithful write (§6a) and flip `RenderFaithful=False` the first time one is refused for +this reason. Simplest, and it keeps 6a and 6b in one code path — but it makes the folder *"faithful +until proven otherwise, one write at a time"*, so a user only learns it is untrackable **after** +attempting an edit. That is exactly the up-front property 6b exists to give, so (C) delivers 6a and +loses the point of 6b. + +**Recommended: (A).** It is where the reads already happen, it runs before any write — so it both sets +the verdict and prevents the corruption in one step — and it produces the up-front folder answer. (B) +is (A) with a cleaner seam, a reasonable refinement. (C) is the fallback that keeps only the per-write +half. Whichever computes the folder pass, the steady-state per-write check (§6a) still runs between +resyncs, so a folder that becomes unfaithful later (postBuild added after adoption) flips the +condition the moment the first such write is refused — 6a keeps 6b current. + +### How it blocks, and how it clears + +`RenderFaithful` is a GitTarget condition. False means the folder is **not tracked**: the resync +commits nothing, the branch worker opens no window for it, and `Ready=False` carries a reason +(`RenderNotFaithful`) with a bounded sample of the diverging `(file, field, token)`. It sits second to +`GitPathAccepted` — structure-only acceptance is the first gate (*parseable, routable?*), +`RenderFaithful` the second (*does our render match reality?*), and a folder must pass both. It is +**recomputable and self-healing**: rebuilt from scratch on every resync, so removing the postBuild +config — or moving the tokens out of the tracked subtree — flips it back to True on the next reconcile +with no manual acknowledgement. + +--- + +## 8. Open questions + +- **5a vs 5b, and sequencing.** 5a (token) is precise and cheap — build first. 5b (general + render-vs-live) is the complete answer but needs the runtime-drift discriminator before it is safe. +- **The runtime-drift discriminator.** Can we separate "an HPA changed replicas" from "postBuild + changed an env" without hand-written per-field policy? **`managedFields` looks promising and should + be measured**: a field owned by the GitOps controller's apply is render context; a field owned by + `hpa`, `kubelet`, or a defaulter is drift. If that holds, 5b becomes safe. +- **Argo overrides leave no token.** `spec.source.kustomize.images` produces no `${}`; only 5b — or + orchestrator awareness — catches it. Is it acceptable to catch the token class first and the + Application-override class later, or do they need to land together? +- **Orchestrator awareness as the third, most complete fence.** Reading the Flux Kustomization / Argo + Application (the [interpreter model](orchestrator-knowledge-boundary.md)) would let us *know* + postBuild/overrides are configured and refuse — or eventually model — them directly, emitting the + `TransformedOutOfBand` claim that doc already reserves. It is the most work; 5a is the same + protection for the substitution class **without** reading the orchestrator's object. +- **Version skew.** Neither fence fully catches a pure version difference that changes a render + subtly but touches nothing we compare; [render-root-scoping.md §3](render-root-scoping.md)'s "pin + to the version Flux ships" stays the mitigation. 5b *does* catch a version difference that actually + moves an untouched object.