Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
394d8bf
docs: working on vision docs
sunib Jun 3, 2026
af4c98e
chore: first poc work
sunib Jun 3, 2026
cbd9d8e
chore: more details to check / get right
sunib Jun 3, 2026
7787485
chore: working on more situations / test based on reviews from codex 5.5
sunib Jun 3, 2026
e8d9924
chore: getting a write-up on next direction with abstraction
sunib Jun 3, 2026
21961df
chore: result of review round (more clarity).
sunib Jun 3, 2026
68b4428
chore: getting the details ironed out and getting started on implemen…
sunib Jun 3, 2026
6eb3787
docs: getting the plan ready for execution
sunib Jun 3, 2026
205bcb7
chore: implementing steps 1 - 5 in the design document (especailly th…
sunib Jun 3, 2026
2546e63
chore: now it should be ready to hook up
sunib Jun 3, 2026
d43d268
feat: support flexible manifest placement / editting
sunib Jun 3, 2026
e0b15ba
chore: fixes and cleanup
sunib Jun 3, 2026
17339c0
docs: getting all design docs together so that we can cleanup
sunib Jun 4, 2026
078276a
ci: add lock on e2e to prevent running it twice
sunib Jun 4, 2026
53d1d0d
chore: refactors and improvements
sunib Jun 4, 2026
25b7dc8
chore: fix regression in e2e
sunib Jun 4, 2026
4b0daa9
ci: more simple lock for now
sunib Jun 4, 2026
655950a
chore: adding concept cli tool to analyse a GitOps folder
sunib Jun 4, 2026
29db5c8
chore: removed the --watched for now (let's start simple)
sunib Jun 4, 2026
27ba3db
docs: continue designing for this approach
sunib Jun 4, 2026
7bd1f32
docs: getting better
sunib Jun 4, 2026
135820b
docs: getting more details in the plan
sunib Jun 4, 2026
815f7e8
chore: Is this really more effecient?
sunib Jun 4, 2026
24457fb
docs: the plan is ambitious now I would say
sunib Jun 4, 2026
d6e174b
docs: planning the implementation
sunib Jun 4, 2026
eb0478a
chore: landing A1 op A2 (partly?)
sunib Jun 4, 2026
7267aa2
chore: finishing phase A2
sunib Jun 4, 2026
1522c97
chore: implemented B1 and B2
sunib Jun 4, 2026
6527f7e
chore: getting b3 ready
sunib Jun 4, 2026
c407fa4
feat: prevent nested gittargets
sunib Jun 4, 2026
46df450
Merge remote-tracking branch 'refs/remotes/origin/poc/manifestedit' i…
sunib Jun 4, 2026
b0f971a
chore: hardening for failing call
sunib Jun 4, 2026
148d377
chore: first try on m3, but checking is always a good idea
sunib Jun 4, 2026
9e7c787
chore: m3 but now in the right way
sunib Jun 4, 2026
354978c
chore: first version of m4 and m5
sunib Jun 4, 2026
9e30d37
chore: refined m4 / m5 implementation
sunib Jun 4, 2026
11f14dc
ci: not running the specific image reload tests on local builds anymo…
sunib Jun 4, 2026
68dfd1a
ci: removing unused task directives
sunib Jun 4, 2026
35a0ab2
chore: implemented m6
sunib Jun 4, 2026
9af75d4
chore: m7 implementation phase 0
sunib Jun 4, 2026
cea1501
chore: m7 improvements
sunib Jun 4, 2026
57d391d
chore: first version of M8
sunib Jun 5, 2026
167a800
feat: immutable gittargets and gitdestinations
sunib Jun 5, 2026
f08ce97
chore: next set of fixes for m8
sunib Jun 5, 2026
39e02a6
feat: require value for GitTarget.Path, since hooking up GitTarget to…
sunib Jun 5, 2026
c2d3ce8
docs: desiging the next phase
sunib Jun 5, 2026
7b15e2d
docs: working on refining the design of this new pipeline
sunib Jun 5, 2026
30f9243
chore: bumping limits to have a reliable e2e run (caching is on the a…
sunib Jun 5, 2026
40ea764
chore: m10 implementation 0
sunib Jun 5, 2026
b10515a
chore: m10 part 2
sunib Jun 5, 2026
c3433cd
chore: m10 prevent wobbling crds by keeping them in table a bit longer
sunib Jun 5, 2026
c8dcffa
chore: cleaning up unused code paths?
sunib Jun 5, 2026
f9227fc
docs: fighting over abstractions, and what the exact value is of cert…
sunib Jun 8, 2026
a0ee87e
chore: add valkey-cli so for easier debugging
sunib Jun 8, 2026
1a7ae1e
chore: let's see if we can install valkey-cli now
sunib Jun 8, 2026
0f34d50
feat: support for subresources (working kubectl scale deployment)
sunib Jun 8, 2026
ce301fa
chore: updated k3d and dading general knowledge on subresources
sunib Jun 8, 2026
8610c02
chore: simpler approach to Scale
sunib Jun 8, 2026
428b5e7
chore: simplify type resolving
sunib Jun 8, 2026
24bd060
chore: let's go to the same (improved) abstraction
sunib Jun 8, 2026
401d194
docs: Let's just commit it then
sunib Jun 8, 2026
f529a87
chore: simplify further
sunib Jun 8, 2026
72df204
chore: re-enable serial and updating architecture
sunib Jun 8, 2026
1bd8af7
docs: capture contextual-namespace + SOPS single-file decisions, add …
sunib Jun 8, 2026
4682b74
chore: prototyping (very!) partial Kustomize support
sunib Jun 9, 2026
e3f0bd8
feat(typeset,watch): M12 first slices — type lifecycle events + per-t…
sunib Jun 9, 2026
18bed75
chore: hopefully fixing e2e tests
sunib Jun 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .claude/settings.local.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
"Bash(go:*)",
"Bash(ls:*)",
"Bash(cat:*)",
"Bash(task test-e2e:*)"
"Bash(task test-e2e:*)",
"Bash(k3d cluster *)",
"Bash(kubectl config *)"
]
}
}
14 changes: 12 additions & 2 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,19 +50,21 @@ RUN apt-get update \
# TASK_VERSION -> https://github.com/go-task/task/releases
# TILT_VERSION -> https://github.com/tilt-dev/tilt/releases
# ACTIONLINT_VERSION -> https://github.com/rhysd/actionlint/releases
# VALKEY_VERSION -> https://github.com/valkey-io/valkey/releases

ENV PATH="/go/bin:/usr/local/go/bin:${PATH}" \
KUBECTL_VERSION=v1.36.1 \
KUSTOMIZE_VERSION=5.8.1 \
KUBEBUILDER_VERSION=4.14.1 \
GOLANGCI_LINT_VERSION=v2.12.2 \
HELM_VERSION=v4.2.0 \
K3D_VERSION=v5.8.3 \
K3D_VERSION=v5.9.0 \
FLUX_VERSION=2.8.8 \
FLUX_OPERATOR_VERSION=0.50.0 \
TASK_VERSION=v3.51.1 \
TILT_VERSION=v0.37.3 \
ACTIONLINT_VERSION=1.7.12
ACTIONLINT_VERSION=1.7.12 \
VALKEY_VERSION=9.1.0

# https://github.com/fluxcd/flux2/releases
# https://fluxoperator.dev/
Expand Down Expand Up @@ -133,6 +135,14 @@ RUN asset="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" \
&& install -m 0755 actionlint /usr/local/bin/actionlint \
&& rm -rf "${tmpdir}"

# Install valkey-cli
# Valkey only ships prebuilt binaries for Ubuntu jammy/noble; the jammy build
# (glibc 2.35) is compatible with this bookworm image (glibc 2.36). Extract only
# the CLI binary from the release tarball.
RUN curl -fsSL "https://download.valkey.io/releases/valkey-${VALKEY_VERSION}-jammy-x86_64.tar.gz" \
| tar -xzO "valkey-${VALKEY_VERSION}-jammy-x86_64/bin/valkey-cli" > /usr/local/bin/valkey-cli \
&& chmod +x /usr/local/bin/valkey-cli
Comment on lines +138 to +144

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show the relevant section of the Dockerfile with line numbers
sed -n '120,170p' .devcontainer/Dockerfile | nl -ba

Repository: ConfigButler/gitops-reverser

Length of output: 115


🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n "VALKEY_VERSION|valkey-cli|checksum|sha256|sha512|gpg|asc|sign" .devcontainer/Dockerfile

Repository: ConfigButler/gitops-reverser

Length of output: 1524


🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n "valkey\.io/releases|download\.valkey\.io|valkey-${VALKEY_VERSION}|valkey-cli" -S .

Repository: ConfigButler/gitops-reverser

Length of output: 124


🏁 Script executed:

#!/bin/bash
set -euo pipefail
# Focus only on valkey-related lines and any checksum/signature verification nearby in the Dockerfile.
rg -n "valkey|checksums|sha256sum|sha512sum|gpg|asc|sig|sign" .devcontainer/Dockerfile

Repository: ConfigButler/gitops-reverser

Length of output: 2188


Verify valkey-cli artifact integrity before installation.

Lines 142-144 download and extract valkey-cli without checksum/signature verification, unlike nearby installs that validate SHA256 checksums. This weakens supply-chain guarantees for the dev/CI image.

🔒 Suggested hardening pattern
+# Pin and verify the release artifact before extracting valkey-cli.
+# (Use the official checksum published for this exact asset/version.)
 RUN curl -fsSL "https://download.valkey.io/releases/valkey-${VALKEY_VERSION}-jammy-x86_64.tar.gz" \
     | tar -xzO "valkey-${VALKEY_VERSION}-jammy-x86_64/bin/valkey-cli" > /usr/local/bin/valkey-cli \
     && chmod +x /usr/local/bin/valkey-cli
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.devcontainer/Dockerfile around lines 138 - 144, The RUN step that downloads
and installs valkey-cli using VALKEY_VERSION currently skips integrity checks;
update the installation to fetch and verify the release checksum (and signature
if available) before placing the binary into /usr/local/bin—specifically,
download the corresponding SHA256 (or .sha256sum) for
valkey-${VALKEY_VERSION}-jammy-x86_64, verify the tarball's checksum (e.g., via
sha256sum) and fail the build on mismatch, then extract the valkey-cli binary
and set chmod +x; optionally verify a detached signature or GPG key if the
project publishes one to further harden the install.


# Set working directory
WORKDIR /workspaces

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ jobs:
e2e_ginkgo_procs: "2"
k3d_agent_count: "0"
- name: full
script: "task test-e2e-full"
script: "task test-e2e"
needs_artifact: false
# Bumped 1 -> 4: the full suite was serialized to avoid the
# cross-target rule-change snapshot coupling, now fixed (per-target
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,10 @@ helm upgrade gitops-reverser \
The default quickstart namespace is `default`, so the `git-creds` Secret above should exist there
unless you explicitly set `quickstart.namespace` to something else.

The starter `GitTarget` writes under `live-cluster` by default. That keeps the first run away from
the repository root. To deliberately target the root instead, add
`--set quickstart.gitTarget.path=.` to the Helm command.

Check that the starter resources become ready:

```bash
Expand Down
3 changes: 0 additions & 3 deletions Tiltfile
Original file line number Diff line number Diff line change
Expand Up @@ -88,9 +88,6 @@ k8s_resource(

test_targets = [
'test-e2e',
'test-e2e-full',
'test-e2e-signing',
'test-e2e-manager',
'test-image-refresh',
]
for name in test_targets:
Expand Down
14 changes: 13 additions & 1 deletion api/v1alpha1/gitprovider_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,20 @@ import (
)

// GitProviderSpec defines the desired state of GitProvider.
//
// Only the repository URL is immutable. The URL is the destination identity that every
// referencing GitTarget materializes into; changing it would silently point those
// targets at a different repository and orphan their existing materialization (the same
// reason a GitTarget's destination is immutable). To repoint, delete and recreate the
// GitProvider. Everything else here is operational and deliberately stays mutable —
// notably allowedBranches (widening or narrowing the writable set is a normal change
// that must not require tearing down every GitTarget), plus auth, push tuning, and
// commit identity/signing.
//
// +kubebuilder:validation:XValidation:rule="self.url == oldSelf.url",message="spec.url is immutable; delete and recreate the GitProvider to point at a different repository"
type GitProviderSpec struct {
// URL of the repository (HTTP/SSH)
// URL of the repository (HTTP/SSH).
// Immutable: delete and recreate the GitProvider to point at a different repository.
URL string `json:"url"`

// SecretRef for authentication credentials (may be nil for public repos)
Expand Down
29 changes: 26 additions & 3 deletions api/v1alpha1/gittarget_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,42 @@ type GitProviderReference struct {
}

// GitTargetSpec defines the desired state of GitTarget.
//
// The destination fields — providerRef, branch, and path — are immutable. A
// GitTarget materializes the watched resources at exactly one (provider, branch,
// folder); changing where it writes would orphan the old materialization and require
// migrating manifests between repositories/branches/folders. Instead of reconciling
// that move, the destination is fixed: to relocate a GitTarget, delete it and create a
// new one. This keeps the one-owner-per-folder invariant and the initial-snapshot gate
// simple — a successful snapshot can never be silently invalidated by a destination
// change.
//
// +kubebuilder:validation:XValidation:rule="self.providerRef == oldSelf.providerRef",message="spec.providerRef is immutable; delete and recreate the GitTarget to change its destination"
// +kubebuilder:validation:XValidation:rule="self.branch == oldSelf.branch",message="spec.branch is immutable; delete and recreate the GitTarget to change its destination"
// +kubebuilder:validation:XValidation:rule="self.path == oldSelf.path",message="spec.path is immutable; delete and recreate the GitTarget to change its destination"
type GitTargetSpec struct {
// ProviderRef references the GitProvider or Flux GitRepository.
// Immutable: delete and recreate the GitTarget to change its destination.
// +required
ProviderRef GitProviderReference `json:"providerRef"`

// Branch to use for this target.
// Must be one of the allowed branches in the provider.
// Immutable: delete and recreate the GitTarget to change its destination.
// +required
Branch string `json:"branch"`

// Path within the repository to write resources to.
// +optional
Path string `json:"path,omitempty"`
// Path within the repository to write resources to, relative to the repository
// root. Required and must be non-empty — there is no default, so a GitTarget can
// never silently write to the repository root. To deliberately target the
// repository root, set it to "." (the ArgoCD/Flux convention); an empty string is
// rejected because it is too easy to leave blank by accident to be a deliberate
// root choice. Any leading slash (absolute path) and ".." are rejected, and a
// trailing slash is normalized away.
// Immutable: delete and recreate the GitTarget to change its destination.
// +required
// +kubebuilder:validation:MinLength=1
Path string `json:"path"`
Comment on lines +78 to +80

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

spec.path becoming required is upgrade-breaking for existing objects without the field.

This change can reject updates (including status updates) for pre-existing GitTarget resources that were created when path was optional, because they can round-trip as empty string and fail MinLength=1.

🛠️ Safer rollout approach
- // +required
- // +kubebuilder:validation:MinLength=1
- Path string `json:"path"`
+ // +optional
+ // TEMPORARY: keep optional for one migration window; enforce required after backfill.
+ Path string `json:"path,omitempty"`

Then run a migration/backfill to set explicit values (e.g. "." or intended folder), and re-enable strict required validation in a follow-up release.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// +required
// +kubebuilder:validation:MinLength=1
Path string `json:"path"`
// +optional
// TEMPORARY: keep optional for one migration window; enforce required after backfill.
Path string `json:"path,omitempty"`
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@api/v1alpha1/gittarget_types.go` around lines 78 - 80, The change made
spec.path required (Path field on GitTarget) is upgrade-breaking; revert the
immediate "+required" tag on Path and keep it optional (retain
kubebuilder:validation:MinLength=1 if you want validation for non-empty strings)
so existing GitTarget objects without the field won't be rejected, then
implement a migration/backfill that sets a sensible default (e.g., ".") for
existing resources and only after the cluster has been backfilled, re-add the
"+required" marker (or convert to kubebuilder:default) in a follow-up release;
locate the Path field on the GitTarget type (Path string `json:"path"`) to make
the change.


// Encryption defines encryption settings for Secret resource writes.
// +optional
Expand Down
4 changes: 2 additions & 2 deletions charts/gitops-reverser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ resources:
memory: 128Mi
limits:
cpu: 1000m
memory: 512Mi
memory: 1Gi

monitoring:
serviceMonitor:
Expand Down Expand Up @@ -169,7 +169,7 @@ nodeSelector:
| `quickstart.namespace` | Namespace for the starter quickstart resources | `default` |
| `quickstart.gitProvider.url` | Repository URL used by the starter `GitProvider` | `""` |
| `quickstart.gitProvider.secretRef.name` | Existing Secret name used by the starter `GitProvider` | `git-creds` |
| `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget` | `live-cluster` |
| `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget`; set `.` only to deliberately target the repo root | `live-cluster` |
| `quickstart.watchRule.rules` | Rules used by the starter `WatchRule` | `configmaps create/update/delete` |
| `queue.redis.addr` | Redis endpoint (`host:port`) for required durable audit queueing | `valkey:6379` |
| `queue.redis.auth.existingSecret` | Name of a pre-created Secret holding the Redis password | `valkey-auth` |
Expand Down
6 changes: 4 additions & 2 deletions charts/gitops-reverser/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,8 @@ rbac:
# Resource limits and requests
resources:
limits:
cpu: 500m
memory: 512Mi
cpu: 1000m
memory: 1Gi
requests:
cpu: 10m
memory: 256Mi
Expand Down Expand Up @@ -267,6 +267,8 @@ quickstart:
gitTarget:
name: example-target
branch: main
# Required relative repository path for the starter GitTarget. The chart
# default avoids repo-root writes; set "." only to deliberately target root.
path: live-cluster
encryption:
provider: sops
Expand Down
20 changes: 9 additions & 11 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ import (
"github.com/ConfigButler/gitops-reverser/internal/controller"
"github.com/ConfigButler/gitops-reverser/internal/git"
"github.com/ConfigButler/gitops-reverser/internal/queue"
"github.com/ConfigButler/gitops-reverser/internal/reconcile"
"github.com/ConfigButler/gitops-reverser/internal/rulestore"
"github.com/ConfigButler/gitops-reverser/internal/telemetry"
"github.com/ConfigButler/gitops-reverser/internal/types"
Expand Down Expand Up @@ -149,35 +148,34 @@ func main() {
)
fatalIfErr(mgr.Add(workerManager), "unable to add worker manager to manager")

// Create ReconcilerManager (will be set up as ControlEventEmitter)
reconcilerManager := reconcile.NewReconcilerManager(
nil, // eventRouter will be set after EventRouter is created
ctrl.Log.WithName("reconciler-manager"),
)

// Watch ingestion manager (placeholder, will get EventRouter set later)
watchMgr := &watch.Manager{
Client: mgr.GetClient(),
Log: ctrl.Log.WithName("watch"),
RuleStore: ruleStore,
EventRouter: nil, // Will be set below
AuditLiveEventsEnabled: true,
SensitiveResources: cfg.sensitiveResources,
}

// Initialize EventRouter with all dependencies
// Initialize EventRouter with all dependencies. The streaming-snapshot resync
// (M8) is driven directly through the worker, so there is no longer a separate
// reconciler manager / two-snapshot handshake.
eventRouter := watch.NewEventRouter(
workerManager,
reconcilerManager,
watchMgr,
mgr.GetClient(),
ctrl.Log.WithName("event-router"),
)
reconcilerManager.SetEventRouter(eventRouter)
reconcilerManager.SetOnReconcilerCreated(watchMgr.MaybeReplaySnapshot)

// Set EventRouter reference in WatchManager
watchMgr.EventRouter = eventRouter

// Inject the live followability registry into the writer, so a GVR-only DELETE
// event resolves to a manifest moved off its canonical path (M6 in the writer).
// The registry is a stable pointer the watch manager refreshes in place.
workerManager.SetMapper(watchMgr.TypeRegistry())

// WatchRule controller (with WatchManager reference for dynamic reconciliation)
fatalIfErr((&controller.WatchRuleReconciler{
Client: mgr.GetClient(),
Expand Down
Loading