Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ folder by **lifecycle**, not by topic.
5. [`design/support-boundary/support-contract.md`](design/support-boundary/support-contract.md)
— **what the operator edits, what it refuses, and why.**

Provider-specific setup that needed writing down:
[`azure-devops-getting-started.md`](azure-devops-getting-started.md) — Azure DevOps end to end, why its
credential Secret is shaped differently from every other provider's, and the three test layers that
cover a provider CI cannot reach.

## The contracts — [`spec/`](spec/)

The code cites these. Breaking one without updating it is how the next person gets
Expand Down Expand Up @@ -66,7 +71,7 @@ says what we support and refuse** — and then its
kustomize field taxonomy, the write boundary, the orchestrator/expansion line, and
how secrets are handled.

Sixteen other open items:
Seventeen other open items:

| Doc | Open question |
|---|---|
Expand All @@ -85,6 +90,7 @@ Sixteen other open items:
| [`e2e-finish-plan.md`](design/e2e-finish-plan.md) | remaining e2e harness work |
| [`sensitive-resource-diagnostics-follow-up.md`](design/sensitive-resource-diagnostics-follow-up.md) | deferred diagnostics |
| [`e2e-git-server-choice.md`](design/e2e-git-server-choice.md) | stay on Gitea or move to Forgejo — the `_csrf` pin is fixable in place on both, so the migration is now a preference call, not a fix; also why we adopt no SDK either way |
| [`azure-devops-multi-ack.md`](design/azure-devops-multi-ack.md) | **decided and built: go-git v6** — why Azure DevOps rejects our fetches, and what to do instead of PR [#292](https://github.com/ConfigButler/gitops-reverser/pull/292)'s bundled `git` binary. The capability filter fails in two independent halves: advertising `multi_ack` is a four-line change, but v5 then cannot parse the multi-ACK **response**, which only a fetch with `have` lines provokes. That is why **Flux ships ADO support on v5 with no git binary — it never fetches**, only `CloneContext`, so it never enters the path v5 cannot serve; our persistent-clone-plus-incremental-fetch design is the opposite, which makes the trim alone insufficient for us. **go-git v6 already implements `multi_ack`** (PR #1204, in every v6 tag; upstream then deleted their ADO example saying it "works out of the box"), and its churn in the packages we import runs 96 → 39 → **1** → **9** removals per alpha, so it is one settled breaking wave rather than a moving target; the migration is four known API removals over two rewritten files, `transport.AuthMethod` being the invasive one. Prices PR #292 as measured rather than argued: the image goes **217 MB → 940 MB**, of which 723 MB is a `cp -rL` that dereferences 165 hardlinks to one binary (a one-character fix), arm64 is unaffected and native, but **Trivy reports zero findings on both images** while the new one carries git 2.54.0, OpenSSH 10.3p1 and OpenSSL 3.5.7 as loose files no package database describes — so the CRITICAL gate is blind to a third of the runtime. Also catches an unflagged non-ADO regression (`Depth: 1` dropped, so every provider full-fetches) and 10% patch coverage on an untestable path. The unlock is that **canonical `git upload-pack` advertises `multi_ack`** (verified), so the Gitea already in the e2e lab plus a 400-injecting proxy is a faithful ADO simulator — no tenant needed, and the only way any option becomes CI-testable. Four options priced, and Option A (v6) is the one shipped. Carries a measured **capability matrix** over our three network calls with two diagrams, which narrows the blast radius to **one call, `repo.Fetch`**: `receive-pack` never advertises `multi_ack` (measured), so **the atomic push is out of scope for every option** — its safety rests on the same-session advertisement plus the server-side `Old`/`New` compare-and-swap in `packp.Command`, neither of which touches `upload-pack`, and we already push from a shallow store today. v6 keeps that pattern 1:1 (`Handshake` → `GetRemoteRefs`/`Push`, same `[]*packp.Command`), which is an argument *for* migrating. Records what the migration actually cost, including the four v6 behaviour changes it surfaced — two of them settings v6 reads from the environment and fails closed on, invisible to unit tests |
| [`watchrule-source-namespace/`](design/watchrule-source-namespace/README.md) | letting a WatchRule address differently-named namespaces on its source cluster — a deny-by-default `allowedSourceNamespaces` on the **GitTarget** (so scope is per-tenant, not a provider-wide union), unlocked by a false-by-default delegation flag on the ClusterProvider. Five PRs: three landed prerequisite scope fixes (the namespace-blind resync sweep that would delete other namespaces' manifests, the cluster-wide/named stream collapse, and ClusterWatchRule's unchecked GitTarget attachment), then the breaking **scope-by-kind** change — `WatchRule.spec.rules[].sourceNamespace` (a name or `"*"` for the target's admitted set) and a cluster-scope-only ClusterWatchRule — and a GitTarget `prune.mode` that makes the resync sweep opt-in, released together with it |

## Deferred, but still wanted — [`future/`](future/)
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ If you only want the supported product docs, start with the files below.
document: Flux `HelmRelease`, Argo CD `Application`, KRO, and core resources all mirror and edit alike
- [`commit-signing.md`](commit-signing.md): how valid Git signatures map to platform verification
- [`github-setup-guide.md`](github-setup-guide.md): GitHub repository and credential setup
- [`azure-devops-getting-started.md`](azure-devops-getting-started.md): Azure DevOps repository and credential setup
- [`attribution-setup-guide.md`](attribution-setup-guide.md): naming real Kubernetes users as commit
authors via kube-apiserver audit delivery
- [`sops-age-guide.md`](sops-age-guide.md): Secret encryption with SOPS + age
Expand Down
124 changes: 124 additions & 0 deletions docs/azure-devops-getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# Getting started with Azure DevOps

Mirror a cluster into an Azure DevOps repository. Only the credential differs from any other
provider, and it differs in a way that trips people up.

Prerequisites: the operator installed (see the [root README](../README.md)), and an ADO repository you
can write to. An empty repository is fine — the operator creates the branch.

## 1. A Personal Access Token

**User settings → Personal access tokens → New Token**, in the organization that owns the repository.
Scope **Code (read & write)**; read alone is not enough, because the operator pushes. Set an expiry
you are willing to rotate, and copy the token when it is shown — ADO never displays it again. The
token inherits its user's permissions, so that user needs write access to the repository.

## 2. Namespace and Secret

ADO sends a PAT as HTTP basic auth with the token as the **password** and ignores the username, so the
Secret carries `password` and no `username`:

```bash
kubectl create namespace my-namespace

kubectl create secret generic ado-creds \
--namespace my-namespace \
--from-literal=password='<your PAT>'
```

A `username` is optional here and ADO ignores whatever you set, so leaving it out is simplest. What
does not work is a `username` with no `password`: the password is what selects basic auth.

## 3. The resources

Replace `<branch>` with your repository's default branch (`main` on a new ADO repository), the same
value in both places:

```yaml
apiVersion: configbutler.ai/v1alpha3
kind: GitProvider
metadata:
name: ado-provider
namespace: my-namespace
spec:
url: https://dev.azure.com/<org>/<project>/_git/<repo>
secretRef:
name: ado-creds
allowedBranches:
- <branch>
---
apiVersion: configbutler.ai/v1alpha3
kind: GitTarget
metadata:
name: ado-target
namespace: my-namespace
spec:
providerRef:
name: ado-provider
branch: <branch>
path: clusters/my-cluster
---
apiVersion: configbutler.ai/v1alpha3
kind: WatchRule
metadata:
name: ado-rule
namespace: my-namespace
spec:
targetRef:
name: ado-target
rules:
- resources: ["configmaps"]
```

```bash
kubectl wait --for=condition=Ready gitprovider/ado-provider -n my-namespace --timeout=60s
```

## 4. Check it works

```bash
kubectl create configmap ado-demo -n my-namespace --from-literal=greeting=hello
```

A commit appears on `<branch>` under
`clusters/my-cluster/my-namespace/configmaps/ado-demo.yaml`. Commits are authored by the configured
committer; audit delivery is only needed to attribute them to the Kubernetes user who made the change
(see [configuration.md](configuration.md)).

## Other credentials

SSH (`ssh://git@ssh.dev.azure.com/v3/<org>/<project>/<repo>`) and Entra ID bearer tokens use the same
Secret keys as any other provider. **Neither is tested against ADO**, unlike the PAT path above; both
are expected to work, as they do for GitHub.

## If fetches fail with HTTP 400

```text
TF401041: Clients must support multi-ack.
```

ADO rejects fetches from clients that do not advertise `multi_ack`, which go-git only implements from
v6. Releases before [#297](https://github.com/ConfigButler/gitops-reverser/pull/297) cannot fetch from
ADO and no configuration changes that — upgrade.

Two things make this confusing to debug: `GitProvider` can still reach `Ready`, because the
connectivity check is a different request that never needed the capability, and pushes still work,
because the push protocol has no `multi_ack` at all. Details and sources:
[`facts/azure-devops-multi-ack-requirement.md`](facts/azure-devops-multi-ack-requirement.md).

## Testing against a real tenant

CI cannot reach ADO, so two layers are opt-in and skip themselves without configuration:

```bash
export E2E_ADO_REPO_URL='https://dev.azure.com/<org>/<project>/_git/<repo>' # written to
export E2E_ADO_PAT='<your PAT>'
export E2E_ADO_EMPTY_REPO_URL='.../_git/empty' # optional; must name a repository with no commits

go test ./internal/git/ -run TestADOLive -v # library
task test-e2e-ado # operator, needs a prepared e2e cluster
```

`TestADOLive_StillRequiresMultiAck` is a canary: if it fails, Microsoft fixed their end.
[`internal/git/ado_multiack_test.go`](../internal/git/ado_multiack_test.go) covers the same rule in CI
with no credential.
11 changes: 10 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,17 @@ which match the built-in Secret types and the tooling around them (`kubectl crea
| Auth | Keys |
|---|---|
| SSH | `ssh-privatekey` (+ optional `ssh-password` passphrase, `known_hosts`) |
| HTTP basic | `username` + `password` |
| HTTP basic | `password` (+ optional `username`) |
| HTTP bearer token | `bearerToken` (GitHub fine-grained PAT, GitLab access token; no username) |

**`password` is what selects HTTP basic auth**, and `username` is optional: a Secret with only
`password` authenticates with an empty username, which is what Azure DevOps expects for a Personal
Access Token. ADO ignores the username entirely; measured, any value including none is accepted.
A `username` with no `password` is an error, because that one is a real mistake.

Note that an empty value and an absent key are the same thing to the credential reader, so
`username: ""` behaves exactly like omitting it.

#### Reusing a Flux or Argo CD credentials Secret

The credential reader's design is **inspired by both Flux and Argo CD**: it accepts their Secret key
Expand Down Expand Up @@ -1198,4 +1206,5 @@ The chart value reference for the starter `quickstart` block lives in

- [commit-signing.md](commit-signing.md) for signing behavior on Git hosting platforms
- [github-setup-guide.md](github-setup-guide.md) for GitHub auth setup
- [azure-devops-getting-started.md](azure-devops-getting-started.md) for Azure DevOps auth and setup
- [sops-age-guide.md](sops-age-guide.md) for `GitTarget.spec.encryption`
Loading