🤖 feat: expose template source files via CoderTemplate spec.files

[ThomasK33]

Summary

Expose Coder template source files via the aggregated API by adding spec.files (a map[string]string of relative paths → UTF-8 contents) to CoderTemplate. This enables kubectl get codertemplate <org>.<name> -o yaml to include template source, and enables GitOps controllers to create/update templates by specifying source files inline.

Background

The aggregated API server already exposes CoderTemplate resources with template metadata, but users and GitOps tooling couldn't view or manage the template's actual source tree (Terraform files, Dockerfiles, etc.) through the Kubernetes API. This change bridges that gap.

Implementation

API type (api/aggregation/v1alpha1/types.go)

• Added Files map[string]string field to CoderTemplateSpec with json:"files,omitempty"
• Regenerated deepcopy via make codegen

Storage helper (internal/aggregated/storage/template_files.go)

• fetchTemplateSourceFiles(): downloads the active version's source zip via codersdk.TemplateVersion → codersdk.DownloadWithFormat, unpacks, validates paths/sizes, filters non-UTF8 files
• buildSourceZip(): creates deterministic zip from file map for upload
• Enforced safety limits: 20 MiB compressed, 40 MiB total extracted, 2000 files max, 2 MiB per file
• Full defensive programming: nil assertions, zip-slip path protection, UTF-8 validation

Storage Get/Create/Update (internal/aggregated/storage/template.go)

• Get: populates spec.files from the active version's source archive
• List: intentionally omits spec.files to keep responses small
• Create with files: builds zip → uploads → creates template version → creates template
• Update with files: diffs current vs desired files; if changed, uploads new zip → creates version → promotes to active. If identical, no-op (GitOps idempotency)
• Update metadata: reconciles displayName, description, icon via UpdateTemplateMeta

Convert helper (internal/aggregated/convert/template.go)

• Added TemplateUpdateMetaRequestFromK8s() for building metadata update payloads

Tests (internal/aggregated/storage/storage_test.go)

• Extended mock Coder server with file upload/download, template version creation, metadata update, and active version promotion handlers
• Added tests: TestTemplateStorageGetPopulatesSpecFiles, TestTemplateStorageListOmitsSpecFiles, TestTemplateStorageCreateWithFiles, TestTemplateStorageUpdateWithChangedFiles, TestTemplateStorageUpdateWithIdenticalFilesIsNoOp, TestTemplateStorageUpdateMetadata
• Updated existing update tests to match relaxed immutability (metadata now mutable)

Docs

• Regenerated docs/reference/api/codertemplate.md to include the new field

Validation

• make verify-vendor ✅
• make build ✅
• make test ✅
• make lint ✅
• make docs-check ✅

Risks

• Non-atomic update: metadata patch and source update are sequential codersdk operations. If source step fails after metadata succeeds, the template may have updated metadata but old source. This is inherent to the codersdk API design.
• Memory pressure: GET requests now download + unpack source archives inline. Bounded by safety limits (20 MiB compressed) but concurrent GETs on large templates could increase memory usage. A future /source subresource could mitigate this.

---

📋 Implementation Plan

Plan: Expose Coder template source files via CoderTemplate.spec.files

Context / Why

Today the aggregated API server exposes CoderTemplate and CoderWorkspace as Kubernetes-native resources, but it only returns template metadata (name, description, version IDs, etc.). In the Coder UI, users can view a template version’s source tree (Terraform, Dockerfile, etc.).

We want kubectl get codertemplate <org>.<name> -o yaml to include the template’s source files so downstream tooling/UIs can display the same information via the aggregated API.

Chosen approach (Option A): Add an inline spec.files map (path → file contents) to CoderTemplate and have the aggregated API server populate it from the active template version’s source archive stored in Coder.

Evidence (repo + SDK)

• Aggregated API types:
  • api/aggregation/v1alpha1/types.go defines CoderTemplateSpec/Status but has no source fields.
  • api/aggregation/v1alpha1/zz_generated.deepcopy.go is generated and will need regen after API changes.
• Aggregated API storage:
  • internal/aggregated/storage/template.go Get() calls Coder (sdk.TemplateByName) and returns convert.TemplateToK8s(...).
  • List() calls sdk.Templates(...) and converts each template; we should avoid per-template source downloads here.
• Conversion layer:
  • internal/aggregated/convert/template.go maps codersdk.Template.ActiveVersionID → CoderTemplate.spec.versionID.
• Coder SDK supports fetching template version + archive:
  • (*codersdk.Client).TemplateVersion(ctx, id) in vendor/github.com/coder/coder/v2/codersdk/templateversions.go.
  • TemplateVersion.Job.FileID (source archive ID) in vendor/.../codersdk/provisionerdaemons.go.
  • (*codersdk.Client).DownloadWithFormat(ctx, fileID, codersdk.FormatZip) in vendor/.../codersdk/files.go.
• API reference docs are generated and include CoderTemplate fields:
  • docs/reference/api/codertemplate.md (generated).

Implementation details

1) API: add spec.files field (text-only)

File: api/aggregation/v1alpha1/types.go

Add a new optional field to CoderTemplateSpec:

// CoderTemplateSpec defines the desired state of a CoderTemplate.
type CoderTemplateSpec struct {
    // ...existing fields...

    // Files is the template source tree for the template’s *active* version.
    //
    // - Keys are relative, slash-delimited paths (e.g. "main.tf", "modules/vpc/main.tf").
    // - Values are UTF-8 file contents.
    // - Callers may set this field on CREATE/UPDATE to manage template source (GitOps).
    // - The aggregated API server populates this field on GET requests.
    // - This field is intentionally omitted from LIST responses to keep them small.
    // - Binary / non-UTF8 files are rejected or skipped (future: add encoding support).
    Files map[string]string `json:"files,omitempty"`
}

Notes/constraints to bake into the implementation:

• Keep this best-effort and bounded: do not allow a single request to download/unpack arbitrarily large archives.
• If we later want binary support, we should migrate to a richer type (e.g. map[string]CoderTemplateFile{content, encoding}) rather than stuffing base64 into a string.

Generated artifacts:

• Regenerate api/aggregation/v1alpha1/zz_generated.deepcopy.go (via make codegen).
• Regenerate docs/reference/api/codertemplate.md (via make docs-reference or the repo’s canonical docs target).

2) Storage: populate spec.files in TemplateStorage.Get() only

File: internal/aggregated/storage/template.go

Change Get() from returning convert.TemplateToK8s(...) directly to:

1. Convert template → k8s object.
2. Fetch active template version (sdk.TemplateVersion(ctx, template.ActiveVersionID)).
3. Download the archive from the files API (sdk.DownloadWithFormat(..., codersdk.FormatZip)).
4. Unzip and build the map.
5. Set obj.Spec.Files = files.

Shape:

obj := convert.TemplateToK8s(namespace, template)

files, err := fetchTemplateSourceFiles(ctx, sdk, template.ActiveVersionID)
if err != nil {
    return nil, err
}
obj.Spec.Files = files

return obj, nil

Helper: unzip + safety limits

Add a helper function (either near TemplateStorage or in a small internal helper file like internal/aggregated/storage/template_files.go) to keep Get() readable:

const (
    // NOTE: These limits gate how much data we’ll pull back and inline into a single
    // Kubernetes API response. Keep them bounded to avoid memory/latency issues.
    maxTemplateSourceZipBytes               = 20 << 20 // 20 MiB compressed archive cap
    maxTemplateSourceTotalUncompressedBytes = 40 << 20 // 40 MiB across all extracted files
    maxTemplateSourceFiles                  = 2000     // cap number of files
    maxTemplateSourceFileBytes              = 2 << 20  // 2 MiB per file
)

func fetchTemplateSourceFiles(
    ctx context.Context,
    sdk *codersdk.Client,
    versionID uuid.UUID,
) (map[string]string, error) {
    // 1) sdk.TemplateVersion(...)
    // 2) sdk.DownloadWithFormat(..., codersdk.FormatZip)
    // 3) validate size + content-type
    // 4) unzip and read each file with limits
    // 5) validate paths are relative + clean (no ../, no leading /)
    // 6) utf8.Valid(...) check; skip invalid
}

Trade-offs of raising limits (e.g. 20–40 MiB archives)

• codersdk.DownloadWithFormat buffers the entire archive in memory; larger limits increase per-request allocations and GC pressure.
• Unzipped content is returned inline as JSON/YAML. A 40 MiB extracted template means a 40+ MiB API response, which can:
  • be slow / time out through the kube-apiserver aggregation proxy,
  • be expensive for kubectl/GitOps controllers to render/diff,
  • increase OOM risk under concurrent GETs.
• GitOps CREATE/UPDATE has an additional constraint: kube-apiserver enforces a max request body size (flag --max-request-bytes), so very large YAML manifests may be rejected before reaching the aggregated API server.

If we need to support very large templates, we should consider:

• making these limits configurable via aggregated-apiserver flags, and/or
• a future /source subresource that streams zip/tar instead of inlining all file contents.

Defensive programming expectations:

• Assert required inputs are non-nil/non-empty.
• Explicitly error if Coder returns an unexpected content-type.
• Reject pathological archives (too many files, total zip bytes too large, per-file too large).
• Protect against zip-slip style paths even though we’re not writing to disk.

Keep LIST fast

File: internal/aggregated/storage/template.go

Do not populate spec.files in List():

• It would multiply Coder API calls by N templates.
• It would make kubectl get codertemplates -o yaml huge.

So List() continues to call convert.TemplateToK8s(...) and returns objects with spec.files unset.

3) GitOps: support creating/updating templates from spec.files

Files: internal/aggregated/storage/template.go, internal/aggregated/convert/template.go

Goal: enable GitOps controllers (ArgoCD/Flux) and kubectl apply to create and reconcile Coder templates from YAML by treating spec.files as the desired template source of truth.

CREATE behavior (template does not exist yet)

In TemplateStorage.Create(...):

• If spec.files is provided (len > 0):
  1. Validate and normalize the file map (paths are clean/relative; UTF-8; enforce size limits).
  2. Build a zip archive in-memory from spec.files.
  3. Upload archive to Coder: sdk.Upload(ctx, codersdk.ContentTypeZip, reader) → fileID.
  4. Create a new template version (unattached):
     • sdk.CreateTemplateVersion(ctx, org.ID, codersdk.CreateTemplateVersionRequest{ StorageMethod: file, FileID: fileID, Provisioner: terraform })
  5. Create the template using that version as the initial active version:
     • sdk.CreateTemplate(ctx, org.ID, codersdk.CreateTemplateRequest{ Name: templateName, VersionID: templateVersion.ID, ...meta })
• Else (no files): keep existing behavior that creates a template from a pre-existing spec.versionID.

UPDATE behavior (reconcile template to match YAML)

Expand TemplateStorage.Update(...) beyond the current “spec.running only” legacy behavior:

• Metadata reconcile

  • Allow spec.displayName, spec.description, and spec.icon to be updated via sdk.UpdateTemplateMeta(...).
  • Keep metadata.name + spec.organization immutable (still derived from name).

• Source reconcile (spec.files)

  • If the update request includes spec.files:
    1. Fetch current source for the template’s active version (use the same fetchTemplateSourceFiles helper).
    2. If desired files are byte-for-byte equal to current files: no-op (prevents creating a new template version every sync).
    3. Otherwise:
       • Zip desired files → upload (/api/v2/files).
       • Create a new template version attached to the template (TemplateID: template.ID).
       • Promote it to active via sdk.UpdateActiveTemplateVersion(template.ID, codersdk.UpdateActiveTemplateVersion{ID: newVersion.ID}).

• Patch / server-side apply compatibility

  • Verify kubectl apply and a JSON merge-patch update flow work end-to-end.
  • Confirm server-side apply / merge-patch work end-to-end. TemplateStorage already satisfies rest.Patcher (Getter+Updater), but we should verify managedFields/resourceVersion behavior in practice.

Notes on idempotency + drift

• The “compare desired files to current files” step is important; without it, GitOps controllers that re-apply manifests could create a new Coder template version on every sync loop.
• For very large templates, fetching + unpacking current source during UPDATE can be expensive. If this becomes a bottleneck, we can add a status-level checksum (or reuse Coder’s job.file_id) to short-circuit equality checks.

4) Tests: extend mock Coder server to serve template source archive

File: internal/aggregated/storage/storage_test.go

The existing newMockCoderServer() already supports template CRUD plus:

• GET /api/v2/templateversions/{id}

Extend it to also support GitOps + source flows:

• GET /api/v2/files/{fileID}?format=zip (download source archive)
• POST /api/v2/files (upload source archive)
• POST /api/v2/organizations/{orgID}/templateversions (create template version)
• PATCH /api/v2/templates/{templateID} (update template metadata)
• PATCH /api/v2/templates/{templateID}/versions (promote active version)

Implementation approach:

• Keep an in-memory filesByID map[uuid.UUID][]byte in mock state and seed it with a deterministic zip for the starter template version.
• Implement upload to stash request bytes and return a new UUID in the hash JSON field.
• Implement create template version to persist TemplateVersion.Job.FileID from the request’s FileID and optionally attach it to a template when TemplateID is set.

Add/extend tests:

• Get() returns spec.files with expected contents (e.g. main.tf).
• List() omits spec.files.
• CREATE with spec.files succeeds and subsequent GET returns those files.
• UPDATE with changed spec.files creates a new template version + promotes it; UPDATE with identical files is a no-op (GitOps idempotency).
• Failure-mode tests for archive/file limits.

5) Docs + examples

• Run docs regeneration so docs/reference/api/codertemplate.md includes the new spec.files field.
• Consider adding a short docs/ page or README note explaining:
  • spec.files is the desired template source for GitOps. The server returns it on GET but omits it on LIST.
  • size/encoding limitations and current hard limits (20MiB zip / 40MiB extracted / 2MiB per file).
  • GitOps caveats: kube-apiserver max request size may need tuning for large manifests; spec.versionID reflects the active version and may change when source changes (configure Argo/Flux ignore rules if needed).
  • security implications (source code becomes readable to anyone with RBAC get on codertemplates).

Validation checklist (what to run in Exec mode)

• make codegen
• make docs-reference (or repo docs target)
• make test
• make lint

---

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh • Cost: $0.92

[ThomasK33]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfc03f7520

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed both review comments:

• P1: Added fetchRawTemplateSourceZip and buildMergedSourceZip to preserve non-UTF8/binary files from the original archive during spec.files updates. Added TestTemplateStorageUpdatePreservesNonUTF8Files.
• P2: Added normalizeFileKeys that runs path.Clean on desired file keys before the no-op comparison. Added TestTemplateStorageUpdateNormalizesPathsForNoOp.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8b29e0d60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed P1: Added post-condition verification after UpdateActiveTemplateVersion — re-fetches the template via sdk.Template(ctx, templateID) and asserts the active version actually changed. Added TestTemplateStorageUpdateVerifiesActiveVersionPromotion with a mock mode that simulates silent promotion failure (returns 200 but doesn't update state).

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 98564f6cde

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed P1: Replaced sdk.DownloadWithFormat with direct sdk.Request + io.LimitReader(resp.Body, maxTemplateSourceZipBytes+1) so the size limit is enforced during the read, capping memory to ~20 MiB regardless of actual archive size.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e890c4ec5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed both comments:

• P1: buildMergedSourceZip now uses zipWriter.CreateHeader(&header) for preserved binary entries, retaining original file metadata (mode bits, modtime, compression). Test extended to verify 0755 mode on binary.dat.
• P2: Added immutability check for spec.versionID — mutations are rejected with BadRequest explaining to use spec.files instead. Added TestTemplateStorageUpdateRejectsVersionIDChange.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b118a4b9d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed P1: buildMergedSourceZip now preserves original ZIP headers for UTF-8 files that already exist in the archive. Uses zipWriter.CreateHeader with the original header (mode bits, modtime) for updated text files, and zipWriter.Create only for newly added files. Test verifies main.tf retains 0755 mode after content update.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 84df43ce9b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed P1: Reordered Update() to pre-validate spec.files (normalize keys + build trial zip) before any sdk.UpdateTemplateMeta call. Invalid files now fail fast before any mutations, preventing metadata/source drift from partial updates.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd380c9aab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed both comments:

• P1: versionID immutability check now only rejects non-empty mismatches (updatedTemplate.Spec.VersionID != "" && ...). Empty/omitted versionID is permitted for GitOps clients. Added TestTemplateStorageUpdateAllowsEmptyVersionIDWhenTogglingRunning.
• P2: fetchTemplateSourceFiles now rejects duplicate normalized paths with duplicate normalized path %q in template source zip.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9b42a34d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed P2: Moved duplicate normalized-path detection in fetchTemplateSourceFiles to before the UTF-8 filter. Now uses a separate seenPaths set that tracks ALL normalized paths (including non-UTF8 entries) to reject duplicates consistently with buildMergedSourceZip.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa23172add

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed both P2 comments:

• Empty files create: Changed condition from len(templateObj.Spec.Files) > 0 to templateObj.Spec.Files != nil so explicit empty maps (files: {}) trigger files-based create.
• Backslash rejection: Added strings.ContainsRune(templatePath, '\\') check in validateTemplateSourcePath before path.Clean.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

https://mux.md/MYpl8#uKlpzFhFXyUtfw