🤖 feat: single-binary dual application (controller + aggregated API server)

[ThomasK33]

Summary

Transforms the coder-k8s binary into a dual-application binary that supports two independent modes via --app=controller|aggregated-apiserver, running from the same container image.

Background

The project needs an aggregated API server alongside the existing controller to serve CoderWorkspace and CoderTemplate resources via aggregation.coder.com/v1alpha1. Both applications must run independently — the aggregated API server does not depend on controller-runtime.

Implementation

Mode-based dispatch

• Refactored main.go into a thin entrypoint that delegates to a testable run(args) function in app_dispatch.go
• --app=controller starts the existing controller-runtime manager (extracted into internal/app/controllerapp/)
• --app=aggregated-apiserver starts the new aggregated API server (internal/app/apiserverapp/)
• Missing or invalid --app values fail fast with assertion-style errors

Aggregated API types

• New API group aggregation.coder.com/v1alpha1 with two resources:
  • CoderWorkspace — spec.running bool, status.autoShutdown *metav1.Time
  • CoderTemplate — same schema
• Types registered via k8s.io/apimachinery/pkg/runtime.SchemeBuilder (not controller-runtime wrapper)
• Deepcopy generated via updated hack/update-codegen.sh

Aggregated API server

• GenericAPIServer with self-signed TLS, anonymous auth, allow-all authz (scaffold defaults)
• API group installation with hardcoded in-memory storage for both resources
• Storage implements rest.Storage, rest.Getter, rest.Lister, rest.Scoper, rest.SingularNameProvider
• Each resource returns 3 deterministic placeholder objects across default and sandbox namespaces
• OpenAPI definitions provided; SkipOpenAPIInstallation=true for the scaffold phase

Deployment manifests

• deploy/controller-deployment.yaml — Deployment with --app=controller
• deploy/apiserver-deployment.yaml — Deployment with --app=aggregated-apiserver
• deploy/apiserver-service.yaml — Service for the aggregated API server
• deploy/apiserver-apiservice.yaml — APIService registration for v1alpha1.aggregation.coder.com
• deploy/rbac.yaml — ServiceAccount, auth-delegator ClusterRoleBinding, authentication-reader RoleBinding

Tests

• Mode dispatch: rejects empty, unknown, and stub modes
• Controller scheme registration and health probe
• Storage: List, Get, NotFound for both workspace and template
• Aggregated server: scheme registration, API group installation + discovery smoke test

Validation

• make test ✅ — all tests pass
• make build ✅ — binary compiles
• make verify-vendor ✅ — vendor is in sync
• gofmt -l ✅ — no formatting issues

Risks

• Low: The aggregated API server uses anonymous auth and allow-all authorization — appropriate for the scaffold phase but must be replaced with delegated auth before production use.
• Low: OpenAPI installation is skipped (SkipOpenAPIInstallation=true); full OpenAPI serving requires generated definitions.

---

📋 Implementation Plan

Plan: Single-binary dual application (controller + aggregated API server)

Context / Why

We need one coder-k8s binary that can run two distinct applications:

• the existing controller-runtime manager, and
• a new aggregated API server serving CoderWorkspace and CoderTemplate.

Per your direction, the aggregated API server must run independently from the controller-runtime instance. The cleanest shape is mode-based dispatch in one binary (one process runs one mode), with separate deployments using the same container image.

Evidence

• main.go currently always starts controller-runtime and has no mode dispatch.
• Dockerfile.goreleaser has a single entrypoint (/coder-k8s), which supports argument-based mode selection without changing images.
• go.mod does not include k8s.io/apiserver yet.
• Existing API types include only CoderControlPlane; no aggregated API group/resources exist yet.
• hack/update-codegen.sh currently generates deepcopy only for api/v1alpha1.

Implementation details

1. Refactor startup into explicit app modes in the existing root binary

   • Keep one main.go binary and add a required/validated selector flag (e.g. --app=controller|aggregated-apiserver).
   • Dispatch to isolated run paths; unknown values fail fast with assertion-style errors.

   switch appMode {
   case "controller":
       return runController(ctx)
   case "aggregated-apiserver":
       return runAggregatedAPIServer(ctx)
   default:
       return fmt.Errorf("assertion failed: unsupported --app %q", appMode)
   }

2. Move current controller-runtime wiring behind runController

   • Extract today’s manager/reconciler setup from main.go into a dedicated package (e.g. internal/app/controllerapp).
   • Preserve behavior (scheme registration, health/readiness probes, reconciler setup) so --app=controller is functionally equivalent to current behavior.
   • Keep defensive assertions (manager != nil, reconciler dependencies != nil).

3. Add aggregated API types for CoderWorkspace and CoderTemplate

   • Create package api/aggregation/v1alpha1 (group aggregation.coder.com, version v1alpha1).
   • Define resources with minimal scaffold fields:
     • spec.running bool
     • status.autoShutdown *metav1.Time
   • Register these types in a dedicated scheme builder and generate deepcopies.

   type CoderTemplateSpec struct {
       Running bool `json:"running"`
   }
   type CoderTemplateStatus struct {
       AutoShutdown *metav1.Time `json:"autoShutdown,omitempty"`
   }

4. Implement aggregated API server app behind runAggregatedAPIServer

   • Add dependencies: k8s.io/apiserver (+ any required companion libs like k8s.io/component-base).
   • Build a GenericAPIServer config with delegated authn/authz and secure serving.
   • Install one API group (aggregation.coder.com/v1alpha1) with two resources:
     • coderworkspaces
     • codertemplates
   • Keep this mode fully independent from controller-runtime manager startup.

5. Add hardcoded storage implementations for scaffolding

   • Add storage packages for both resources (e.g. internal/aggregated/storage/...).
   • Implement rest.Storage + rest.Getter + rest.Lister with deterministic hardcoded objects.
   • Return placeholder running states and fixed autoShutdown timestamps.
   • Add compile-time interface assertions and panic/return assertion failures for impossible conditions.

6. Update codegen and wiring scripts

   • Update hack/update-codegen.sh to include both API packages:
     • ./api/v1alpha1
     • ./api/aggregation/v1alpha1
   • Keep existing defensive checks for missing directories or empty go list results.

7. Deploy as two independent apps from one image

   • Add/adjust manifests so both workloads use the same image but different args:

   # controller deployment
   args: ["--app=controller"]
   
   # aggregated apiserver deployment
   args: ["--app=aggregated-apiserver"]

   • Add Service + APIService for aggregated mode (v1alpha1.aggregation.coder.com).
   • Include delegated-auth RBAC (system:auth-delegator and extension-apiserver-authentication-reader).

8. Tests and validation

   • Add tests for mode parsing/dispatch (controller mode, aggregated mode, invalid mode).
   • Add API scheme registration tests for new types.
   • Add storage tests for list/get + not found behavior.
   • Add aggregated server install smoke test (API group/resource registration).
   • Validate with make codegen, make verify-vendor, make test, make build.

Design choice: one process per mode (not both at once)

This keeps operational boundaries clear and matches your requirement that the aggregated API server run independently from controller-runtime. If we later want an "all" mode, we can add it explicitly without changing this base architecture.

---

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh • Cost: $0.36

[ThomasK33]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f875bf32ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed feedback: added coder-k8s-controller ServiceAccount in deploy/rbac.yaml and referenced it via serviceAccountName in deploy/controller-deployment.yaml.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 582530c2f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Added coder-k8s-controller ClusterRole + ClusterRoleBinding with RBAC rules matching the kubebuilder markers on CoderControlPlaneReconciler.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6a93a9bf6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Addressed: added args: ["--app=controller"] to config/e2e/deployment.yaml so the e2e workflow passes the required flag.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

https://mux.md/8LT8Z#tuVYPdCNfTF2eA