Operator should read keycloak-admin-secret from kagenti-system, not agent namespaces

[Alan-Cha]

Problem

The operator reads keycloak-admin-secret from each agent namespace (team1, team2, etc.):

File: internal/controller/clientregistration_controller.go:167

adminSecret := &corev1.Secret{}
if err := r.uncachedReader().Get(ctx, 
    types.NamespacedName{Namespace: ns, Name: keycloakAdminSecret}, 
    adminSecret); err != nil {

Where ns is the workload's namespace (team1, team2, etc.).

Security issue: Compromised agent namespace = full Keycloak realm admin access (create/delete users, realms, clients, roles).

Proposed Fix

1. Change operator to read from kagenti-system

const operatorNamespace = "kagenti-system"

adminSecret := &corev1.Secret{}
if err := r.uncachedReader().Get(ctx, 
    types.NamespacedName{
        Namespace: operatorNamespace,  // ← Change this
        Name: keycloakAdminSecret,
    }, 
    adminSecret); err != nil {

The operator has ClusterRole permissions with cluster-wide secret access.

2. Remove admin secret from agent namespaces in Helm chart

File: charts/kagenti/templates/agent-namespaces.yaml:105-122 (kagenti repo)

Delete this entire section that replicates admin credentials:

{{- if not $root.Values.keycloak.adminExistingSecret }}
apiVersion: v1
kind: Secret
metadata:
  name: keycloak-admin-secret
  namespace: {{ . }}  # team1, team2, etc.
type: Opaque
stringData:
  KEYCLOAK_ADMIN_USERNAME: {{ $root.Values.keycloak.adminUsername | quote }}
  KEYCLOAK_ADMIN_PASSWORD: {{ $root.Values.keycloak.adminPassword | quote }}
{{- end }}

The secret should only exist in kagenti-system namespace (for the operator to read).

Impact

Before	After
Admin creds in every agent namespace	Admin creds only in kagenti-system
Compromised agent namespace = realm admin	Compromised agent namespace = no admin access

Implementation

This requires changes to TWO repos:

1. kagenti-operator: Change controller code to read from kagenti-system
2. kagenti: Remove secret creation from agent-namespaces.yaml Helm template

Related

• Long-term: kagenti/kagenti#1421 proposes SPIFFE DCR to eliminate admin credentials entirely

[Alan-Cha]

@akram Is the reason that the client secrets are still in the agent namespace for backwards compatibility with the old client registration sidecar?

I think we want to completely sunset the client registration sidecar so that we can completely remove client secrets from agent namespaces.

[akram]

@Alan-Cha I have noticed that also this week when doing additional tests for the redbank demo .... I don't know why it is still expecting the creds to be in the namespace. That looks like a regression.

I will investigate it and we can work on a fix and restoring the feature properly.