feat: auto-label namespaces with Istio ambient mesh on AgentRuntime reconcile by r3v5 · Pull Request #403 · rossoctl/operator

r3v5 · 2026-06-04T14:25:32Z

Summary

Automatically labels namespaces with Istio ambient mesh labels when an AgentRuntime CR is reconciled, enabling ztunnel L4 mTLS for agent-to-agent traffic without manual namespace configuration.

Jira: [RHAIENG-5502]

What it does

Patches namespace with istio-discovery=enabled and istio.io/dataplane-mode=ambient during AgentRuntime reconciliation
Sets IstioMeshEnrolled status condition on the AgentRuntime CR (True/False with reason)
Opt-out: annotate namespace with kagenti.io/istio-mesh=disabled to skip labeling
Labels persist after AgentRuntime deletion (no cleanup by design)
Short-circuits PATCH when labels already present (avoids unnecessary API calls)
Non-fatal: reconcile continues if namespace patch fails (e.g., missing RBAC, no Istio installed)

Changes

File	What
`internal/controller/agentruntime_istio.go`	New — `ensureIstioMeshLabels()` method, constants, RBAC marker
`internal/controller/agentruntime_controller.go`	Step 4.6 wiring — call method, set condition, emit event
`config/rbac/role.yaml`	Generated — `patch` verb added for namespaces
`internal/controller/agentruntime_istio_test.go`	5 unit tests (envtest)
`test/e2e/e2e_test.go`	5 e2e tests (Kind)
`test/e2e/fixtures.go`	4 e2e fixture functions
`docs/api-reference.md`	Document `IstioMeshEnrolled` condition
`docs/architecture.md`	Add namespace `patch` to RBAC table

Design decisions

Opt-out, not opt-in: auto-labels by default; skip only with explicit annotation
Non-fatal errors: mesh enrollment is additive security, not a functional prerequisite — clusters without Istio still have working AgentRuntimes
No cleanup on deletion: labels are namespace-level infrastructure; removing them when one CR is deleted could disrupt other workloads
Every reconcile re-checks: handles drift without a dedicated namespace watcher (YAGNI)
Inside AgentRuntime controller: not a separate controller — follows existing pattern of namespace-scoped operations (e.g., ensureNamespaceConfigMaps)

Test plan

make test — all existing + 5 new unit tests pass
make manifests — RBAC generated correctly
E2E tests — 5 new tests pass on Kind
Kind + Istio ambient — namespace labeled, ztunnel HBONE proxy, SPIFFE certs issued, L4 mTLS verified via ztunnel logs
OpenShift (ROSA) + OSSM 3 — namespace auto-labeled, opt-out works, drift recovery works
OpenShift cross-namespace mTLS — two AgentRuntimes in separate namespaces, different SPIFFE identities, L4 mTLS confirmed via ztunnel logs (port 15008 HBONE)

Cross-Namespace L4 mTLS Verification — OpenShift (RHAIENG-5502)

Setup

Two agents in separate namespaces, neither pre-labeled with Istio labels.
The operator auto-labels each namespace when its AgentRuntime CR is reconciled.

Namespace	Workload	Role
`mesh-agent-a`	`agent-a` (quay.io/redhattraining/hello-world-nginx:v1.0, port 8080) + Service	HTTP server
`mesh-agent-b`	`agent-b` (quay.io/curl/curl:8.5.0, sleep container)	HTTP client

1. Cross-Namespace Traffic

$ kubectl exec $AGENT_B_POD -n mesh-agent-b -- curl -s http://agent-a.mesh-agent-a.svc:8080
<html>
  <body>
    <h1>Hello, world from nginx!</h1>
  </body>
</html>

2. ztunnel Logs — L4 mTLS Proof

$ kubectl logs $ZTUNNEL_POD -n istio-ztunnel --tail=10 | grep mesh-agent

ztunnel enrolled pods from both namespaces

2026-06-04T12:50:14.634493Z  info  inpod::statemanager  pod received, starting proxy
    uid="e53e25db-f789-48f1-b739-29e6a488f8eb"
    name="agent-b-869cf9c95d-d2vq4"
    namespace="mesh-agent-b"

Inbound (ztunnel terminates mTLS, delivers to agent-a)

2026-06-04T12:51:02.466086Z  info  access  connection complete
    src.addr=10.129.0.189:37150
    src.workload="agent-b-869cf9c95d-d2vq4"
    src.namespace="mesh-agent-b"
    src.identity="spiffe://cluster.local/ns/mesh-agent-b/sa/default"
    dst.addr=10.129.0.187:15008
    dst.hbone_addr=10.129.0.187:8080
    dst.service="agent-a.mesh-agent-a.svc.cluster.local"
    dst.workload="agent-a-586747b957-mgkx2"
    dst.namespace="mesh-agent-a"
    dst.identity="spiffe://cluster.local/ns/mesh-agent-a/sa/default"
    direction="inbound"
    bytes_sent=308 bytes_recv=92 duration="0ms"

Outbound (ztunnel intercepts agent-b, wraps in mTLS)

2026-06-04T12:51:02.466166Z  info  access  connection complete
    src.addr=10.129.0.189:41658
    src.workload="agent-b-869cf9c95d-d2vq4"
    src.namespace="mesh-agent-b"
    src.identity="spiffe://cluster.local/ns/mesh-agent-b/sa/default"
    dst.addr=10.129.0.187:15008
    dst.hbone_addr=10.129.0.187:8080
    dst.service="agent-a.mesh-agent-a.svc.cluster.local"
    dst.workload="agent-a-586747b957-mgkx2"
    dst.namespace="mesh-agent-a"
    dst.identity="spiffe://cluster.local/ns/mesh-agent-a/sa/default"
    direction="outbound"
    bytes_sent=92 bytes_recv=308 duration="4ms"

Key mTLS Indicators

Different SPIFFE identities per namespace: ns/mesh-agent-b → ns/mesh-agent-a
HBONE tunnel port 15008: traffic wrapped in mTLS, not plaintext on port 8080
Both directions logged: ztunnel proxies both sides of the connection

3. Namespace Labels

$ kubectl get ns mesh-agent-a mesh-agent-b --show-labels
NAME           STATUS   AGE    LABELS
mesh-agent-a   Active   118s   istio-discovery=enabled,istio.io/dataplane-mode=ambient,...
mesh-agent-b   Active   117s   istio-discovery=enabled,istio.io/dataplane-mode=ambient,...

Both namespaces auto-labeled by operator — no manual labeling required.

4. AgentRuntime Status — mesh-agent-a

$ kubectl get agentruntime runtime-agent-a -n mesh-agent-a -o yaml
apiVersion: agent.kagenti.dev/v1alpha1
kind: AgentRuntime
metadata:
  name: runtime-agent-a
  namespace: mesh-agent-a
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: agent-a
  type: agent
status:
  conditions:
  - lastTransitionTime: "2026-06-04T12:50:13Z"
    message: Deployment agent-a resolved
    observedGeneration: 1
    reason: TargetFound
    status: "True"
    type: TargetResolved
  - lastTransitionTime: "2026-06-04T12:50:13Z"
    message: Namespace mesh-agent-a enrolled in Istio ambient mesh
    observedGeneration: 1
    reason: NamespaceLabeled
    status: "True"
    type: IstioMeshEnrolled
  - lastTransitionTime: "2026-06-04T12:50:13Z"
    message: Configuration resolved successfully
    observedGeneration: 1
    reason: ConfigResolved
    status: "True"
    type: ConfigResolved
  - lastTransitionTime: "2026-06-04T12:50:13Z"
    message: Workload agent-a configured with config-hash 57de3ab75a7e
    observedGeneration: 1
    reason: Configured
    status: "True"
    type: Ready
  phase: Active

5. AgentRuntime Status — mesh-agent-b

$ kubectl get agentruntime runtime-agent-b -n mesh-agent-b -o yaml
apiVersion: agent.kagenti.dev/v1alpha1
kind: AgentRuntime
metadata:
  name: runtime-agent-b
  namespace: mesh-agent-b
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: agent-b
  type: agent
status:
  conditions:
  - lastTransitionTime: "2026-06-04T12:50:14Z"
    message: Deployment agent-b resolved
    observedGeneration: 1
    reason: TargetFound
    status: "True"
    type: TargetResolved
  - lastTransitionTime: "2026-06-04T12:50:14Z"
    message: Namespace mesh-agent-b enrolled in Istio ambient mesh
    observedGeneration: 1
    reason: NamespaceLabeled
    status: "True"
    type: IstioMeshEnrolled
  - lastTransitionTime: "2026-06-04T12:50:14Z"
    message: Configuration resolved successfully
    observedGeneration: 1
    reason: ConfigResolved
    status: "True"
    type: ConfigResolved
  - lastTransitionTime: "2026-06-04T12:50:14Z"
    message: Workload agent-b configured with config-hash 57de3ab75a7e
    observedGeneration: 1
    reason: Configured
    status: "True"
    type: Ready
  phase: Active

Summary

Check	Result
Namespaces auto-labeled with Istio ambient labels	PASS
IstioMeshEnrolled condition True on both AgentRuntimes	PASS
Cross-namespace HTTP traffic works	PASS
ztunnel proxies traffic through HBONE port 15008	PASS
Different SPIFFE identities per namespace	PASS
L4 mTLS between mesh-agent-a and mesh-agent-b	PASS

Closes #399

Signed-off-by: Ian Miller milleryan2003@gmail.com
🤖 Co-Authored with Claude Code

Add ensureIstioMeshLabels() to AgentRuntime reconciler that patches namespaces with istio-discovery=enabled and istio.io/dataplane-mode=ambient for ztunnel L4 mTLS enrollment. - Opt-out via kagenti.io/istio-mesh=disabled annotation - IstioMeshEnrolled status condition (True/False with reason) - Non-fatal: reconcile continues if patch fails - Short-circuits when labels already present - RBAC: namespace patch verb added to manager-role Signed-off-by: Ian Miller <milleryan2003@gmail.com>

5 envtest cases for ensureIstioMeshLabels: - bare namespace gets labels - idempotent on already-labeled namespace - opt-out annotation skipped - existing labels preserved - non-"disabled" annotation value still labels Signed-off-by: Ian Miller <milleryan2003@gmail.com>

5 e2e test cases in new Describe("Istio Mesh Enrollment E2E"): - namespace auto-labeled on AgentRuntime create - IstioMeshEnrolled condition set to True - opt-out annotation prevents labeling - labels survive AgentRuntime deletion - drift recovery via reconcile Signed-off-by: Ian Miller <milleryan2003@gmail.com>

- api-reference.md: document IstioMeshEnrolled condition variants - architecture.md: add namespace patch verb to RBAC table Signed-off-by: Ian Miller <milleryan2003@gmail.com>

pdettori

Clean, well-structured PR. The ensureIstioMeshLabels() implementation correctly uses MergeFrom(DeepCopy()) for safe namespace patching, handles opt-out via annotation, short-circuits when labels already present, and is properly non-fatal. RBAC is correctly scoped (only patch added to namespaces, services split into read-only rule). Unit tests cover the main paths and E2E tests verify real cluster behavior including drift recovery. All CI checks pass.

Areas reviewed: Go (implementation + tests), YAML/RBAC, Docs
Commits: 4, all signed-off ✓
CI: 15/15 passing ✓

pdettori · 2026-06-08T13:38:19Z

 | `SkillsDiscovered` | True | `SkillsFound` | Linked skills discovered from `kagenti.io/skills` annotation on the target workload |
+| `SkillsMounted` | True | `SkillsApplied` | OCI skill ImageVolumes applied to the target workload |
+| `SkillsMounted` | False | `FeatureGateDisabled` | Skills defined but `skillImageVolumes` feature gate is disabled |
+| `SkillsMounted` | False | `UnsupportedWorkloadKind` | Skills defined but the target workload kind (e.g., Sandbox) does not support skill ImageVolumes |


suggestion: SkillsMounted condition docs (lines 483-485) appear unrelated to the Istio mesh enrollment feature — this condition isn't implemented anywhere in this diff. Consider moving to a separate commit or the PR that introduces the SkillsMounted logic, to keep this PR focused on its stated scope.

r3v5 requested a review from a team as a code owner June 4, 2026 14:25

clawgenti mentioned this pull request Jun 8, 2026

Weekly Report 2026-06-08 rossoctl/rossoctl#1840

Closed

r3v5 added 4 commits June 8, 2026 10:52

docs: add IstioMeshEnrolled condition and namespace RBAC

8cb9a4c

- api-reference.md: document IstioMeshEnrolled condition variants - architecture.md: add namespace patch verb to RBAC table Signed-off-by: Ian Miller <milleryan2003@gmail.com>

r3v5 force-pushed the auto-label-namespaces-with-istio-for-agent-runtime branch from b9c12f3 to 8cb9a4c Compare June 8, 2026 09:54

pdettori approved these changes Jun 8, 2026

View reviewed changes

pdettori merged commit 2585a3a into rossoctl:main Jun 8, 2026
15 checks passed

r3v5 mentioned this pull request Jun 9, 2026

Feat: add observedGeneration; simplify conditions #416

Merged

5 tasks

clawgenti mentioned this pull request Jun 15, 2026

Weekly Report 2026-06-15 rossoctl/rossoctl#1971

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: auto-label namespaces with Istio ambient mesh on AgentRuntime reconcile#403

feat: auto-label namespaces with Istio ambient mesh on AgentRuntime reconcile#403
pdettori merged 4 commits into
rossoctl:mainfrom
r3v5:auto-label-namespaces-with-istio-for-agent-runtime

r3v5 commented Jun 4, 2026 •

edited

Loading

Uh oh!

pdettori left a comment

Uh oh!

pdettori Jun 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

r3v5 commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What it does

Changes

Design decisions

Test plan

Cross-Namespace L4 mTLS Verification — OpenShift (RHAIENG-5502)

Setup

1. Cross-Namespace Traffic

2. ztunnel Logs — L4 mTLS Proof

ztunnel enrolled pods from both namespaces

Inbound (ztunnel terminates mTLS, delivers to agent-a)

Outbound (ztunnel intercepts agent-b, wraps in mTLS)

Key mTLS Indicators

3. Namespace Labels

4. AgentRuntime Status — mesh-agent-a

5. AgentRuntime Status — mesh-agent-b

Summary

Uh oh!

pdettori left a comment

Choose a reason for hiding this comment

Uh oh!

pdettori Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

r3v5 commented Jun 4, 2026 •

edited

Loading