feat(pipeline): Config hot-reload via fsnotify + atomic pipeline swap

[huang195]

Summary

Closes the observability gap where auth decisions and per-plugin actions were invisible in /v1/sessions and abctl. Introduces a unified plugin-invocation contract that every plugin emits, a 5-value action vocabulary shared across gate and parser plugins, a SessionDenied phase for rejected requests, and a rewritten abctl event pane that renders one row per plugin invocation with clear request↔response pairing.

New wire-schema

Every event on /v1/sessions/{id} and /v1/events carries an invocations field — a per-plugin record of what each plugin did during the pipeline pass:

{
  "phase": "request" | "response" | "denied",
  "invocations": {
    "inbound": [
      {
        "plugin": "jwt-validation",
        "action": "allow",
        "phase": "request",
        "reason": "authorized",
        "path": "/message/stream",
        "tokenSubject": "alice",
        "tokenAudience": ["..."],
        "tokenScopes": ["openid", "..."]
      }
    ],
    "outbound": [
      {
        "plugin": "token-exchange",
        "action": "modify",
        "phase": "request",
        "reason": "token_replaced",
        "routeMatched": true,
        "routeHost": "weather-tool-mcp",
        "targetAudience": "spiffe://...",
        "cacheHit": false
      }
    ]
  }
}

5-value action vocabulary

Every plugin MUST emit exactly one of these per invocation:

Action	Meaning	Example
allow	Gate plugin permitted the request	jwt-validation on valid token
deny	Gate plugin rejected the request; pipeline stops	jwt-validation on bad token
skip	Plugin ran but didn't act on this message	jwt-validation bypass path
modify	Plugin mutated the message	token-exchange replaced the Authorization header
observe	Plugin attached diagnostic data without altering flow	parsers extracting MCP/A2A/Inference state

Reason codes discriminate within an action value (skip/path_bypass vs skip/no_matching_route) for finer-grained filtering.

What is new in abctl

• One row per plugin invocation. An event with N plugins produces N visually-linked rows.
• ACTION column renders the 5-value vocab. Filter /allow, /deny, /skip, /modify, /observe.
• Leftmost # column gives each event an integer ID; paired req/resp events share one number. Method-aware pairing so notifications/initialized doesn't steal the tools/list response.
• Continuation rows blank TIME/DIR/PHASE/STATUS/DURATION/HOST/# when they share an event with the row above — event reads as one visual block.
• SessionDenied events render with PHASE=denied carrying full diagnostic context.

Backward compatibility

Breaking for pre-existing consumers of the session wire format. SessionEvent.Auth -> SessionEvent.Invocations, and InboundAuth.Decision / OutboundAuth.Action (free-form strings) unify into Invocation.Action (5-value typed enum). Session API has no persistent state, so no migration needed. No operator or chart changes.

Test plan

• go test ./... in authlib, cmd/authbridge, cmd/abctl — all green
• go vet ./... across all three modules — clean
• Manual: rebuilt authbridge-envoy image, deployed to Kind, drove A2A + MCP + inference traffic; verified one-row-per-plugin rendering, pair IDs, SSE response bodies, filters

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

[cwiklik]

Review Summary

Well-designed config hot-reload implementation:

1. pipeline.Holder — atomic.Pointer[Pipeline] wrapper with delegating methods (Run, RunResponse, NeedsBody, Ready). Concurrent access validated under -race (10 goroutine hammering + 1 swapper).
2. reloader package — fsnotify watcher targeting the directory (handles k8s ConfigMap symlink swaps), 250ms debounce to coalesce rapid events, SHA-256 content-hash dedup (ignores mtime-only touches), validateReloadable refuses mode/listener changes, bounded Start with rollback on partial failure (if outbound.Start fails, inbound is unwound), drain-window-based old pipeline cleanup in background goroutine.
3. Observability — /reload/status endpoint (last attempt, last success, error string, counters) and live /config reflecting the active post-swap config.

Correctness notes:

• Status mutations use copy-on-write under atomic.Pointer — safe because all mutations are serialized in the single watchLoop goroutine
• context.Background() for drain/unwind is deliberate and bounded (15s timeout)
• fsnotify v1.8.0 is well-maintained, no supply-chain concern

Nit: PR body describes session events/invocation wire schema (PR #386's scope) rather than the hot-reload feature this PR uniquely adds. Title is accurate though.

Areas reviewed: Go source (Holder, reloader, statserver, main.go wiring, listener refactors), Go tests (concurrency, debounce, hash dedup, mode-change refusal, builder error), Documentation
Commits: 14, all signed-off (DCO pass)
CI: All 17 checks passing
Stacking: Shares first 7 commits with PR #386; if #386 merges first, this PR's diff shrinks to the hot-reload commits only