feat(plugins): Per-plugin config + /readyz aggregation + /stats merge

[huang195]

Summary

Full per-plugin configuration cutover for AuthBridge. Plugins now own
their configuration end-to-end: they decode their subtree of the
runtime YAML, apply defaults, validate, and build their own internal
auth.Auth from it. The pre-migration top-level inbound: /
outbound: / identity: / bypass: / routes: blocks are gone.
Everything a plugin needs lives under
pipeline.*.plugins[].config.

Depends on #377 (structured rejections + plugin lifecycle hooks),
which has been merged into main. This branch is now rebased
directly on main.

Interfaces and YAML shape

• pipeline.Configurable — optional; plugins that need configuration
  implement it. Configure(raw json.RawMessage) error decodes with
  DisallowUnknownFields, applies its own defaults, validates, then
  builds internal state.
• pipeline.Initializer / pipeline.Shutdowner — already existed
  from feat(pipeline): Structured rejections + lifecycle hooks + plugin spec docs #377. Now used by jwt-validation and token-exchange to run
  background credential-file pollers.
• pipeline.Readier — optional; plugins that have deferred state
  report ready/not-ready. Pipeline.Ready() / NotReadyPlugin() aggregate.
• plugins.StatsSource — optional; plugins that maintain counters
  expose them for /stats aggregation.
• config.PluginEntry — {name, id, config: RawMessage}. YAML
  accepts both bare strings (- a2a-parser) and the full form
  (- {name: jwt-validation, config: {...}}). Explicit
  config: null is normalized to nil.
• plugins.Build — type-asserts Configurable, calls Configure with
  the entry's raw bytes, errors if a non-Configurable plugin got a
  config block.
• authlib/plugins/CONVENTIONS.md — the four-step pattern
  (decode → applyDefaults → validate → construct) for plugin
  authors, plus a note on the plugin-config / top-level strictness
  asymmetry.

Plugin migrations

jwt-validation.Configure decodes {issuer, jwks_url, audience, audience_file, audience_mode, bypass_paths}, derives
jwks_url from issuer when omitted, builds its internal
auth.Auth. Init spawns a background poll for audience_file when
the file isn't readable at Configure time; the poll runs on a
process-lifetime context and is cancelled in Shutdown. Ready()
reports when the audience has loaded.

token-exchange.Configure decodes {token_url, keycloak_url, keycloak_realm, default_policy, no_token_policy, identity: {...}, routes: {...}, audience_from_host}, derives token_url from
keycloak_url + keycloak_realm, builds its internal auth.Auth
(exchanger / cache / router / client-auth). Init handles async
credential-file loading via auth.UpdateIdentity. cfg stays
immutable after Configure returns; the background poller uses
goroutine-local variables. Ready() tracks credential arrival via an
atomic.Bool.

Kagenti-convention defaults

Most deployments don't need to spell these out:

Plugin	Field	Default
jwt-validation	jwks_url	<issuer>/protocol/openid-connect/certs
jwt-validation	audience_file	/shared/client-id.txt
jwt-validation	bypass_paths	/.well-known/*, /healthz, /readyz, /livez
jwt-validation	audience_mode	static
token-exchange	token_url	derived from keycloak_url + keycloak_realm
token-exchange	default_policy	passthrough
token-exchange	no_token_policy	deny (WARN if defaulted)
token-exchange	routes.file	/etc/authproxy/routes.yaml
token-exchange	identity.client_id_file	/shared/client-id.txt (both types)
token-exchange	identity.client_secret_file	/shared/client-secret.txt (client-secret)
token-exchange	identity.jwt_svid_path	/opt/jwt_svid.token (spiffe)

A minimum-viable config fits in ~15 lines. File-path defaults only
kick in when the matching inline value is also empty, so inline
credentials suppress file-read defaults. identity.type stays
required (spiffe vs. client-secret is deployment-level).

Behavior changes worth reading

• Schema cutover is breaking. Old-shape YAML (top-level
  inbound: / outbound: / identity: / bypass: / routes:)
  silently yields empty pipelines under yaml.v3's default
  forgiveness. config.Validate now rejects empty pipelines with
  an error naming the offending section and the likely cause
  (un-migrated schema).
• no_token_policy default changed. The pre-PR default was
  mode-specific (envoy-sidecar: client-credentials, waypoint: allow,
  proxy-sidecar: deny); the new uniform default is deny. Operators
  who relied on either old default get a boot-time WARN naming the
  three old defaults and how to pin behavior with one line of YAML.
• Credential polling is indefinite. The old 60s bounded poll
  became "poll forever" under per-plugin Init goroutines. A WARN
  fires every ~60s while a file is still missing so operators notice
  wrong paths / missing volume mounts.
• /readyz now gates on plugin readiness. Previously unconditional
  OK after Start; now ANDs Ready() across plugins, returns 503 +
  the not-ready plugin name while any plugin is still waiting. Kubelet
  holds traffic off the pod until all plugins have credentials.
• /stats now works. Previously returned empty counters; now
  aggregates per-plugin counters at each request.

Concurrency and lifetime

• bgCancel on both plugins is held in atomic.Pointer[context.CancelFunc]
  — safe even if Shutdown is called from a different goroutine than
  Init. Shutdown is idempotent via atomic Swap(nil).
• Pipeline.Start on Init failure unwinds successful Inits via
  reverse-order Shutdown before returning — closes the
  orphaned-goroutine window for embedded / multi-tenant callers.
• Plugins spawn background pollers with context.Background()-derived
  contexts, not Init's 60s budget, so a slow client-registration
  during pod boot doesn't orphan the plugin after Init's deadline.
• token-exchange.cfg is immutable after Configure returns;
  pollCredentials reads locals and feeds values through
  auth.UpdateIdentity.

Race detector clean on auth, pipeline, plugins, observe packages.

Schema changes (breaking)

• Removed from config.Config: Inbound, Outbound, Identity,
  Bypass, Routes — their content moved into per-plugin config.
• Removed from config/resolve.go: Resolve, deriveKeycloakURLs,
  deriveJWKSURL, resolveRouter, ResolveCredentialFiles,
  ResolveClientAuth. Kept ReadCredentialFile /
  WaitForCredentialFile as shared helpers used by both plugins.
• Registry factory: func() pipeline.Plugin (no more *auth.Auth).
• main.go: no global auth.Auth, no credential-load goroutine, no
  UpdateIdentity hotswap. Each plugin owns its own auth instance.

Test utilities

authbridge/authlib/plugins/plugintesting/ — stub plugin adapters
around a pre-built auth.Auth for listener tests. Lives in a
dedicated sub-package so production binaries can't import it; the
stubs mimic the real plugins' OnRequest without touching plugin
internals.

Demo manifests

• authbridge/authproxy/authbridge-combined.yaml — minimum-viable
  form using defaults.
• authbridge/demos/mcp-parser/README.md — minimum-viable form.
• authbridge/cmd/authbridge/README.md — both minimum and full
  forms, side-by-side, with a defaults table.
• authbridge/CLAUDE.md — updated "Configuration loading" bullets
  and the credential-wait gotcha.
• authbridge/authlib/README.md — package table + Usage example
  updated to go through plugins.Build rather than config.Resolve.

Test plan

• go build ./... and go vet ./... clean on both authlib and
  cmd/authbridge modules.
• go test ./... passes.
• go test -race ./auth/... ./pipeline/... ./plugins/... ./observe/... clean.
• TestAuthbridgeCombinedYAML_Loads parses the shipped default
  config with env vars set and asserts both pipelines Build + Validate.
• Configure validation matrix for both plugins (missing
  required, unknown fields, defaults, file paths, inline suppression).
• pipeline.Configurable contract coverage.
• Empty-pipeline rejection with error-message contract.
• PluginEntry YAML: bare strings, full form, null config.
• Pipeline.Start unwinds successful Inits on failure.
• Plugin Ready() matrix: sync-loaded, pending, per-host, inline
  credentials, pending credentials.
• auth.MergeStats correctness + non-destructive + nil/empty safe.
• plugins.CollectStats skips non-StatsSource plugins, nil-safe.
• Credential-file heartbeat branch reachable without panic.
• Local kind-cluster smoke test — deploy the weather-agent,
  verify inbound validation + outbound passthrough work end-to-end.
  (Pre-merge gate; not yet run.)

Draft

Still draft. #377 is merged and the branch is rebased on main.
Remaining gate before ready-for-review: local kind-cluster smoke
test (listed in Test plan above).

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

[cwiklik]

Major architectural cutover: plugins now own their configuration end-to-end via pipeline.Configurable. The old shared auth.Auth handler + top-level config sections are removed in favor of per-plugin config subtrees under pipeline.*.plugins[].config.

Well-designed safety mechanisms: empty pipeline rejection (prevents "open proxy" failure mode on upgrade), DisallowUnknownFields (catches config typos at startup), Start unwind (cleans up initialized plugins on downstream failure), and re-entrancy guards.

110 test functions covering edge cases. The Ready()/NotReadyPlugin() API gives operators clear kubectl describe diagnostics.

Areas reviewed: Go (architecture, concurrency, interfaces), Security, Docs
Commits: 9, all signed-off | CI: all passing

Two minor suggestions below — neither blocks merge.

[cwiklik]

nit: normalizeYAMLMaps traverses the full config subtree on every plugin's yamlNodeToJSON call. Fine for startup-only code, but worth noting if this ever moves to a hot path.

[cwiklik]

suggestion: Init spawns a context.Background() goroutine that polls indefinitely when the file never appears. The WARN log every ~60s is good operational visibility, but consider adding a metric counter (e.g. credential_file_poll_attempts) so operators can alert on it without grepping logs. Could be a follow-up.