Make PrefillDecode actually disaggregate

[dennis-upbound]

Problem

Across a full benchmark on a live GKE cluster, the vLLM prefill engine handled
0 requests, the decode engine served everything, remote_engine_id was
null, and no KV cache transferred. PrefillDecode was silently running
decode-only.

Root cause — three stacked defaults, all wrong on our EPP image

EPP image: llm-d-inference-scheduler:v0.8.0 (embeds gateway-api-inference-extension v1.5.0). Fixing any one alone leaves it broken:

1. prefix-based-pd-decider had no params → nonCachedTokens = 0, which the
   decider treats as disabled (always decode-only).
2. The decider reads a PrefixCacheMatchInfo attribute that nothing produced.
   GIE v1.5.0 split production out of prefix-cache-scorer into a separate
   plugin and made prepare-data default-on — so the prepareDataPlugins
   feature gate the v0.8.0 docs still tell you to set is unregistered and
   crashloops the EPP (feature gate 'prepareDataPlugins' is unknown or unregistered). The producer is now an explicit plugin:
   approx-prefix-cache-producer.
3. That producer defaults to autoTune: true, which leaves its block size 0
   and never populates the attribute. Must pin autoTune: false +
   blockSizeTokens.

(The v0.8.0 sample configs and docs/disaggregation.md are stale — they still
list the removed gate. This is effectively a packaging bug upstream; we work
around it with the correct v1.5.0 data-path config.)

Fix

Two pieces, both required for a request to actually disaggregate:

1. Arm the EPP decider. Add approx-prefix-cache-producer (autoTune: false,
   blockSizeTokens derived from the engine's --block-size/--page-size), set
   nonCachedTokens: 16, drop the feature gate, and wire the scorers into both
   profiles.
2. Inject the NIXL KV-transfer plumbing the schema can't express. Cross-pod
   NIXL needs VLLM_NIXL_SIDE_CHANNEL_HOST set to the pod IP (a fieldRef env)
   and a Memory-backed /dev/shm (a volume) — neither expressible in the
   ModelDeployment engine template (env.valueFrom allows only secret/configMap
   refs; no volumes) — the schema gap is tracked in ModelDeployment engine template can't express volumes #180. _inject_nixl_plumbing
   adds them to both phase engines,
   the same way the pd-sidecar is injected. Without the side-channel host the
   prefill advertises an unreachable address and the handshake fails (500/503,
   zero transfers). Also documents the engine-image prerequisite: the image must
   ship the NIXL runtime — recent vanilla vllm/vllm-openai tags do.

Verified live:

prompts	prefill engine request_prefill_time	behavior
5× long unique	+5	✅ disaggregates (KV flows prefill→decode over NIXL)
3× tiny ("hi")	+0 (flat)	✅ selectively skips prefill, decode-only
3× long unique	+3	✅ disaggregates

So disaggregation engages and is correctly selective (short/cache-hot prompts
skip the prefill hop and its KV-transfer cost).

Hard offload evidence (re-validated 2026-06-17, vanilla vllm/vllm-openai:v0.19.1)

Clean run: armed EPP + injected NIXL plumbing, 1 prefill (kv_producer) + 1 decode
(kv_consumer), five ~600-token prompts. vLLM's own NIXL counters:

engine	prompt_tokens	generation_tokens	nixl_xfer count	nixl_bytes_transferred
prefill (producer)	6220	5	—	—
decode (consumer)	(decode-side)	160	5	367 MB (3.67e8)

The prefill processes all prompt tokens and generates ~nothing (5 = handoff
tokens); the decode records exactly 5 NIXL transfers totaling 367 MB of KV
cache pulled from the prefill, then does the generation. That is the offload,
measured end-to-end — no custom kv-connector image, just vanilla vLLM that ships
NIXL.

Validated the failure modes too: the unarmed EPP config serves everything
decode-only (prefill prompt_tokens stays 0); missing VLLM_NIXL_SIDE_CHANNEL_HOST
returns 500/503 with zero transfers — i.e. both halves of this PR are load-bearing.

Alternatives considered

• always-disagg-pd-decider — also works on this image and is simpler, but
  disaggregates unconditionally (pays KV-transfer overhead even on tiny prompts).
  prefix-based is the better default; keep always-disagg in mind as a fallback if
  the producer wiring ever regresses.
• Downgrade to inference-scheduler v0.7.1 (GIE v1.4.0) — there the original
  featureGates: [prepareDataPlugins] config is correct, but it gives up v0.8.0.
  Rejected; the v1.5.0 config above is config-only and keeps us current.

Future considerations

• We may want to expose the EPP config as a specTemplate for full customization later on

Test plan

• compose-model-replica unit tests pass (43 + 8 subtests) — incl. test_epp_config_arms_the_pd_decider, test_kv_block_size, test_injects_nixl_plumbing
• ruff check / ruff format clean
• Verified live: selective disaggregation works (long → prefill engages, short → skips)
• Verified live: KV offload measured — 5 transfers / 367 MB prefill→decode over NIXL on vanilla vllm/vllm-openai:v0.19.1
• nix flake check (CI, incl. docs Vale)