Prefill/decode disaggregation design

[dennis-upbound]

Towards #34.

Issue #34 asks for prefill/decode disaggregation, but its original sketch predates the KServe drop and describes it through KServe's LLMInferenceService.prefill section. There's no current design or user-facing documentation for how disaggregation fits the backend-dispatcher architecture.

This adds two docs, no code:

• design/disaggregation.md specifies disaggregation on the current architecture: a self-contained prefill block on the deployment, KV transfer via the engine's --kv-transfer-config and NIXL, routing through the same swappable EPP on a GAIE InferencePool that unified serving uses, the correctness constraints to enforce as matching matures, and when disaggregation is worth it. It parallels design/modelcache.md, with an Alternatives considered section covering the KServe sketch, a bespoke proxy, a routing discriminator, Modelplane choosing per-role hardware, and an implicit prefill:decode ratio.
• docs/concepts.md gains a Disaggregated Serving section covering the prefill block, the compute-bound vs bandwidth-bound split, KV transfer, and the ModelCache and co-location requirements. It also updates the ModelCache source bullet to the shipped source enum shape.

It also carries one unrelated CI fix (Skip the checklist requirement on draft PRs), bundled here because it's what surfaced the failure on this draft. The checklist-completed job ran require-checklist-action on every pull_request event, so a WIP draft that hadn't filled out the template checklist failed the check. The commit gates the job on the PR not being a draft and adds the ready_for_review trigger, so it's skipped while a PR is a draft and runs once it's marked ready. It's a self-contained commit and can be split into its own PR if you'd rather not bundle it.

I have:

• Read and followed Modelplane's contribution process.
• Run nix flake check (or ./nix.sh flake check) and made sure it passes. Docs and CI-config change; the sandboxed check job passes in CI on this PR.
• Added or updated tests covering any composition function changes. No composition function changes.
• Signed off every commit with git commit -s.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
modelplane-docs	Error		Jun 10, 2026 11:08pm

[negz]

So for prefill decode we have the usual top level stuff, and prefill, and routing?

I think this sketch belongs in the main body of the design, not in an alternative, and could use a little more detail. It'd be useful to show it in context of a full deployment (that'd answer my question above).

Some other thoughts:

• I'm a little wary of defaulting it if omitted. It's convenient but asymmetrical with the other blocks, where we don't e.g. guess an engine config if you omit it.
• Where does the EPP pod run? (Does it need its own node selector etc etc?)

[dennis-upbound]

Good points — all addressed:

• Shape: yes — top-level workers (decode), prefill, and routing. Moved the sketch into the main body: the example up top is now a full deployment showing all three, and the Routing section describes routing.template directly. Trimmed the alternative to just the discriminator-vs-template rationale.
• Defaulting: agreed, dropped it. routing is now required when prefill is set (no guessed EPP), and both workers.count and prefill.workers.count are explicit (no default ratio) — symmetrical with how the engine is specified, per your point.
• Where the EPP runs: added — it’s a lightweight CPU Deployment (watches the pods, scores requests; no model, no GPU), on ordinary nodes in the serving namespace, no special nodeSelector, one per InferencePool. Its pod shape comes from routing.template.