test(e2e): serve Qwen3.5 0.8B on shared vLLM path

[fredespi]

Summary

• Serve Qwen/Qwen3.5-0.8B in the shared explicit-vLLM GPU precondition used by the real serve and chat scenarios.
• Keep the default-engine scenarios on their existing Lemonade-preferred targets, so engine resolution and the current xfail matrix remain unchanged.
• Pass the canonical Hugging Face ID directly to the explicit --engine vllm path; no model-catalog entry or CI prewarm change is required.
• Risk: the repository's current catalog records an unverified qwen3_5 architecture path with the pinned Transformers stack, so the MI300X GPU lane is the authoritative compatibility check.

Test plan

• cargo fmt --all -- --check
• cargo test -p e2e-cucumber --all-targets (23 passed)
• cargo clippy -p e2e-cucumber --all-targets -- -D warnings
• GPU CI lane confirms Qwen3.5 0.8B becomes ready and responds on vLLM.

[volen-silo]

Reviewed against the model catalog, the affected scenarios, and the xfail matrix. The explicit-engine size cut is a clean win, but the swap also silently changes default-engine resolution on Instinct, which invalidates the xfail matrix for two of the four scenarios the PR names. Requesting changes to either scope the swap or update expectations.toml (and re-verify on GPU).

🔴 The swap changes engine selection, not just model size

host_serve_target().0 feeds two different code paths:

1. setup_gpu_model — serves with an explicit --engine vllm. Here the model is the only variable, so 1.5B → 0.5B is a pure win. ✅
2. user_serves_default_engine (line 435) — serves with no --engine, so the CLI resolves the engine from the model's preferred_engines. This is where the behavior changes.

In model_catalog.json:

• old Qwen/Qwen2.5-1.5B-Instruct → preferred_engines: ["lemonade"], loader: llamacpp, dtype: gguf
• new Qwen/Qwen2.5-0.5B-Instruct → preferred_engines: ["vllm"], loader: transformers, dtype: float16

So rocm serve <model> --managed (no engine) on an Instinct host now resolves to vLLM instead of lemonade. That's the exact assumption the xfail entries for the default-engine scenarios rest on:

[["serve-default-engine-working-endpoint"]]   # and serve-default-engine-inference
when = { effective_engine = "vllm" }
bug = "EAI-7052"
reason = "Default serve resolves to a GGUF recipe on lemonade; its Vulkan backend hangs on Instinct, so the endpoint never reaches ready."

That reason is only true for a lemonade-preferred model. With the vLLM-preferred 0.5B it no longer holds.

Impact on the MI300X lane (e2e-gpu, non-blocking but the PR's "real verdict")

• serve-default-engine-working-endpoint asserts only engine auto-selected + model reachable (assert_model_reachable — a /v1/models check, no inference). vLLM does reach /v1/models ready on Instinct (EAI-7333 is inference-only), so both assertions now pass → XPASS against the EAI-7052 xfail → reconciliation exits 1 ("bug appears fixed; update expectations.toml").
• serve-default-engine-inference asserts inference. vLLM ready-but-refuses-inference (EAI-7333) means it still fails — so it stays green in reconciliation, but now for a different bug than its entry claims (EAI-7333, not the EAI-7052 lemonade hang). Stale/misleading attribution.

(Pre-existing wrinkle this now exposes: the feature comment at model_serving.feature:64–66 already attributes the xfail to EAI-7333, while expectations.toml says EAI-7052 — the swap makes that mismatch matter.)

Two ways to resolve

• (a) Scope the swap (minimal, behavior-preserving): give user_serves_default_engine its own model constant that stays lemonade-preferred (or otherwise resolves as before), so only setup_gpu_model's explicit-vLLM path takes the 0.5B size win. The default-engine scenarios and their xfail rationale are untouched.
• (b) Embrace it and update the matrix: drop the effective_engine = "vllm" / EAI-7052 entry for serve-default-engine-working-endpoint (it should now pass), re-attribute serve-default-engine-inference to EAI-7333, reconcile the feature comment, and confirm on the MI300X lane. This is arguably better coverage (a real default-vLLM endpoint instead of a permanent lemonade-hang xfail), but it must be done in this PR, not left to a red GPU lane.

🟡 Minor: now-inverted comment at serving_steps.rs:430

The illustrative example (e.g. Qwen2.5-1.5B → a GGUF recipe on lemonade) was accurate for the old model but is now the opposite of what happens (0.5B → vLLM). Update or generalize it.

✅ Good

• The explicit-engine path (setup_gpu_model) swap is sound: 0.5B is a real catalog entry, vLLM-preferred, safetensors, and already proven vLLM-servable by serve-vllm-default-on-instinct (user_serves_vllm_capable_default).
• The model-size policy doc comment is a genuinely useful guardrail and matches the code (vLLM 0.5B / lemonade 0.6B floors; 27B only in @nightly). All other repo 1.5B references are unrelated fixtures, correctly left alone.

[fredespi]

Thanks — confirmed against expectations.toml and the catalog, and you're right. host_serve_target().0 also feeds user_serves_default_engine (no --engine), so the 1.5B→0.5B swap flipped default-engine resolution on Instinct from lemonade to vLLM, which invalidates the EAI-7052 xfail for serve-default-engine-working-endpoint / -inference (XPASS → reconciliation exit 1).

Went with option (a) — scope the swap (4cd2c53): added default_engine_serve_target() (stays lemonade-preferred: 1.5B on Instinct, 0.6B GGUF on lemonade hosts) for the default-engine step, and kept the 0.5B size cut only on the explicit --engine vllm path (setup_gpu_model → serve-vllm-inference, serve-readiness-contract). Default-engine resolution is now byte-identical to pre-swap, so the xfail matrix is untouched and needs no GPU re-verification — Task #3's intent was a pure size cut with no behavior change, and option (b) (embrace default-vLLM + rewrite the matrix) is better filed as its own change.

The now-accurate illustrative comment (Qwen2.5-1.5B → a GGUF recipe on lemonade) at the default-engine step matches again since that path is back on 1.5B. The model_ids_match doc example correctly tracks the vLLM path at 0.5B.

[rominf]

Automated review: Approve.

The blocking concern from the earlier review is resolved at the current head. The model-size change no longer affects default-engine resolution: user_serves_default_engine uses the new default_engine_serve_target, preserving the pre-change 1.5B lemonade path, while host_serve_target now has only the explicit-vLLM setup caller. The 0.5B optimization is therefore scoped to the intended vLLM scenarios.

I also verified the touched model-policy comments against model_catalog.json; they match the catalog.

The non-blocking GPU job is red because existing expected-failure entries XPASSed, with no unexpected failures. The base already exhibited part of this expectation drift, and the default-engine path is unchanged by this PR. Refreshing those expectations belongs in a separate change.

[rominf]

Post-review update: commit 77076770 replaces the shared explicit-vLLM model with Qwen/Qwen3.5-0.8B. The separate default-engine target introduced in 4cd2c53 is unchanged, so the EAI-7052 resolution/xfail behavior remains isolated. Local fmt, focused tests (23 passed), and clippy are green. The new Qwen3.5 architecture is not yet a verified path in the repository's pinned runtime, so the MI300X lane is the decisive serve/readiness/inference check.

Fixed