Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 36 additions & 4 deletions tests/e2e-cucumber/tests/e2e/serving_steps.rs
Original file line number Diff line number Diff line change
Expand Up @@ -275,14 +275,41 @@ async fn setup_mock_custom_port(world: &mut E2eWorld) {
/// single-engine steps (`setup_gpu_model`'s old vLLM body and
/// `setup_lemonade_model`), but lets a scenario tagged only `@requires-gpu` run
/// on whichever engine the platform actually uses.
///
/// MODEL-SIZE POLICY: GPU serve scenarios use small models that still satisfy
/// their assertions — real GPU serves are the E2E wall-clock long pole on serial
/// hardware, so weight-load time is pure overhead here. Current paths use vLLM
/// `Qwen3.5-0.8B` and lemonade `Qwen3-0.6B-GGUF`. Large-model behaviour is
/// exercised exactly once, in the `@nightly` `serve-large-model-inference`
/// scenario (Qwen3.6-27B) — never on the per-PR path. Do not raise a serve
/// scenario's model unless a smaller one genuinely cannot prove the assertion.
fn host_serve_target() -> (&'static str, &'static str, &'static str) {
if e2e_cucumber::capability::host_capability().effective_serve_engine == "lemonade" {
// GGUF via lemonade's llama.cpp backend; endpoint reports e.g.
// Qwen3-0.6B-Q4_0.gguf, so "Qwen3-0.6B" is the distinctive substring.
("Qwen3-0.6B-GGUF", "lemonade", "Qwen3-0.6B")
} else {
// Safetensors via vLLM; "Qwen2.5-1.5B" is the distinctive substring.
("Qwen/Qwen2.5-1.5B-Instruct", "vllm", "Qwen2.5-1.5B")
// Safetensors via vLLM; "Qwen3.5-0.8B" is the distinctive substring.
("Qwen/Qwen3.5-0.8B", "vllm", "Qwen3.5-0.8B")
}
}

/// The model the default-engine (no `--engine`) serve step should request.
///
/// This is deliberately NOT `host_serve_target` — that step passes no `--engine`,
/// so the CLI resolves the engine from the model's own `preferred_engines`, and
/// the model choice therefore determines WHICH engine runs. The default-engine
/// xfail matrix depends on that resolution: on an Instinct host the request must
/// resolve to a GGUF recipe on lemonade (EAI-7052 — see expectations.toml), NOT
/// to vLLM. So this stays lemonade-preferred on every host; the smaller model
/// used by vLLM applies only to the explicit-engine path. The model hangs at load
/// on Instinct (EAI-7052, xfail) so its size is irrelevant to
/// wall-clock here — behaviour preservation is what matters.
fn default_engine_serve_target() -> &'static str {
if e2e_cucumber::capability::host_capability().effective_serve_engine == "lemonade" {
"Qwen3-0.6B-GGUF"
} else {
"Qwen/Qwen2.5-1.5B-Instruct"
}
}

Expand Down Expand Up @@ -422,7 +449,12 @@ async fn user_serves_default_engine(world: &mut E2eWorld) {
// resolved (parsed from the serve plan), not the requested one — hardcoding the
// requested model made this time out on hosts where the recipe resolved to a
// different model, for reasons unrelated to engine selection.
let model = host_serve_target().0;
//
// Uses `default_engine_serve_target` (a lemonade-preferred model), NOT
// `host_serve_target`: the model here drives engine resolution, and the xfail
// matrix requires this to resolve to lemonade on Instinct (EAI-7052). See that
// fn's doc comment.
let model = default_engine_serve_target();
ensure_serve_port_free().await;
let (stdout, _, rc) = crate::run_rocm(world, &["serve", model, "--managed"]);
// The model the CLI resolved (what actually gets served) can differ from the
Expand Down Expand Up @@ -692,7 +724,7 @@ async fn assert_response_model_correct(world: &mut E2eWorld) {

/// Whether a chat response's `model` field identifies the model we served.
///
/// vLLM echoes the exact id we passed (`Qwen/Qwen2.5-1.5B-Instruct`), so a plain
/// vLLM echoes the exact id we passed (`Qwen/Qwen3.5-0.8B`), so a plain
/// containment holds. Lemonade instead reports the concrete GGUF artifact it
/// loaded — e.g. serving `Qwen3-0.6B-GGUF` yields `Qwen3-0.6B-Q4_0.gguf` — so an
/// exact/containment check on the catalog name fails even though it IS the right
Expand Down
Loading