test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484) - #136
Merged
Conversation
rominf
approved these changes
Jul 21, 2026
rominf
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed the mock/real lane split, expectation scoping, managed-service setup, and GPU-lane behavior. The mock lane now exercises the intended HTTP contract while preserving real inference coverage, and the focused checks pass.
The chat scenarios "tool definitions accepted" and "end-to-end local
model" only assert that a served model answers over HTTP (a tools-bearing
request is accepted; a non-empty reply comes back) — neither checks real
generation. They were @requires-gpu and so only ran on the scarce serial
GPU runners, yet nothing about them needs a real GPU.
Make the shared "a model is served in the background" precondition
capability-aware: a real `rocm serve --managed` on a host with an AMD GPU
(preserving real-serve coverage where hardware exists), and the in-process
MockServer + a planted managed-service record on a no-GPU host. Drop
@requires-gpu from both scenarios so they also run on the GitHub-hosted
mock lane every PR, with no coverage loss — real inference stays covered by
the @requires-gpu serve-*-inference scenarios.
"a managed runtime is active" becomes a no-op on a no-GPU host: a managed
TheRock SDK can only be installed where a GPU family selects wheels, and
the mock serve needs no runtime. This only affects the two migrated
scenarios; all other users stay @requires-gpu.
Scope the EAI-7423 lemonade-serve xfails for these two ids to
therock_family=gfx*, so they apply only on a real GPU lemonade host and
the scenarios expect-pass on the mock lane (where MockServer, not a real
lemonade serve, backs them) instead of XPASS-ing.
serve-absent-gpu-index-rejected was evaluated for the same move but stays
@requires-gpu: on a no-GPU host the GPU-required pre-flight refuses ("no
usable AMD GPU") before the --gpu index is validated, so its
index-specific rejection can only be observed with a real device present.
Part of EAI-7484 (mock/real split for GPU serve scenarios).
Signed-off-by: fredespi <fredrik.espinoza@gmail.com>
fredespi
force-pushed
the
test-e2e-mock-real-split
branch
from
July 22, 2026 11:30
477510d to
0c2c5c0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Move the two mockable serve-chat E2E scenarios off the scarce serial GPU runners onto the GitHub-hosted no-GPU lane, so they gate every PR at no coverage cost. Part of the E2E speedup effort ([EAI-7484]).
The chat scenarios tool definitions accepted (#4) and end-to-end local model (#5) only assert that a served model answers over HTTP — a tools-bearing request is accepted, and a non-empty reply comes back. Neither checks real generation, so neither needs a GPU.
Changes
a model is served in the backgroundnow does a realrocm serve --managedon a host with an AMD GPU (real-serve coverage preserved where hardware exists) and uses the in-processMockServer+ a planted managed-service record on a no-GPU host.@requires-gpufrom chat scenarios 4 and 5 so they also run on the mock lane every PR. Real inference stays covered by the@requires-gpuserve-*-inferencescenarios.a managed runtime is activebecomes a no-op on a no-GPU host (a managed SDK can only install where a GPU family selects wheels; the mock serve needs no runtime). Only affects the two migrated scenarios.therock_family = "gfx*", so they apply only on a real GPU lemonade host and the scenarios expect-pass on the mock lane instead of XPASS-ing.serve-absent-gpu-index-rejected(Replace the openssl CLI with pure-Rust signing & verification #13) stays on GPU (comment updated): on a no-GPU host the GPU-required pre-flight refuses ("no usable AMD GPU") before the--gpuindex is validated, so its index-specific rejection can only be observed with a real device present.Test plan
Linux container gate green:
cargo clippy --workspace --all-targets -- -D warningse2e-cucumberlib tests (25 passed)cargo xtask e2emock lane: 4 xfail / 0 XPASS / 0 unexpected; chat Fix CI: prek hooks, headless/flaky test fixes, and drop vendored Codex #4 and Speed up CI #5 now pass on the no-GPU lane.Real-GPU behaviour (real serve on GPU hosts) is unchanged and will be exercised by the GPU CI lanes.