Skip to content

test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484) - #136

Merged
fredespi merged 1 commit into
mainfrom
test-e2e-mock-real-split
Jul 22, 2026
Merged

test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484)#136
fredespi merged 1 commit into
mainfrom
test-e2e-mock-real-split

Conversation

@fredespi

Copy link
Copy Markdown
Collaborator

Summary

Move the two mockable serve-chat E2E scenarios off the scarce serial GPU runners onto the GitHub-hosted no-GPU lane, so they gate every PR at no coverage cost. Part of the E2E speedup effort ([EAI-7484]).

The chat scenarios tool definitions accepted (#4) and end-to-end local model (#5) only assert that a served model answers over HTTP — a tools-bearing request is accepted, and a non-empty reply comes back. Neither checks real generation, so neither needs a GPU.

Changes

  • Capability-aware serve precondition: a model is served in the background now does a real rocm serve --managed on a host with an AMD GPU (real-serve coverage preserved where hardware exists) and uses the in-process MockServer + a planted managed-service record on a no-GPU host.
  • Drop @requires-gpu from chat scenarios 4 and 5 so they also run on the mock lane every PR. Real inference stays covered by the @requires-gpu serve-*-inference scenarios.
  • a managed runtime is active becomes a no-op on a no-GPU host (a managed SDK can only install where a GPU family selects wheels; the mock serve needs no runtime). Only affects the two migrated scenarios.
  • xfail scoping: the EAI-7423 lemonade-serve xfails for these two ids are scoped to therock_family = "gfx*", so they apply only on a real GPU lemonade host and the scenarios expect-pass on the mock lane instead of XPASS-ing.
  • serve-absent-gpu-index-rejected (Replace the openssl CLI with pure-Rust signing & verification #13) stays on GPU (comment updated): on a no-GPU host the GPU-required pre-flight refuses ("no usable AMD GPU") before the --gpu index is validated, so its index-specific rejection can only be observed with a real device present.

Test plan

Linux container gate green:

Real-GPU behaviour (real serve on GPU hosts) is unchanged and will be exercised by the GPU CI lanes.

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the mock/real lane split, expectation scoping, managed-service setup, and GPU-lane behavior. The mock lane now exercises the intended HTTP contract while preserving real inference coverage, and the focused checks pass.

@fredespi fredespi changed the title test(e2e): run mockable serve chat scenarios on the no-GPU lane test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484) Jul 21, 2026
The chat scenarios "tool definitions accepted" and "end-to-end local
model" only assert that a served model answers over HTTP (a tools-bearing
request is accepted; a non-empty reply comes back) — neither checks real
generation. They were @requires-gpu and so only ran on the scarce serial
GPU runners, yet nothing about them needs a real GPU.

Make the shared "a model is served in the background" precondition
capability-aware: a real `rocm serve --managed` on a host with an AMD GPU
(preserving real-serve coverage where hardware exists), and the in-process
MockServer + a planted managed-service record on a no-GPU host. Drop
@requires-gpu from both scenarios so they also run on the GitHub-hosted
mock lane every PR, with no coverage loss — real inference stays covered by
the @requires-gpu serve-*-inference scenarios.

"a managed runtime is active" becomes a no-op on a no-GPU host: a managed
TheRock SDK can only be installed where a GPU family selects wheels, and
the mock serve needs no runtime. This only affects the two migrated
scenarios; all other users stay @requires-gpu.

Scope the EAI-7423 lemonade-serve xfails for these two ids to
therock_family=gfx*, so they apply only on a real GPU lemonade host and
the scenarios expect-pass on the mock lane (where MockServer, not a real
lemonade serve, backs them) instead of XPASS-ing.

serve-absent-gpu-index-rejected was evaluated for the same move but stays
@requires-gpu: on a no-GPU host the GPU-required pre-flight refuses ("no
usable AMD GPU") before the --gpu index is validated, so its
index-specific rejection can only be observed with a real device present.

Part of EAI-7484 (mock/real split for GPU serve scenarios).

Signed-off-by: fredespi <fredrik.espinoza@gmail.com>
@fredespi
fredespi force-pushed the test-e2e-mock-real-split branch from 477510d to 0c2c5c0 Compare July 22, 2026 11:30
@fredespi
fredespi added this pull request to the merge queue Jul 22, 2026
Merged via the queue into main with commit cae7781 Jul 22, 2026
18 of 20 checks passed
@fredespi
fredespi deleted the test-e2e-mock-real-split branch July 22, 2026 16:13
@fredespi fredespi changed the title test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484) EAI-7484 test(e2e): run mockable serve chat scenarios on the no-GPU lane Jul 28, 2026
@fredespi fredespi changed the title EAI-7484 test(e2e): run mockable serve chat scenarios on the no-GPU lane test(e2e): run mockable serve chat scenarios on the no-GPU lane (EAI-7484) Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants