[https://nvbugs/6478692][fix] Pass max_workers=16 to _build in EPDVariant.nano_omni_fp8 (mirroring… - #16672
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughThe nano omni FP8 multimodal EPD test now explicitly limits its worker pool to 16, and the corresponding VideoMME test waiver is removed. ChangesMultimodal EPD test enablement
Estimated code review effort: 2 (Simple) | ~5 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot run --only-qa-verify test accuracy/test_epd_disagg_multimodal.py::TestVideoMMEEPD::test_disaggregated_videomme[nemotron_nano_v3_omni_fp8] |
|
PR_Github #60696 [ run ] triggered by Bot. Commit: |
|
PR_Github #60696 [ run ] completed with state |
mzweilz
left a comment
There was a problem hiding this comment.
LGTM if the tests pass.
|
LGTM if the test can pass. |
The nano_omni_fp8 EPDVariant did not override max_workers and defaulted to VideoMME.MAX_BATCH_SIZE=128, which oversubscribes the CPU-side multimodal preprocessing / hashing path on H100 and trips a native-library abort (SIGABRT) with many threads inside _media_to_raw_chw and apply_mm_hashes. The passing qwen3vl_2b variant already caps max_workers at 16 for the same reason. Cap the submission concurrency at 16 to match the working sibling variant, and remove the waiver so the test runs in CI again. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
537c3eb to
57ade4d
Compare
|
/bot run |
|
PR_Github #63095 [ run ] triggered by Bot. Commit: |
|
PR_Github #63095 [ run ] completed with state |
…PDVariant.nano_omni_fp8` (mirroring… (#16672) Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>
Summary
nano_omni_fp8did not overridemax_workers, defaulting toVideoMME.MAX_BATCH_SIZE=128, which oversubscribed the CPU-side multimodal preprocessing/hashing path on H100 and tripped a native-library SIGABRT.max_workers=16to_buildinEPDVariant.nano_omni_fp8(mirroringqwen3vl_2b), and remove the corresponding entry fromwaives.txt.Test plan
Links
Summary by CodeRabbit