[https://nvbugs/6316983][fix] In _is_supported_with_reason, return False when `fwd.mrope_rotary_cos_sin is… - #15651
Closed
tensorrt-cicd wants to merge 1 commit into
Closed
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughThe FMHA backend now rejects requests that supply ChangesmRoPE rejection and waiver cleanup
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
yuxianq
approved these changes
Jun 26, 2026
Collaborator
|
/bot run |
Collaborator
Author
|
PR_Github #56013 [ run ] triggered by Bot. Commit: |
Collaborator
Author
|
PR_Github #56013 [ run ] completed with state
|
tensorrt-cicd
force-pushed
the
repair-bot-bug6316983
branch
from
June 29, 2026 19:08
ae14fcd to
fd897f0
Compare
trtllm-agent
force-pushed
the
repair-bot-bug6316983
branch
from
July 6, 2026 02:08
fd897f0 to
00c1433
Compare
… use PR NVIDIA#14618 made the trtllm-gen FMHA the default Blackwell attention backend; while it ships mRoPE plumbing (mrope_rotary_cos_sin is passed to trtllm_gen_context_{preprocess,postprocess}), its mRoPE numerics regress Qwen2.5-VL MMMU by ~6 points on every Blackwell SKU (B200/GB200/GB300), dropping accuracy from ~51 to ~44.9 and tripping the 47.343 threshold in test_llm_api_pytorch_multimodal:: TestQwen2_5_VL_7B::test_auto_dtype. Gate trtllm-gen off in _is_supported_with_reason whenever fwd.mrope_rotary_cos_sin is not None; the dispatcher then falls through to FallbackFmha (legacy attention path), which already plumbs mrope_rotary_cos_sin through fmha/fallback.py and handles it correctly. This is a per-request gate, so pure-text requests on the same model still take the fast trtllm-gen path. Also drop the three existing waivers for this bug (B200/GB200/GB300) now that the test passes (verified MMMU=47.78 on GB300, threshold 47.343). Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>
trtllm-agent
force-pushed
the
repair-bot-bug6316983
branch
from
July 7, 2026 00:42
00c1433 to
cf802f4
Compare
yuxianq
self-requested a review
July 7, 2026 07:08
Collaborator
|
#15661 fixes this issue, close this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_is_supported_with_reason, return False whenfwd.mrope_rotary_cos_sin is not None, causing the dispatcher to fall through to FallbackFmha; remove the three stale nvbugs/6316983 waivers (B200/GB200/GB300). No new waivers added.Test plan
Links
Summary by CodeRabbit