Skip to content

[https://nvbugs/5938603][fix] Cherry-pick Fix E/PD disagg chunked prefill bug (#11805) - #11847

Merged
pcastonguay merged 1 commit into
NVIDIA:release/1.3.0rc5.post1from
pcastonguay:cherry_pick_epd_fix
Mar 3, 2026
Merged

[https://nvbugs/5938603][fix] Cherry-pick Fix E/PD disagg chunked prefill bug (#11805)#11847
pcastonguay merged 1 commit into
NVIDIA:release/1.3.0rc5.post1from
pcastonguay:cherry_pick_epd_fix

Conversation

@pcastonguay

@pcastonguay pcastonguay commented Mar 3, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Bug Fixes

    • Improved multimodal data validation in Qwen models by enforcing stricter presence checks for multimodal embeddings, preventing false positives when data is present but invalid.
  • Tests

    • Added smoke test for chunked prefill with multimodal Qwen models to verify engine stability and prevent crashes.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

…IDIA#11805)

Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
@pcastonguay
pcastonguay requested review from a team as code owners March 3, 2026 02:45
@pcastonguay

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@pcastonguay pcastonguay changed the title [https://nvbugs/5938603][fix] Fix E/PD disagg chunked prefill bug (#1… [https://nvbugs/5938603][fix] Cherry-pick Fix E/PD disagg chunked prefill bug (#11805) Mar 3, 2026
@coderabbitai

coderabbitai Bot commented Mar 3, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e922a86 and 2a340bf.

📒 Files selected for processing (3)
  • tensorrt_llm/_torch/models/modeling_qwen2vl.py
  • tensorrt_llm/_torch/models/modeling_qwen3vl.py
  • tests/unittest/_torch/multimodal/test_mm_encoder_standalone.py

📝 Walkthrough

Walkthrough

Changes tighten multimodal embedding condition checks in two Qwen VL model files from truthiness tests to explicit None checks, and add a new smoke test for chunked prefill functionality with the Qwen VL model.

Changes

Cohort / File(s) Summary
Multimodal Embedding Condition Fixes
tensorrt_llm/_torch/models/modeling_qwen2vl.py, tensorrt_llm/_torch/models/modeling_qwen3vl.py
Changed multimodal embedding condition from truthiness check to explicit None check, preventing false positives when multimodal_embedding is falsy but present.
Chunked Prefill Test
tests/unittest/_torch/multimodal/test_mm_encoder_standalone.py
Added new smoke test function test_chunked_prefill_multimodal_smoke that verifies chunked prefill functionality with Qwen VL models does not crash and produces non-empty outputs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description contains only the template structure with no substantive content filled in; the Description and Test Coverage sections are empty. Complete the Description section explaining the issue and solution, and the Test Coverage section listing relevant tests that safeguard these changes.
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the fix for a specific E/PD disagg chunked prefill bug (NVBugs 5938603) and references a related PR.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #37415 [ run ] triggered by Bot. Commit: 2a340bf Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #37415 [ run ] completed with state SUCCESS. Commit: 2a340bf
/LLM/release-1.3.0rc5.post1/L0_MergeRequest_PR pipeline #13 completed with status: 'FAILURE'

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@pcastonguay

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@pcastonguay
pcastonguay enabled auto-merge (squash) March 3, 2026 12:33
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #37507 [ run ] triggered by Bot. Commit: 2a340bf Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #37507 [ run ] completed with state SUCCESS. Commit: 2a340bf
/LLM/release-1.3.0rc5.post1/L0_MergeRequest_PR pipeline #16 completed with status: 'SUCCESS'

Link to invocation

@brb-nv brb-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@pcastonguay
pcastonguay merged commit 28fdceb into NVIDIA:release/1.3.0rc5.post1 Mar 3, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants