[https://nvbugs/6390307][fix] Update trtllm-gen FMHA JIT libraries by bobboli · Pull Request #15974 · NVIDIA/TensorRT-LLM

bobboli · 2026-07-06T09:59:05Z

Description

Refresh the x86_64 and aarch64 FMHA JIT static libraries from the matching FMHA source snapshot.

Force numInstsKv=2 for skip-softmax generation kernels.
Preserve sm_103a in multi-instance NVRTC linking.
Synchronize FmhaAutoTuner.h, FmhaOptions.h, and KernelTraits.h with the libraries.
Keep all other exported headers and existing cubins unchanged.
Remove the related NVBug 6388157 and 6390307 test waivers.

Addresses https://nvbugs/6390307 and https://nvbugs/6428176

Test Coverage

CUDA 13.0 x86_64 and aarch64 FMHA public-library builds passed.
tekit:latest Release build_wheel_targets passed.
B300 4-GPU skip-softmax integration tests passed for BF16 and FP8 KV configurations.
8/8 native B300 Blackwell Ultra FMHA tests passed.

PR Checklist

Please check this after reviewing the checklist in the PR template as appropriate for this PR.

bobboli · 2026-07-07T08:24:51Z

/bot run --post-merge --disable-fail-fast

coderabbitai · 2026-07-07T08:30:15Z

📝 Walkthrough

Walkthrough

This PR modifies a diagnostic check in KernelTraits.h to suppress an informational hint for generation kernels that skip softmax, updates the corresponding prebuilt trtllmGen library binaries (aarch64 and x86_64) via Git LFS pointers, and removes now-obsolete test waivers for the related skip-softmax attention test.

Changes

Skip-softmax attention fix

Layer / File(s)	Summary
Suppress diagnostic hint for skip-softmax generation kernels `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/KernelTraits.h`	Adds `isGenerationSkipsSoftmax` check to conditionally bypass the `TLLM_CHECK_INFO` hint when generation kernels skip softmax when possible.
Updated prebuilt kernel libraries `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/{aarch64,x86_64}-linux-gnu/libTrtLlmGen*.a`	Git LFS pointers (oid/size) updated for four static library artifacts to reflect rebuilt binaries.
Removed obsolete test waivers `tests/integration/test_lists/waives.txt`	Removed four SKIP waiver entries for `test_skip_softmax_attention_4gpus` across sparsity/fp8kv configurations, including DGX_B200-specific entries.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#15930: Both PRs modify tests/integration/test_lists/waives.txt to add/remove integration test skip entries.
NVIDIA/TensorRT-LLM#15959: Both PRs remove SKIP waiver entries from waives.txt for related skip-softmax tests.
NVIDIA/TensorRT-LLM#16034: Both PRs modify waives.txt to change CI test SKIP/waiver entries.

Suggested reviewers: QiJune, fredricz-20070104, Tabrizian

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise, specific, and matches the main change: refreshing trtllm-gen FMHA JIT libraries.
Description check	✅ Passed	The description covers the change, the rationale, and relevant test coverage, with the required sections present.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

🧹 Nitpick comments (1)

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/KernelTraits.h (1)
182-184: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Declare the local as const.

isGenerationSkipsSoftmax is never reassigned after initialization.
♻️ Proposed fix
-      bool isGenerationSkipsSoftmax =
+      bool const isGenerationSkipsSoftmax =
           options.mSkipsSoftmaxWhenPossible && !isContextKernel(options.mFmhaKernelType);
As per coding guidelines, "A variable that is not modified after its initialization should be declared as const" and should use east-const style (int const x).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/KernelTraits.h`
around lines 182 - 184, The local flag in KernelTraits::isGenerationSkipsSoftmax
is only initialized and never reassigned, so declare it as const using
east-const style. Update the declaration in the generation-kernel softmax check
to use a const local while keeping the existing condition logic unchanged.
Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/KernelTraits.h`:
- Around line 182-184: The local flag in KernelTraits::isGenerationSkipsSoftmax
is only initialized and never reassigned, so declare it as const using
east-const style. Update the declaration in the generation-kernel softmax check
to use a const local while keeping the existing condition logic unchanged.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 38e1df50-b447-424f-af3e-089cc7c7c45e

📥 Commits

Reviewing files that changed from the base of the PR and between 0b0fd54 and 83709cc.

📒 Files selected for processing (6)

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/aarch64-linux-gnu/libTrtLlmGen.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/aarch64-linux-gnu/libTrtLlmGenFmhaLib.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/x86_64-linux-gnu/libTrtLlmGen.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/x86_64-linux-gnu/libTrtLlmGenFmhaLib.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/trtllmGen_fmha_export/KernelTraits.h
tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

tensorrt-cicd · 2026-07-07T08:30:35Z

PR_Github #57976 [ run ] triggered by Bot. Commit: 83709cc Link to invocation

tensorrt-cicd · 2026-07-08T06:40:20Z

PR_Github #57976 [ run ] completed with state FAILURE. Commit: 83709cc
/LLM/main/L0_MergeRequest_PR pipeline #46650 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

bobboli · 2026-07-08T14:46:28Z

/bot run --post-merge --disable-fail-fast

tensorrt-cicd · 2026-07-08T14:53:10Z

PR_Github #58250 [ run ] triggered by Bot. Commit: 944a450 Link to invocation

bobboli · 2026-07-08T14:59:13Z

/bot kill

tensorrt-cicd · 2026-07-08T15:06:00Z

PR_Github #58251 [ kill ] triggered by Bot. Commit: 944a450 Link to invocation

tensorrt-cicd · 2026-07-08T15:09:30Z

PR_Github #58250 [ run ] completed with state ABORTED. Commit: 944a450

Link to invocation

tensorrt-cicd · 2026-07-08T15:09:35Z

PR_Github #58251 [ kill ] completed with state SUCCESS. Commit: 944a450
Successfully killed previous jobs for commit 944a450

Link to invocation

Refresh x86_64 and aarch64 FMHA JIT libraries for skip-softmax generation and sm_103a NVRTC linking. Synchronize FmhaAutoTuner, FmhaOptions, and KernelTraits with the matching FMHA snapshot. Retain all other exported headers and cubins. Remove the related test waivers. Refs NVBug 6388157 Refs NVBug 6390307 Refs TRTLLM-12988 Refs TRTLLM-12990 Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>

bobboli · 2026-07-08T15:16:17Z

/bot run --post-merge --disable-fail-fast

tensorrt-cicd · 2026-07-08T15:22:53Z

PR_Github #58257 [ run ] triggered by Bot. Commit: 501d7cd Link to invocation

tensorrt-cicd · 2026-07-09T05:45:33Z

PR_Github #58257 [ run ] completed with state FAILURE. Commit: 501d7cd
/LLM/main/L0_MergeRequest_PR pipeline #46897 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

bobboli · 2026-07-09T06:01:57Z

/bot run --reuse-test

tensorrt-cicd · 2026-07-09T06:10:26Z

PR_Github #58388 [ run ] triggered by Bot. Commit: 501d7cd Link to invocation

tensorrt-cicd · 2026-07-09T06:55:53Z

PR_Github #58388 [ run ] completed with state SUCCESS. Commit: 501d7cd
/LLM/main/L0_MergeRequest_PR pipeline #47014 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

bobboli · 2026-07-09T11:04:38Z

/bot run --reuse-test

tensorrt-cicd · 2026-07-09T11:10:18Z

PR_Github #58455 [ run ] triggered by Bot. Commit: 501d7cd Link to invocation

tensorrt-cicd · 2026-07-09T14:06:50Z

PR_Github #58455 [ run ] completed with state SUCCESS. Commit: 501d7cd
/LLM/main/L0_MergeRequest_PR pipeline #47067 completed with status: 'SUCCESS'

CI Report

Link to invocation

github-actions · 2026-07-09T14:10:05Z

✅ LFS objects already in storage (4 files) — no sync needed.

These LFS-tracked files are already present in this repository's LFS storage:

cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/aarch64-linux-gnu/libTrtLlmGen.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/aarch64-linux-gnu/libTrtLlmGenFmhaLib.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/x86_64-linux-gnu/libTrtLlmGen.a
cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/lib/x86_64-linux-gnu/libTrtLlmGenFmhaLib.a

github-actions Bot assigned bobboli Jul 6, 2026

bobboli requested review from PerkzZheng and Tom-Zheng July 6, 2026 10:12

bobboli changed the title ~~[TRTLLM-12988][fix] Update trtllm-gen FMHA JIT libraries~~ [https://nvbugs/6390307][fix] Update trtllm-gen FMHA JIT libraries Jul 6, 2026

bobboli force-pushed the fix/update-trtllm-gen-fmha-libs branch 3 times, most recently from 1dd1179 to 83709cc Compare July 7, 2026 08:20

bobboli marked this pull request as ready for review July 7, 2026 08:24

PerkzZheng approved these changes Jul 7, 2026

View reviewed changes

bobboli enabled auto-merge (squash) July 7, 2026 08:26

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

bobboli force-pushed the fix/update-trtllm-gen-fmha-libs branch 2 times, most recently from 7cec8b7 to 944a450 Compare July 8, 2026 14:40

bobboli force-pushed the fix/update-trtllm-gen-fmha-libs branch from 944a450 to 501d7cd Compare July 8, 2026 15:11

bobboli merged commit 6e3c3bf into NVIDIA:main Jul 9, 2026
13 checks passed

Uh oh!

Conversation

bobboli commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

Uh oh!

bobboli commented Jul 7, 2026

Uh oh!

coderabbitai Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

bobboli commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

bobboli commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

bobboli commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

bobboli commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

bobboli commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bobboli commented Jul 6, 2026 •

edited

Loading

coderabbitai Bot commented Jul 7, 2026 •

edited

Loading