Skip to content

[https://nvbugs/6473397][test] Raise BF16 Laguna DFlash memory skip threshold to prevent OOM - #16627

Merged
DomBrown merged 1 commit into
NVIDIA:mainfrom
DomBrown:bug/nvbugs-6473397-laguna-bf16-oom-skip
Jul 20, 2026
Merged

[https://nvbugs/6473397][test] Raise BF16 Laguna DFlash memory skip threshold to prevent OOM#16627
DomBrown merged 1 commit into
NVIDIA:mainfrom
DomBrown:bug/nvbugs-6473397-laguna-bf16-oom-skip

Conversation

@DomBrown

@DomBrown DomBrown commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Bug Fixes
    • Improved test compatibility by requiring additional device memory for the BF16+DFLASH accuracy test.
    • Removed outdated platform-specific test waivers so supported environments can run the accuracy test normally.

Description

  • TestLagunaXS_2_1::test_bf16_dflash OOMs on 80 GB devices: BF16 Laguna-XS-2.1 weights are ~64 GiB, leaving too little headroom for KV cache + CUDA graph capture
    at max_batch_size=128, max_seq_len=4096.
  • Raise skip_less_device_memory from 80000 → 90000 on BF16 only. FP8 and NVFP4 variants fit on 80 GB and are unchanged.
  • Drop the now-redundant A100/H100 waives (nvbugs/6473397) — the marker handles skipping structurally.

Test plan

  • pytest test_bf16_dflash skips cleanly on H100 80 GB
  • Verified passing on RTX PRO 6000 Blackwell (~96 GiB)

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

…hreshold

TestLagunaXS_2_1::test_bf16_dflash OOMs on 80 GB devices (A100 80GB,
H100 80GB): the BF16 Laguna-XS-2.1 checkpoint occupies ~64 GiB of
weights, leaving only ~14 GiB for KV cache, workspace, and CUDA graph
capture at max_batch_size=128 / max_seq_len=4096, which is insufficient.
Reducing batch size would slow evaluation and change what the test
exercises without a guarantee of fitting; the model itself is the
dominant memory cost.

Raise skip_less_device_memory from 80000 to 90000 on the BF16 variant
so pytest skips cleanly on 80 GB devices while still running on
>=94 GB SKUs (H100 NVL, H20, H200, RTX PRO 6000 Blackwell, B200).
Verified passing on RTX PRO 6000 Blackwell (~96 GiB). FP8 and NVFP4
variants are unaffected; they fit on H100 80 GB and remain at 80000.

Also drop the now-redundant A100/H100 waives for this test, since the
marker handles skipping structurally.

Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>
@DomBrown

Copy link
Copy Markdown
Collaborator Author

/bot run

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2991cbfc-78d7-4537-87ae-5a32ad8a4edb

📥 Commits

Reviewing files that changed from the base of the PR and between 5d8a786 and d982fb0.

📒 Files selected for processing (2)
  • tests/integration/defs/accuracy/test_llm_api_pytorch.py
  • tests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

The BF16 DFLASH accuracy test now skips on devices with less than 90000 device memory units. Its A100 and H100 waiver entries are removed.

Changes

BF16 DFLASH test eligibility

Layer / File(s) Summary
Update test threshold and waivers
tests/integration/defs/accuracy/test_llm_api_pytorch.py, tests/integration/test_lists/waives.txt
Raises the BF16 DFLASH memory threshold from 80000 to 90000 and removes the A100 and H100 waiver entries.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

Suggested reviewers: bowenfu

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is specific, matches the change, and follows the ticket/type format.
Description check ✅ Passed The PR includes the required summary, explanation, test plan, and checklist, with only a minor heading mismatch.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60371 [ run ] triggered by Bot. Commit: d982fb0 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #60371 [ run ] completed with state SUCCESS. Commit: d982fb0
/LLM/main/L0_MergeRequest_PR pipeline #48710 completed with status: 'SUCCESS'

CI Report

Link to invocation

@DomBrown
DomBrown merged commit 6a5caee into NVIDIA:main Jul 20, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants