[https://nvbugs/6329052][fix] Change the config for DSV3 disagg conditional test - #16528
Conversation
📝 WalkthroughWalkthroughThe DeepSeek-V3-Lite disaggregated test configuration now uses FLASHINFER with four hidden layers. Its A100 and L40S test waive entries are removed. ChangesDeepSeek disaggregation test enablement
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot run --disable-fail-fast --stage-list "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
|
PR_Github #59913 [ run ] triggered by Bot. Commit: |
|
PR_Github #59913 [ run ] completed with state
|
…V3-Lite disagg conditional test test_workers_conditional_disaggregation_deepseek_v3_lite_bf16 failed on pre-Hopper GPUs: - L40S (nvbugs/6329052): the full ~50GB bf16 weights do not fit in 44GB, so the model OOMs during executor creation. - A100 / SM80 (nvbugs/6274934): the TRTLLM MLA FMHA kernel is unavailable, AttentionOp::initialize aborts and the disagg server never becomes ready (300s timeout). Set attn_backend=FLASHINFER (MLA runs on SM80/SM89) and model_kwargs.num_hidden_layers=4 (shrinks the model so it fits) in the disagg conditional cache-reuse config, and drop the now-obsolete A100/L40S waives. The QA functional test entry is kept so cross-GPU coverage is retained. Signed-off-by: Xiaowei Shi <39303645+Shixiaowei02@users.noreply.github.com>
28b86d0 to
31c45b9
Compare
|
/bot run --disable-fail-fast --stage-list "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
|
PR_Github #59916 [ run ] triggered by Bot. Commit: |
|
PR_Github #59916 [ run ] completed with state |
|
/bot skip --comment "Related tests have passed." |
|
PR_Github #59951 [ skip ] triggered by Bot. Commit: |
|
PR_Github #59951 [ skip ] completed with state |
Description
This pull request updates the configuration for DeepSeek-V3-Lite and adjusts test waivers for related disaggregated tests. The main changes are the addition of new model configuration options and the removal of test waivers for specific DeepSeek-V3-Lite disaggregated tests, likely reflecting improved support or bug fixes.
Configuration updates:
disagg_config_cache_reuse_deepseek_v3.yamlto setattn_backendtoFLASHINFERand addedmodel_kwargswithnum_hidden_layers: 4for the DeepSeek-V3-Lite model.Test waiver updates:
test_workers_conditional_disaggregation_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16]from bothA100andL40Stest lists, indicating this test is no longer being skipped due to bug 6329052. [1] [2]Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.Summary by CodeRabbit