[None][fix] Resolve NVFP4 mixed-precision base layers for the DSpark draft

[tianyuz-nv]

Dev Engineer Review

• Updated DSparkDraftModel to normalize global MIXED_PRECISION quantization using the target DeepSeek-V4 model’s normalizer.
• Applies the resolved quant_config only when normalization changes the configuration, preserving existing behavior for other checkpoints.
• Draft configuration updates remain copy-on-write and consistently handle sparse attention, quantization dictionaries, and quantization configuration.
• No test, configuration, or test-list files were changed.

QA Engineer Review

No test changes.

Description

NVFP4 DeepSeek-V4 checkpoints declare a global MIXED_PRECISION quant algo
(per-layer NVFP4 routed experts over an FP8 base). The target model resolves
this base in _normalize_deepseek_v4_nvfp4_mixed_precision_config
(MIXED_PRECISION → FP8_BLOCK_SCALES; the routed experts stay NVFP4 through
quant_config_dict).

The DSpark draft model builds its ModelConfig separately
(get_draft_model → DSparkDraftModel._derive_draft_model_config) and never
went through that normalization, so the inherited DeepseekV4DecoderLayer hits
the MIXED_PRECISION is ambiguous assertion while building the draft stages and
an NVFP4 DSpark checkpoint fails to load.

This PR extends the same normalization to the draft. DSparkDraftModel
now derives the resolved global quant_config via a new
_draft_normalized_quant_config helper (reusing the target normalizer on a
throwaway shallow copy) and sets it on the draft ModelConfig. The path is a
strict no-op for checkpoints whose global algo is not MIXED_PRECISION
(e.g. MXFP4) and for checkpoints the target normalizer leaves unchanged, so
those draft configs stay byte-identical.

Test Coverage

Validated end-to-end on nvidia/DeepSeek-V4-Pro-nvfp4-DSpark (TP4/EP4): with
this change the draft model loads and DSpark speculative decoding runs a full
draft-length 1–7 acceptance sweep (GSM8K / HumanEval / MATH500, chat and
thinking) with no acceptance-length regression versus the MXFP4 checkpoint.
Without this change the same run aborts while building the draft stages with
MIXED_PRECISION is ambiguous.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[longlee0622]

/bot run

[coderabbitai]

Walkthrough

DSpark draft configuration derivation now supports NVFP4 checkpoints using global MIXED_PRECISION quantization by normalizing and applying the resolved quant_config alongside existing draft configuration updates.

Changes

DSpark NVFP4 configuration

Layer / File(s)	Summary
Normalize draft quantization configuration tensorrt_llm/_torch/models/modeling_dspark.py	Adds MIXED_PRECISION detection and NVFP4 quant-config normalization, then applies changed sparse attention, quantization dictionary, and global quantization settings to copied draft configurations.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: longlee0622

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise, follows the required template, and accurately summarizes the DSpark NVFP4 mixed-precision fix.
Description check	✅ Passed	The description includes the required problem, solution, test coverage, and checklist sections and is sufficiently specific.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/models/modeling_dspark.py`:
- Around line 516-539: Update _draft_normalized_quant_config with an explicit
type annotation for model_config and a return annotation covering both the
resolved quant configuration and None. Use the existing model configuration and
quant-config types from the module or project conventions, without changing the
helper’s behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e87f6c35-bd5c-4ef7-8477-fa6aedc0faa0

📥 Commits

Reviewing files that changed from the base of the PR and between 56dedb1 and 835e279.

📒 Files selected for processing (1)

• tensorrt_llm/_torch/models/modeling_dspark.py

[tensorrt-cicd]

PR_Github #61531 [ run ] triggered by Bot. Commit: 835e279 Link to invocation

[tensorrt-cicd]

PR_Github #61531 [ run ] completed with state SUCCESS. Commit: 835e279
/LLM/main/L0_MergeRequest_PR pipeline #49745 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tianyuz-nv]

/bot run

[tensorrt-cicd]

PR_Github #61567 [ run ] triggered by Bot. Commit: 835e279 Link to invocation

[tensorrt-cicd]

PR_Github #61567 [ run ] completed with state SUCCESS. Commit: 835e279
/LLM/main/L0_MergeRequest_PR pipeline #49778 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[longlee0622]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61580 [ run ] triggered by Bot. Commit: 835e279 Link to invocation

[tensorrt-cicd]

PR_Github #61580 [ run ] completed with state FAILURE. Commit: 835e279
/LLM/main/L0_MergeRequest_PR pipeline #49791 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tianyuz-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #61796 [ run ] triggered by Bot. Commit: 835e279 Link to invocation

[tensorrt-cicd]

PR_Github #61796 [ run ] completed with state SUCCESS. Commit: 835e279
/LLM/main/L0_MergeRequest_PR pipeline #49995 completed with status: 'SUCCESS'

CI Report

Link to invocation