[None][ci] Update golden telemetry manifest#16208
Conversation
Signed-off-by: William Zhang <133824995+2ez4bz@users.noreply.github.com>
|
/bot run |
2ez4bz
added
the
api-compatible
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe golden manifest adds cache, dispatch, and multimodal parameters for TorchLlmArgs and TrtLlmArgs, moves video pruning under multimodal configuration, and permits ChangesLLM argument manifest schema
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
PR_Github #58520 [ run ] triggered by Bot. Commit: |
|
/bot skip |
GitHub Bot Help
Provide a user friendly way for developers to interact with a Jenkins server. Run See details below for each supported subcommand. Details
Launch build/test pipelines. All previously running jobs will be killed.
kill
Kill all running builds associated with pull request. skip
Skip testing for latest commit on pull request. reuse-pipeline
Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break. |
|
/bot skip --comment "Regenerate manifest" |
|
/bot skip --comment "unblocks CI breakage, no incremental risk" |
|
PR_Github #58527 [ skip ] triggered by Bot. Commit: |
|
PR_Github #58520 [ run ] completed with state |
|
PR_Github #58527 [ skip ] completed with state |
|
PR_Github #58526 [ ] completed with state |
…ing, reload reachability, loader concurrency, scratch/workspace lifecycle) - swiglu_limit wiring: MegaMoECuteDsl joins the per-expert tensor path in modeling_deepseekv4 (DSv4 NVFP4 checkpoints all set swiglu_limit=10.0 and creation asserted on the unconditional scalar); the scalar is now gated by the positive list of backends that consume it (incl. CuteDslB12xFusedMoE), configs whose backend consumes neither form get a loud warning instead of a crash. NOTE: prior feat-branch validation silently dropped the clamp for this backend -- it now reaches the kernel codegen for the first time (GSM8K re-validation scheduled). - allow_partial_loading threaded through DeepseekV4ForCausalLM -> DeepseekV4WeightLoader -> every branch (the model_loader signature introspection previously hard-rejected DSv4 reloads, making the streamed partial reload unreachable). Suffix-bucketed streaming is supported for regular params, one-sided gate/up (the fused Linear helpers shard-copy), and MoE experts (cross-bucket quant-method stashes); fused-A groups (incl. NVFP4 scales), quantized kv_b_proj, FP8 o_a_proj and raw-V4 compressor pairs fail loud with a ship-whole-group message; non-partial behavior unchanged. - Loader concurrency: a class-level lock serializes the streamed transient window (materialize -> base load -> eager finalize) across modules -- the generic ThreadPoolExecutor loader would otherwise stack pool-width layers' CUDA transients. - Profiling scratch: allocation is deferred behind a factory that only materializes on a REAL profiling launch (the pre-hook), so a spec-decode draft whose tuning key cache-hits the target's entries never allocates the multi-GiB scratch; the no-scratch fail-loud moved to the same point; the setter pair is cleared in a finally. - reset_megamoe_workspace_state(): bench-only helper dropping ALL module-global MegaMoE state (workspaces, providers, fences, captured ptrs, capture flag) between sweep points -- multi-point benchmarks otherwise accumulate per-point default/winner workspaces. - llm_args golden manifest + telemetry doc regenerated: adds the MEGAMOE_CUTEDSL enum and (base-staleness) the three fields main later added via NVIDIA#16208 -- merges trivially with latest main. Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
…ing, reload reachability, loader concurrency, scratch/workspace lifecycle) - swiglu_limit wiring: MegaMoECuteDsl joins the per-expert tensor path in modeling_deepseekv4 (DSv4 NVFP4 checkpoints all set swiglu_limit=10.0 and creation asserted on the unconditional scalar); the scalar is now gated by the positive list of backends that consume it (incl. CuteDslB12xFusedMoE), configs whose backend consumes neither form get a loud warning instead of a crash. NOTE: prior feat-branch validation silently dropped the clamp for this backend -- it now reaches the kernel codegen for the first time (GSM8K re-validation scheduled). - allow_partial_loading threaded through DeepseekV4ForCausalLM -> DeepseekV4WeightLoader -> every branch (the model_loader signature introspection previously hard-rejected DSv4 reloads, making the streamed partial reload unreachable). Suffix-bucketed streaming is supported for regular params, one-sided gate/up (the fused Linear helpers shard-copy), and MoE experts (cross-bucket quant-method stashes); fused-A groups (incl. NVFP4 scales), quantized kv_b_proj, FP8 o_a_proj and raw-V4 compressor pairs fail loud with a ship-whole-group message; non-partial behavior unchanged. - Loader concurrency: a class-level lock serializes the streamed transient window (materialize -> base load -> eager finalize) across modules -- the generic ThreadPoolExecutor loader would otherwise stack pool-width layers' CUDA transients. - Profiling scratch: allocation is deferred behind a factory that only materializes on a REAL profiling launch (the pre-hook), so a spec-decode draft whose tuning key cache-hits the target's entries never allocates the multi-GiB scratch; the no-scratch fail-loud moved to the same point; the setter pair is cleared in a finally. - reset_megamoe_workspace_state(): bench-only helper dropping ALL module-global MegaMoE state (workspaces, providers, fences, captured ptrs, capture flag) between sweep points -- multi-point benchmarks otherwise accumulate per-point default/winner workspaces. - llm_args golden manifest + telemetry doc regenerated: adds the MEGAMOE_CUTEDSL enum and (base-staleness) the three fields main later added via NVIDIA#16208 -- merges trivially with latest main. Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
3 participants
Summary by CodeRabbit
New Features
W4A16_NVFP4quantization algorithm.Updates
Description
#16013 Introduced a new reference file that unfortunately was merged
without changes in
mainthat would have required it to be updated(test-time conflict).
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.