[https://nvbugs/6442074][fix] Rename MTPEagleDynamicTreeWorker.forward to _forward_impl#16724
Conversation
…d to _forward_impl Signed-off-by: qgai <qgai@nvidia.com>
|
/bot run --disable-fail-fast |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Walkthrough
ChangesMTP dynamic tree worker
Estimated code review effort: 1 (Trivial) | ~2 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
PR_Github #60962 [ run ] triggered by Bot. Commit: |
forward → _forward_impl (signature unchanged); the base…
#16728
|
PR_Github #60962 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #60978 [ run ] triggered by Bot. Commit: |
|
PR_Github #60978 [ run ] completed with state |
…d to _forward_impl (NVIDIA#16724) Signed-off-by: qgai <qgai@nvidia.com>
Summary
TypeErroron current main:MTPEagleDynamicTreeWorker(merged in [None][feat] Support Nemotron dynamic-tree MTP decoding #15582) overridesSpecWorkerBase.forward, which is rejected by the__init_subclass__guard added in [https://nvbugs/6442074][fix] Make one-model spec-dec attn-metadata save/restore exception-safe #16382 (https://nvbugs/6442074). Sincespeculative/utils.pyimportsmtp_dynamic_treeat module level, any import of the PyTorch backend fails.Changes
tensorrt_llm/_torch/speculative/mtp_dynamic_tree.py: rename theforwardoverride to_forward_implper the guard's contract (1 line).SpecWorkerBase.forwardnow wraps it with the guaranteed spec-dec attn-metadata cleanup, same asEagle3OneModelDynamicTreeWorker.Test plan
python -c "import tensorrt_llm._torch.speculative.utils"raises the guard TypeErrorDev Engineer Review
MTPEagleDynamicTreeWorker.forwardto_forward_impl, satisfying theSpecWorkerBase.__init_subclass__guard.mtp_dyn.forward).SpecWorkerBase.forwardremains the public entry point, preserving speculative-decoding attention-metadata cleanup.QA Engineer Review
No test changes.