Skip to content

Revert unverified EAGLE3 model examples; keep triage code + baseline - #1623

Merged
yeyu-nvidia merged 1 commit into
mainfrom
yeyu/revert-eagle3-unverified-examples
Jun 3, 2026
Merged

Revert unverified EAGLE3 model examples; keep triage code + baseline#1623
yeyu-nvidia merged 1 commit into
mainfrom
yeyu/revert-eagle3-unverified-examples

Conversation

@yeyu-nvidia

@yeyu-nvidia yeyu-nvidia commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: Revert / cleanup (follow-up to #1417)

Per review feedback (@h-guo18): main should be production-ready and user-facing. Most of the EAGLE3 model example YAMLs added in #1417 are not yet verified to work end-to-end in modelopt (~80% fail at some pipeline stage), which is confusing to ship. The agreed plan is to land the triage infrastructure now and re-add each model's launcher YAML in a dedicated follow-up PR once it is verified green.

Removed (unverified, to be re-added per-model once verified):

  • Per-model launcher configs (hf_offline_eagle3.yaml + eagle3_quick_check.yaml) for: DeepSeek-V3.2, GLM-5, MiniMax-M2.5, Ministral-3-8B, Ministral-3-14B, Kimi-K2.5, Kimi-K2.5-NVFP4, GPT-OSS-20B, Qwen3.5-9B, Qwen3.5-27B, Qwen3.5-35B-A3B, Step-3.5-Flash.
  • Per-model status docs: tools/launcher/examples/EAGLE3_TRIAGE.md, examples/speculative_decoding/pipeline/eagle3/eagle3_triage_chart.md (volatile status — tracked internally instead).

Kept (the durable triage infrastructure from #1417):

  • Verified baseline example tools/launcher/examples/Qwen/Qwen3-8B/eagle3_quick_check.yaml.
  • Launcher common scripts (vLLM native-extractor dump, etc.) and compute_hidden_states_vllm.py.
  • modelopt code fixes: FakeBaseModel VLM detection, consolidated.safetensors load, use_cache export templates.
  • New-model triage guide (eagle3_new_model_triage_guide.md); its "document results" step now points at the internal tracker rather than the removed chart.

Testing

No code paths change — this only removes example YAMLs and two status docs and edits one doc reference. Pre-commit (ruff/markdownlint/yaml/license) passes on the kept/edited files.

Before your PR is "Ready for review"

  • Is this change backward compatible?: ✅ (removes unverified examples only; kept infra unchanged)
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
  • Did you write any new necessary tests?: N/A
  • Did you update Changelog?: N/A
  • Did you get Claude approval on this PR?: ❌ (pending)

Additional Information

Follow-up to #1417. Next step (tracked separately): verify each removed model end-to-end in modelopt, then re-add its YAML in a dedicated PR. Note: the nmm-sandbox weekly EAGLE3 CI is being trimmed to the Qwen3-8B baseline to match.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Updated EAGLE3 triage guide to streamline the verification workflow; contributors now record test outcomes (status, experiment IDs, errors, and applied fixes) in the team's internal triage tracker before submitting model launcher configurations.
  • Chores

    • Removed legacy EAGLE3 example pipeline configurations and deprecated triage documentation to reduce maintenance overhead.

Per review (Hao Guo): main should be production-ready, and most of the EAGLE3
model example YAMLs added in #1417 are not yet verified to work. Land the triage
infrastructure now and re-add model examples in follow-up PRs once each is
verified end-to-end.

Removed:
- All unverified per-model launcher configs (hf_offline_eagle3.yaml +
  eagle3_quick_check.yaml) for DeepSeek-V3.2, GLM-5, MiniMax-M2.5, Ministral-3-8B/14B,
  Kimi-K2.5 (+NVFP4), GPT-OSS-20B, Qwen3.5-9B/27B/35B-A3B, Step-3.5-Flash.
- Per-model status docs (EAGLE3_TRIAGE.md, eagle3_triage_chart.md).

Kept:
- Verified baseline example Qwen/Qwen3-8B/eagle3_quick_check.yaml.
- Triage/infra code: launcher common scripts (vLLM native-extractor dump),
  compute_hidden_states_vllm.py, and the modelopt fixes (FakeBaseModel VLM
  detection, consolidated.safetensors load, use_cache export).
- New-model triage guide (status tracking pointed to the internal tracker).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ye Yu <yeyu@nvidia.com>
@yeyu-nvidia
yeyu-nvidia requested a review from a team as a code owner June 3, 2026 20:07
@coderabbitai

coderabbitai Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 87456e93-d489-46b4-829a-98fef24766a6

📥 Commits

Reviewing files that changed from the base of the PR and between a7b0a92 and f5f061f.

⛔ Files ignored due to path filters (1)
  • .claude/scheduled_tasks.lock is excluded by !**/*.lock
📒 Files selected for processing (27)
  • examples/speculative_decoding/pipeline/eagle3/eagle3_new_model_triage_guide.md
  • examples/speculative_decoding/pipeline/eagle3/eagle3_triage_chart.md
  • tools/launcher/examples/DeepSeek/DeepSeek-V3.2/eagle3_quick_check.yaml
  • tools/launcher/examples/DeepSeek/DeepSeek-V3.2/hf_offline_eagle3.yaml
  • tools/launcher/examples/EAGLE3_TRIAGE.md
  • tools/launcher/examples/GLM/GLM-5/eagle3_quick_check.yaml
  • tools/launcher/examples/GLM/GLM-5/hf_offline_eagle3.yaml
  • tools/launcher/examples/MiniMax/MiniMax-M2.5/eagle3_quick_check.yaml
  • tools/launcher/examples/MiniMax/MiniMax-M2.5/hf_offline_eagle3.yaml
  • tools/launcher/examples/Mistral/Ministral-3-14B/eagle3_quick_check.yaml
  • tools/launcher/examples/Mistral/Ministral-3-14B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Mistral/Ministral-3-8B/eagle3_quick_check.yaml
  • tools/launcher/examples/Mistral/Ministral-3-8B/hf_offline_eagle3.yaml
  • tools/launcher/examples/MoonshotAI/Kimi-K2.5/eagle3_quick_check.yaml
  • tools/launcher/examples/MoonshotAI/Kimi-K2.5/hf_offline_eagle3.yaml
  • tools/launcher/examples/NVIDIA/Kimi-K2.5-NVFP4/eagle3_quick_check.yaml
  • tools/launcher/examples/NVIDIA/Kimi-K2.5-NVFP4/hf_offline_eagle3.yaml
  • tools/launcher/examples/OpenAI/GPT-OSS-20B/eagle3_quick_check.yaml
  • tools/launcher/examples/OpenAI/GPT-OSS-20B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-27B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-27B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-35B-A3B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-35B-A3B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-9B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-9B/hf_offline_eagle3.yaml
  • tools/launcher/examples/StepFun/Step-3.5-Flash/eagle3_quick_check.yaml
  • tools/launcher/examples/StepFun/Step-3.5-Flash/hf_offline_eagle3.yaml
💤 Files with no reviewable changes (26)
  • examples/speculative_decoding/pipeline/eagle3/eagle3_triage_chart.md
  • tools/launcher/examples/OpenAI/GPT-OSS-20B/hf_offline_eagle3.yaml
  • tools/launcher/examples/StepFun/Step-3.5-Flash/eagle3_quick_check.yaml
  • tools/launcher/examples/EAGLE3_TRIAGE.md
  • tools/launcher/examples/NVIDIA/Kimi-K2.5-NVFP4/hf_offline_eagle3.yaml
  • tools/launcher/examples/MoonshotAI/Kimi-K2.5/eagle3_quick_check.yaml
  • tools/launcher/examples/MiniMax/MiniMax-M2.5/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-27B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-35B-A3B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-9B/eagle3_quick_check.yaml
  • tools/launcher/examples/DeepSeek/DeepSeek-V3.2/hf_offline_eagle3.yaml
  • tools/launcher/examples/GLM/GLM-5/eagle3_quick_check.yaml
  • tools/launcher/examples/Mistral/Ministral-3-14B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-9B/hf_offline_eagle3.yaml
  • tools/launcher/examples/Mistral/Ministral-3-14B/eagle3_quick_check.yaml
  • tools/launcher/examples/Mistral/Ministral-3-8B/hf_offline_eagle3.yaml
  • tools/launcher/examples/DeepSeek/DeepSeek-V3.2/eagle3_quick_check.yaml
  • tools/launcher/examples/NVIDIA/Kimi-K2.5-NVFP4/eagle3_quick_check.yaml
  • tools/launcher/examples/MoonshotAI/Kimi-K2.5/hf_offline_eagle3.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-35B-A3B/hf_offline_eagle3.yaml
  • tools/launcher/examples/StepFun/Step-3.5-Flash/hf_offline_eagle3.yaml
  • tools/launcher/examples/Mistral/Ministral-3-8B/eagle3_quick_check.yaml
  • tools/launcher/examples/OpenAI/GPT-OSS-20B/eagle3_quick_check.yaml
  • tools/launcher/examples/Qwen/Qwen3.5-27B/hf_offline_eagle3.yaml
  • tools/launcher/examples/MiniMax/MiniMax-M2.5/eagle3_quick_check.yaml
  • tools/launcher/examples/GLM/GLM-5/hf_offline_eagle3.yaml

📝 Walkthrough

Walkthrough

This PR consolidates the EAGLE3 triage workflow by updating contributor guidance to use an internal triage tracker instead of maintaining multiple documentation files, and removes all associated example pipeline configurations and deprecated triage charts.

Changes

EAGLE3 Triage Workflow Consolidation

Layer / File(s) Summary
Triage guide update and deprecation cleanup
examples/speculative_decoding/pipeline/eagle3/eagle3_new_model_triage_guide.md, examples/speculative_decoding/pipeline/eagle3/eagle3_triage_chart.md, tools/launcher/examples/EAGLE3_TRIAGE.md, tools/launcher/examples/DeepSeek/DeepSeek-V3.2/eagle3_quick_check.yaml, tools/launcher/examples/DeepSeek/DeepSeek-V3.2/hf_offline_eagle3.yaml, tools/launcher/examples/GLM/GLM-5/*, tools/launcher/examples/MiniMax/MiniMax-M2.5/*, tools/launcher/examples/Mistral/Ministral-3-*/*, tools/launcher/examples/MoonshotAI/Kimi-K2.5/*, tools/launcher/examples/NVIDIA/Kimi-K2.5-NVFP4/*, tools/launcher/examples/OpenAI/GPT-OSS-20B/*, tools/launcher/examples/Qwen/Qwen3.5-*/*, tools/launcher/examples/StepFun/Step-3.5-Flash/*
Step 6 of the triage guide directs contributors to record run outcomes (per-task status, experiment IDs, errors, applied fixes) in the internal triage tracker and capture new failure patterns before submitting a dedicated PR for the verified model's launcher YAML. Deprecated eagle3_triage_chart.md (207 lines of model test matrix, triage decision tree, per-model results, and failure catalog) and EAGLE3_TRIAGE.md (272 lines of pipeline overview and known issues) are removed. Example EAGLE3 offline and quick-fail pipeline YAML configurations for DeepSeek-V3.2, GLM-5, MiniMax-M2.5, Mistral Ministral (3-14B and 3-8B), MoonshotAI Kimi-K2.5, NVIDIA Kimi-K2.5-NVFP4, OpenAI GPT-OSS-20B, Qwen (3.5-27B, 3.5-35B-A3B, 3.5-9B), and StepFun Step-3.5-Flash are all deleted.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • NVIDIA/Model-Optimizer#1417: Introduced the EAGLE3 triage documentation and example pipeline YAMLs that are being removed/consolidated in this PR.

Suggested reviewers

  • ChenhanYu
  • kevalmorabia97
  • shengliangxu
  • h-guo18
🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main change: reverting unverified EAGLE3 model examples while retaining triage code and a baseline example.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed This PR contains no Python code changes—only removes YAML configs and Markdown docs, and edits one Markdown file. The security anti-patterns check applies to Python changes, of which there are none.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch yeyu/revert-eagle3-unverified-examples

Comment @coderabbitai help to get the list of available commands and usage tips.

@h-guo18 h-guo18 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yeyu-nvidia
yeyu-nvidia enabled auto-merge (squash) June 3, 2026 20:11
@codecov

codecov Bot commented Jun 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.79%. Comparing base (a7b0a92) to head (f5f061f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1623      +/-   ##
==========================================
+ Coverage   74.57%   74.79%   +0.21%     
==========================================
  Files         482      482              
  Lines       52960    52960              
==========================================
+ Hits        39495    39610     +115     
+ Misses      13465    13350     -115     
Flag Coverage Δ
examples 42.89% <ø> (+0.40%) ⬆️
regression 15.20% <ø> (+0.10%) ⬆️
unit 53.93% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yeyu-nvidia
yeyu-nvidia merged commit 5bd04c3 into main Jun 3, 2026
41 checks passed
@yeyu-nvidia
yeyu-nvidia deleted the yeyu/revert-eagle3-unverified-examples branch June 3, 2026 20:56
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-06-03 20:57 UTC

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this file should not be committed in. @yeyu-nvidia

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — agreed, it's an ephemeral scheduler lock that slipped in. Removed from tracking + added a .gitignore rule in #1758.

sychen52 pushed a commit that referenced this pull request Jun 16, 2026
## What
Remove `.claude/scheduled_tasks.lock` from version control and add a
`.gitignore` rule so it is never committed again.

## Why
This file is an **ephemeral Claude Code scheduler lock** — its contents
are runtime process state (`sessionId`, `pid`, `procStart`,
`acquiredAt`), not source. It was accidentally committed in #1623 and is
currently tracked on `main`.

Reported by @sychen52 in [review of
#1623](#1623 (review)).

## Changes
- `git rm --cached .claude/scheduled_tasks.lock`
- Add `.claude/scheduled_tasks.lock` to `.gitignore`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated repository configuration to exclude internal runtime lock
files from version control.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Ye Yu <yeyu@nvidia.com>
@kevalmorabia97 kevalmorabia97 added cherry-pick-0.45.0 cherry-pick-done Added by bot once PR is cherry-picked to the release branch and removed cherry-pick-done Added by bot once PR is cherry-picked to the release branch labels Jun 23, 2026
kevalmorabia97 pushed a commit that referenced this pull request Jun 23, 2026
## What
Remove `.claude/scheduled_tasks.lock` from version control and add a
`.gitignore` rule so it is never committed again.

## Why
This file is an **ephemeral Claude Code scheduler lock** — its contents
are runtime process state (`sessionId`, `pid`, `procStart`,
`acquiredAt`), not source. It was accidentally committed in #1623 and is
currently tracked on `main`.

Reported by @sychen52 in [review of
#1623](#1623 (review)).

## Changes
- `git rm --cached .claude/scheduled_tasks.lock`
- Add `.claude/scheduled_tasks.lock` to `.gitignore`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated repository configuration to exclude internal runtime lock
files from version control.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Ye Yu <yeyu@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants