Skip to content

Add ONNX export and quantization skill - #238

Merged
justinchuby merged 2 commits into
mainfrom
skill-onnx-export-quantization
May 4, 2026
Merged

Add ONNX export and quantization skill#238
justinchuby merged 2 commits into
mainfrom
skill-onnx-export-quantization

Conversation

@justinchuby

Copy link
Copy Markdown
Member

Document the Gemma4 ONNX export and INT4 quantization workflow as a reusable skill.

What this adds

New skill at .agents/skills/onnx-export-quantization/SKILL.md covering:

  • mobius CLI exportmobius build flags, EP variants (default/cuda/onnx-standard), multi-model outputs
  • Olive quantization — Q4_K_M (k-quant) and NF4 (4-bit NormalFloat) with code examples
  • GPU acceleration — cupy for 19-51x kquant speedup
  • HuggingFace upload — standard directory layout (dtype × EP matrix), upload verification
  • Common issues — MoE weight mapping, hybrid attention v_proj, BF16 type mismatches, incomplete uploads
  • Testing — L4 golden data generation, L5 end-to-end smoke test, expected tolerances

This captures the workflow used for the Gemma4 ONNX exports so future model exports follow the same pattern.

Document the full workflow for exporting models with mobius CLI and
quantizing with Olive:
- mobius build CLI flags and EP variants (default/cuda/onnx-standard)
- INT4 quantization: Q4_K_M (k-quant) and NF4 with Olive
- GPU-accelerated quantization with cupy (19-51x speedup)
- HuggingFace Hub upload structure and verification
- Common issues: MoE weight mapping, hybrid attention, BF16 mismatches
- Testing quantized models (L4 golden data, L5 smoke test)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown

Performance Comparison

Comparing cefc55e86508a0

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 60 60 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 107 107 +0.0%
gpt2 model_size_bytes 388 KB 388 KB +0.0%
gpt2 num_nodes 53 53 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 61 61 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 58 58 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 98 98 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 59 59 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 56 56 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 61 61 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 58 58 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 275 275 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 129 129 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 408 408 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 166 166 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

@codecov

codecov Bot commented May 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new reusable skill document under .agents/skills/ describing the mobius ONNX export workflow and Olive INT4 quantization workflow (aimed at standardizing how Gemma4 exports are produced, validated, and uploaded).

Changes:

  • Add a new skill doc for mobius build export flags, EP variants, and multi-model output layout.
  • Document INT4 quantization approaches (Q4_K_M / NF4), including optional GPU acceleration with cupy.
  • Document HuggingFace Hub directory layout, common failure modes, and suggested testing (golden + smoke tests).

Comment thread .agents/skills/onnx-export-quantization/SKILL.md Outdated
Comment thread .agents/skills/onnx-export-quantization/SKILL.md Outdated
Comment thread .agents/skills/onnx-export-quantization/SKILL.md Outdated
Comment thread .agents/skills/onnx-export-quantization/SKILL.md
Comment thread .agents/skills/onnx-export-quantization/SKILL.md
@justinchuby
justinchuby requested a review from rui-ren May 4, 2026 17:43
Address all 5 review findings plus remove --optimize from examples:

1. Fix --optimize description: clarify it applies mobius rewrite rules
   (group_query_attention, packed_attention, skip_norm), not general
   constant folding. Remove from basic command examples since it's
   optional, not default.

2. Fix EP descriptions: 'default' is portable ONNX (not CPU-specific),
   'onnx-standard' inlines custom-domain functions (not DML-specific).
   Add note about 'mobius list eps' for all available EPs.

3. Replace Olive direct-API snippets with config-driven olive.run()
   pattern matching the repo's examples/olive/ convention. Reference
   the ministral example for a complete working setup.

4. Add tokenizer + processor config files to the multi-model
   quantization copy step — without these ORT GenAI won't load.

5. Replace fabricated generate_golden_data() with the real
   scripts/generate_golden.py entrypoint and compare_golden() from
   mobius._testing.parity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
@justinchuby
justinchuby enabled auto-merge (squash) May 4, 2026 17:49
@justinchuby
justinchuby merged commit 474f5b1 into main May 4, 2026
20 checks passed
@justinchuby
justinchuby deleted the skill-onnx-export-quantization branch May 4, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants