Add DeepSeek-V4-Flash export support - #405
Merged
Merged
Conversation
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Performance Comparison
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
justinchuby
force-pushed
the
dsv4-flash-export
branch
from
July 16, 2026 15:36
cad765c to
cfdb09e
Compare
Member
Author
Member
Author
|
Updated with DeepSeek-V4-Flash MTP + CSA export (commit
Known runtime gaps (dense fallback used until these land):
Validation: Ruff clean; 82 passed / 0 failed in the DeepSeek+MTP+CSA test scope (152 in the agent's full run). |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
justinchuby
force-pushed
the
dsv4-flash-export
branch
from
July 25, 2026 00:08
7e26e6e to
6b5e2e9
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds DeepSeek-V4-Flash export support to mobius, including a new model/task pair, GGUF config + tensor mapping for the deepseek4 GGUF architecture, and ORT GenAI export metadata (including an optional MTP sidecar).
Changes:
- Introduces
DeepSeekV4CausalLMModelandDeepSeekV4Task(decoder + optional MTP sidecar) and registersdeepseek_v4across the registry/tasks. - Adds DeepSeek4 GGUF metadata extraction + tensor-name mapping, plus loader support for deferred quantized projections.
- Extends ORT GenAI auto-export to recognize
deepseek_v4, handle multi-component packages’ decoder filename, and emitmtp_config.jsonwhen present; adds unit tests and design doc.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/model_coverage_test.py | Adds deepseek_v4 to “registered without test_model_id” coverage list. |
| tests/_test_configs.py | Adds representative tiny config coverage for deepseek_v4. |
| src/mobius/tasks/_deepseek_v4.py | New task building decoder and optional mtp sidecar graphs with KV cache I/O. |
| src/mobius/tasks/init.py | Exposes DeepSeekV4Task and registers task key deepseek-v4. |
| src/mobius/models/deepseek_v4.py | New DeepSeek-V4 dense-fallback model implementation + weight remapping. |
| src/mobius/models/deepseek_v4_flash_test.py | Adds config extraction, graph-build, mapping, quantization, and ORT GenAI metadata tests. |
| src/mobius/models/init.py | Exports DeepSeekV4CausalLMModel. |
| src/mobius/integrations/ort_genai/auto_export.py | Adds model-type alias, fixes decoder filename for multi-component packages, emits mtp_config.json. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Adds deepseek4 tensor mapping and “@ suffix” rule for no-suffix params. |
| src/mobius/integrations/gguf/_tensor_mapping_test.py | Adds mapping tests for deepseek4. |
| src/mobius/integrations/gguf/_reader_test.py | Adds arch→model_type mapping and DeepSeek4 config extraction test. |
| src/mobius/integrations/gguf/_config_mapping.py | Adds DeepSeek4 metadata field mapping + YaRN rope_scaling assembly + swiglu_limit list handling. |
| src/mobius/integrations/gguf/_builder.py | Treats deferred GGUF quantized projections as repack targets. |
| src/mobius/_registry.py | Registers deepseek_v4 model type, default model id, and metadata tags. |
| src/mobius/_configs/_base.py | Adds DeepSeek-V4 config fields and HF config extraction logic. |
| DSV4_FLASH_EXPORT.md | Documents DeepSeek-V4-Flash authoritative config and architecture gaps. |
Comment on lines
+137
to
+145
| self.hash_routing = layer_id < config.num_hash_layers | ||
| self.weight = nn.Parameter([self.num_experts, config.hidden_size]) | ||
| if self.hash_routing: | ||
| self.tid2eid = nn.Parameter( | ||
| [config.vocab_size, self.top_k], dtype=ir.DataType.INT32 | ||
| ) | ||
| else: | ||
| self.bias = nn.Parameter([self.num_experts]) | ||
|
|
Comment on lines
+163
to
+165
| if self.hash_routing: | ||
| selected_experts = op.Gather(self.tid2eid, input_ids, axis=0) | ||
| else: |
Member
Author
|
Addressed both Copilot findings: the router bias is now materialized for hash-routed layers so all GGUF checkpoint biases load, and hash-route expert indices are cast to INT64 before GatherElements. |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/mobius/tasks/_deepseek_v4.py:64
- KV cache output stamping is hard-coded to
num_kv_heads=1. This should useconfig.num_key_value_headsto keep the present KV output types consistent with the cache inputs and the resolved config.
_register_kv_cache_outputs(
builder,
presents,
batch=batch,
num_kv_heads=1,
Comment on lines
+36
to
+44
| past_key_values = _make_kv_cache_inputs( | ||
| builder, | ||
| num_layers, | ||
| 1, | ||
| config.head_dim, | ||
| config.dtype, | ||
| batch, | ||
| past_sequence_length, | ||
| ) |
Comment on lines
+864
to
+870
| num_hash_layers=( | ||
| getattr(config, "num_hash_layers", None) | ||
| or getattr(config, "n_hash_layers", 0) | ||
| or _leading_layer_type_count( | ||
| getattr(config, "mlp_layer_types", None), "hash_moe" | ||
| ) | ||
| ), |
Comment on lines
+670
to
+673
| hidden_states = op.Expand( | ||
| op.Unsqueeze(hidden_states, [-2]), | ||
| [1, 1, self.hc_mult, 1], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
deepseek_v4and map the real GGUFdeepseek4architectureonnx-genai/ort-genairuntime aliasesValidation
151 passedfocused DeepSeek/GGUF/config tests37 passedDeepSeek graph/checker tests759 passed, 210 skippedV4/GGUF/model-coverage tests5 passedruntime CLI testsDeferred follow-ups
Closes no issue.