From 68fc4789c31958b14cccf80057852d556a5daef8 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 12 Aug 2026 17:29:59 +0000 Subject: [PATCH] model(deepseek-v4): DeepSeek-V4-Pro is the SAME architecture as V4-Flash -- gate the config descent's shape-generality (#504) `deepseek-ai/DeepSeek-V4-Pro` is `DeepseekV4ForCausalLM` / `model_type: deepseek_v4`. A key-by-key diff of both shipped `config.json` files has ZERO new and ZERO removed keys: hidden 4096->7168, layers 43->61, heads 64->128, experts 256->384, moe_inter 2048->3072, q_lora 1024->1536, o_groups 8->16, index_topk 512->1024, routed_scaling 1.5->2.5, and compress_ratios[0:2] [0,0]->[128,128]. Everything else is identical, including head_dim 512, hc_mult 4, num_hash_layers 3, sqrtsoftplus/noaux_tc and expert_dtype fp4. No architecture work is owed: Pro is a CONFIG VARIANT of this row, not a new model row. Grounded three ways rather than asserted. Upstream (pin 555967922, vLLM 0.26.0.dev0): one `vllm/models/deepseek_v4/` package serves both, every dimension read off `config` (`nvidia/model.py:535-679`, `attention.py:193-211`, `:708`). Compressor presence is `compress_ratio > 1` (`attention.py:334`) over `max(1, compress_ratios[layer_id])` (`:209`), and the compressor asserts `compress_ratio in [4, 128]` (`compressor.py:171`). Ratio 0 maps to 1 and so means NO compressor -- the same polarity as our `has_compressor(l) = ratio != 0` (`deepseek_v4.h:127`). Read the whole ~38-file package for literal Flash dimensions: none. Real checkpoints: normalizing layer/expert indices out of both `model.safetensors.index.json` files gives 98 distinct tensor-name patterns on each side with ZERO unique to either (Pro 145116 tensors, Flash 69187). Predicting compressor/indexer layers from `compress_ratios` alone reproduces both exactly -- Flash 41/21, Pro 61/30 -- and Pro's layers 0-1 carry the 4-tensor ratio-128 compressor group where Flash has none. Our side: the only shape assertion in 6930 lines of V4 sources is `head_dim == 512` (`deepseek_v4_weights.cpp:173`), which Pro satisfies; all dims come off config (`deepseek_v4.cpp:1725-1731`) and all CUDA shared memory is `extern __shared__`. The new CPU gate (5 cases / 197 assertions, no fixture, no GPU) drives `ParseDeepseekV4Params` with the real Pro values and the REAL compress_ratios arrays, asserts the 61/30 and 41/21 counts and the layer-0/1 delta, and carries Flash as the control arm so nothing can be satisfied Flash-shaped. RED evidence by mutation, tree restored byte-for-byte (`md5sum -c` OK) after each: hardcoding `num_hidden_layers = 43` reds 3 cases / 5 assertions; adding `layer >= 2 &&` to `has_compressor` reds 2 / 3; hardcoding `o_groups = 8` reds 2 / 3. Restored: 5/5 197 SUCCESS. All nine V4 suites green on the same build -- scaffold 62, dsa 38, compressor 164, moe 716, mhc 125, forward 26, mtp 29, gguf_load 931, pro_variant 197. Hardware verdict recorded, not worked around: 1.599T params is 805 GiB native, 864 GiB for `nvidia/DeepSeek-V4-Pro-NVFP4`, and 534.7 GiB for the smallest real GGUF (`teamblobfish` Q2_K-XL). One GB10 is 119 GiB, so Q2 is ~4.5x over and the ~400 GB 2-bit floor means no quantization closes it. The pinned oracle needs the same memory, so Pro's load/forward/strict gates are PENDING an external resource -- a named blocker, not a correctable finding. No lifecycle transition, so no STATUS/BENCHMARKS write is owed. Docs surfaces are untouched deliberately: no user-visible capability changed, because Pro cannot be run on this hardware. Filed while assessing this: #505 -- `DsaTopkKernel` sizes `chosen[512]`/ `picked[64]` by literal while `index_topk` is 512 (Flash) / 1024 (Pro). Latent today only because `dsa_dense = (be.gguf != nullptr)` (`deepseek_v4.cpp:668`) forces the indexer off on the real path; it becomes a silent thread-stack overflow once the real-geometry DSA residual lands. Spec section 5 records the upstream candidate-window contract (`indexer.py:270-290`, full causal prefix in COMPRESSED-key space) for whoever builds that out. `test_cpu_x86_llamacpp_floor` is red in preflight from box contention, not from this change: the pristine base SHA fdd45263 fails the same test with NO_QUIET_WINDOW at load 43+ driven by two concurrent sessions' builds, and the harness reads none of the four paths here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/roadmap_v1.md | 2 + .agents/specs/deepseek-v4-pro.md | 192 ++++++++++++ tests/CMakeLists.txt | 8 + .../models/test_deepseek_v4_pro_variant.cpp | 295 ++++++++++++++++++ 4 files changed, 497 insertions(+) create mode 100644 .agents/specs/deepseek-v4-pro.md create mode 100644 tests/vllm/models/test_deepseek_v4_pro_variant.cpp diff --git a/.agents/roadmap_v1.md b/.agents/roadmap_v1.md index eb5a46133..596919f94 100644 --- a/.agents/roadmap_v1.md +++ b/.agents/roadmap_v1.md @@ -139,6 +139,8 @@ issue is not yet placed. Keyed record: update in place, never append. | [#285](https://github.com/mudler/vllm.cpp/issues/285) | — | The operator lock refuses a second coordinator; it should only RECORD who is working where (spec `specs/operator-record.md`) | bug | | [#296](https://github.com/mudler/vllm.cpp/issues/296) | — | Two limitations recorded when #285 landed: a stale TTL comment, and a publish-NAME pin `os.rename` escapes (spec `specs/operator-record.md`, "Follow-up") | bug | | [#408](https://github.com/mudler/vllm.cpp/issues/408) | — | 12 of 54 `tests/scripts` suites are executed by nothing, and `check-test-registration.py`'s fixed `REQUIRED_TESTS` cannot see the class (found while repairing #274) | bug | +| [#504](https://github.com/mudler/vllm.cpp/issues/504) | `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` | DeepSeek-V4-Pro is the same architecture as V4-Flash (zero new config keys): record the variant and gate the config descent's shape-generality (spec `specs/deepseek-v4-pro.md`) | feature | +| [#505](https://github.com/mudler/vllm.cpp/issues/505) | `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` | `DsaTopkKernel` sizes `chosen[512]`/`picked[64]` by literal while `index_topk` is 512 (Flash) / 1024 (Pro); latent behind `dsa_dense` today, silent stack overflow once the real-geometry DSA residual lands (found while assessing #504) | bug | ## Top-level portfolio diff --git a/.agents/specs/deepseek-v4-pro.md b/.agents/specs/deepseek-v4-pro.md new file mode 100644 index 000000000..23b9d8bd0 --- /dev/null +++ b/.agents/specs/deepseek-v4-pro.md @@ -0,0 +1,192 @@ +# DeepSeek-V4-Pro — variant assessment (records + one test) + +**Issue:** [#504](https://github.com/mudler/vllm.cpp/issues/504). +**Row:** `MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm` (`DeepseekV4ForCausalLM`, ✅). +**Claim:** `CLAIM-DEEPSEEK-V4-PRO-VARIANT`. +**Base:** `origin/main` @ `fdd452637fdd5550e41bcf808300fcc9158d754f`. +**Pinned oracle:** `${VLLM_SOURCE}` @ `5559679229bc961848b121ccdeaa8fa5d79bec98` (vLLM 0.26.0.dev0). +**State:** records + test. NO download, NO GPU, NO forward. Pro is memory-infeasible +on this hardware (§4); nothing here claims it runs. + +--- + +## 0. Scope + +`deepseek-ai/DeepSeek-V4-Pro` is `DeepseekV4ForCausalLM` / `model_type: deepseek_v4` +— the **same architecture** as DeepSeek-V4-Flash, which this row already supports. +This spike establishes that with three independent groundings, adds the gate that +keeps it true, and records the hardware verdict. + +**Verdict: no architecture work is owed.** Pro is a **config variant of this row**, +not a new model row. What blocks running it is memory (§4) and the already-named +real-geometry DSA residual (§5) — neither is a Pro-specific gap. + +Not new, despite the framing that prompted this: the HF repo was created +2026-04-22 and last modified 2026-06-22, with ~1.4M downloads at time of writing. +Upstream support predates our pin. + +## 1. Config diff — zero new keys + +Key-by-key over both shipped `config.json` files (fetched 2026-08-12): **zero new +keys, zero removed keys.** Every difference is a scaled value. + +| key | Flash | Pro | +|---|---|---| +| `hidden_size` | 4096 | 7168 | +| `num_hidden_layers` | 43 | 61 | +| `num_attention_heads` | 64 | 128 | +| `n_routed_experts` | 256 | 384 | +| `moe_intermediate_size` | 2048 | 3072 | +| `q_lora_rank` | 1024 | 1536 | +| `o_groups` | 8 | 16 | +| `index_topk` | 512 | 1024 | +| `routed_scaling_factor` | 1.5 | 2.5 | +| `compress_ratios[0:2]` | `[0, 0, …]` | `[128, 128, …]` | + +Identical on both: `head_dim` 512, `qk_rope_head_dim` 64, `o_lora_rank` 1024, +`hc_mult` 4, `hc_eps`, `hc_sinkhorn_iters` 20, `num_hash_layers` 3, +`n_shared_experts` 1, `num_experts_per_tok` 6, `sliding_window` 128, +`scoring_func: sqrtsoftplus`, `topk_method: noaux_tc`, `swiglu_limit` 10.0, +`expert_dtype: fp4`, `index_head_dim` 128, `index_n_heads` 64, +`compress_rope_theta` 160000, `vocab_size` 129280, YaRN `rope_scaling`. + +Heads per output-LoRA group is **8 on both** (64/8 and 128/16), so the `wo_a` bmm +batch shape is invariant even though both operands doubled. + +## 2. Upstream anchors — one package serves both + +All paths under `${VLLM_SOURCE}/vllm/`. + +- `model_executor/models/registry.py:95` — `DeepseekV4ForCausalLM` → + `vllm.models.deepseek_v4`. One entry; no Pro-specific arch string. +- `models/deepseek_v4/nvidia/model.py:535-679` — `hidden_size`, + `n_routed_experts`, `moe_intermediate_size` and every MoE dimension read off + `config`. `model.py:981` `hc_dim = hc_mult * config.hidden_size`. +- `models/deepseek_v4/attention.py:193-211` — `hidden_size`, `n_heads`, + `q_lora_rank`, `o_groups`, `window_size` all off `config`. +- `attention.py:209` — `compress_ratio = max(1, config.compress_ratios[layer_id])`, + guarded by `attention.py:208` `if layer_id < config.num_hidden_layers` (the + trailing MTP entry). `attention.py:334` creates the compressor only when + `compress_ratio > 1`, so **ratio 0 means no compressor** upstream. +- `models/deepseek_v4/compressor.py:171` — `assert compress_ratio in [4, 128]`. + Pro is inside that asserted set; `compressor.py:247` `overlap = ratio == 4` is + the Lightning-Indexer arm. +- `attention.py:708` — `topk_tokens = config.index_topk`. + +There is no Pro-specific code path upstream. Verified by reading the whole +`vllm/models/deepseek_v4/` package (~38 files) for literal Flash dimensions: none. + +## 3. Our side — shape-generic, now gated + +The only shape assertion across the 6930 lines of V4 sources is +`head_dim == 512` (`deepseek_v4_weights.cpp:173`), which Pro satisfies. Every +dimension is read from config (`deepseek_v4.cpp:1725-1731`), all CUDA shared +memory is dynamic (`extern __shared__`), and `compress_ratios` is consumed **by +value**: `has_compressor(l) = ratio != 0`, `has_indexer(l) = ratio == 4` +(`deepseek_v4.h:127-128`). That matches upstream's `> 1` / `== 4` polarity +exactly. + +**Checkpoint-level grounding.** Normalizing layer and expert indices out of both +real `model.safetensors.index.json` files gives **98 distinct tensor-name patterns +on each side, with zero patterns unique to either** (Pro 145116 tensors, Flash +69187). Predicting compressor/indexer layers from `compress_ratios` alone +reproduces both real checkpoints exactly: + +| | predicted compressor / indexer | actual in checkpoint | +|---|---|---| +| Flash (43L) | 41 / 21 | 41 / 21 | +| Pro (61L) | 61 / 30 | 61 / 30 | + +Pro's layers 0-1 carry the 4-tensor ratio-128 compressor group where Flash has +none — the single structural difference between the two configs. + +**Gate:** `tests/vllm/models/test_deepseek_v4_pro_variant.cpp`, 5 cases / +197 assertions, CPU-only, no fixture. It drives `ParseDeepseekV4Params` with the +real Pro values and the **real** `compress_ratios` arrays, asserts the 61/30 and +41/21 counts above, and asserts the layer-0/1 delta directly, with Flash as the +control arm so the assertions cannot be satisfied by a Flash-shaped implementation. + +**RED evidence (mutation, tree restored byte-for-byte after each):** + +| mutation | site | result | +|---|---|---| +| A: `p.num_hidden_layers = 43` | `deepseek_v4_weights.cpp:111` | 3 cases / 5 assertions FAIL | +| B: `has_compressor` gains `layer >= 2 &&` | `deepseek_v4.h:127` | 2 cases / 3 assertions FAIL | +| C: `p.o_groups = 8` | `deepseek_v4_weights.cpp:132` | 2 cases / 3 assertions FAIL | + +Restored: `md5sum -c` OK on both files, then 5/5 · 197 SUCCESS again. + +**No regression:** all nine V4 suites green on the same build — scaffold 62, +dsa 38, compressor 164, moe 716, mhc 125, forward 26, mtp 29, gguf_load 931, +pro_variant 197. + +## 4. Hardware verdict — memory-infeasible on one GB10 + +1.599T params (49B activated). + +| artifact | size | +|---|---| +| `deepseek-ai/DeepSeek-V4-Pro` (fp8 tower + fp4 experts) | **805 GiB** | +| `nvidia/DeepSeek-V4-Pro-NVFP4` | 864 GiB | +| `teamblobfish/DeepSeek-V4-Pro-GGUF` Q2_K-XL (smallest found) | **534.7 GiB** | +| `…` Q4_K_M-XL | 889 GiB | + +One GB10 is 119 GiB unified. Q2 is ~4.5x over, and 1.6T params at 2 bits has a +~400 GB floor, so no quantization closes this. The pinned oracle needs the same +memory, so there is **no gateable denominator on this hardware either** — Flash +already required 2 Sparks; Pro needs roughly 5-8 at Q2. + +Consequence: the load/forward/strict gates for Pro are **PENDING an external +resource**, not failing. That is a named external blocker, not a correctable +finding. + +## 5. Residual — real-geometry DSA (pre-existing, not Pro-specific) + +`dsa_dense = (be.gguf != nullptr)` (`deepseek_v4.cpp:668`) forces `is_indexer` +and `is_comp` false on the real keep-quant path, so the shipped Flash run uses +**dense MLA**. That is exact — not an approximation — only while +`seq_len <= index_topk`: the indexer cannot select more tokens than exist, so +top-k over a shorter prefix is the full causal set. For Flash that window is 512 +tokens; **for Pro it would be 1024**. Long-context V4 on either model needs the +real-geometry compressor cache + indexer selection, which is already a named +residual on this row. + +Grounding note for whoever builds it out: upstream's candidate window is +`ks = row_start`, `ke = row_start + (pos + 1) // COMPRESS_RATIO` +(`v1/attention/backends/mla/indexer.py:270-290`) — the full causal prefix in +**compressed**-key space. Our synthetic-geometry path uses `we[t] = t + 1` over +uncompressed keys (`deepseek_v4.cpp:806-808`), which is consistent at the +collapsed geometry but is not the real-geometry contract. Reconcile against that +kernel, not against our host reference. + +Related: [#505](https://github.com/mudler/vllm.cpp/issues/505) — `DsaTopkKernel` +sizes `chosen[512]` / `picked[64]` by literal while `index_topk` is 512 (Flash) / +1024 (Pro). Latent in a test-only path today precisely because of `dsa_dense` +above; it becomes a silent thread-stack overflow the moment this residual lands. + +## 6. Stop conditions + +- Do **not** attempt to download or load Pro on a single GB10. §4 is arithmetic, + not a measurement to retry. +- Do **not** open a new model row for Pro. It is this row's config variant, and + the §3 gate is what keeps that claim honest. +- Any Pro speed or correctness claim requires multi-Spark hardware **and** the §5 + residual; without both, report PENDING against the named resource. + +## Outcome + +Measured: the config diff (zero new keys), the 98-pattern tensor-name identity, +and the 61/30 · 41/21 compressor/indexer prediction against both real +checkpoints. Rejected: opening a separate Pro model row — three independent +groundings show one code path serves both, so a second row would duplicate a +record that cannot diverge. Rejected: any load or forward gate here — §4 makes it +memory-infeasible, and asserting a synthetic Pro forward would gate our own +arithmetic rather than the model. The `head_dim == 512` scope assertion stays as +it is: it is the one genuine narrowing, it holds for every shipped V4 checkpoint, +and widening it without a checkpoint that needs it would be untested surface. + +## Now + +Row unchanged at ✅ for Flash. Pro recorded as a covered config variant with the +§3 gate landed and the §4 hardware verdict PENDING an external resource. No +lifecycle transition, so no `STATUS`/`BENCHMARKS` write is owed by this change. diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 9f9888f54..cc1d0a3fb 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -166,6 +166,14 @@ target_include_directories(test_deepseek_v2_forward PRIVATE ${CMAKE_SOURCE_DIR}/ vllm_cpp_add_test(test_deepseek_v4_scaffold vllm/models/test_deepseek_v4_scaffold.cpp) target_include_directories(test_deepseek_v4_scaffold PRIVATE ${CMAKE_SOURCE_DIR}/src) +# DeepSeek-V4-Pro shape-generality gate (#504): the Pro config descends through +# the SAME ParseDeepseekV4Params with zero new keys, and compress_ratios alone +# reproduces the compressor/indexer layer counts of BOTH real checkpoints +# (Flash 41/21, Pro 61/30). No checkpoint / GPU — Pro's 805 GiB does not fit one +# GB10, so a load gate is memory-infeasible (.agents/specs/deepseek-v4-pro.md). +vllm_cpp_add_test(test_deepseek_v4_pro_variant vllm/models/test_deepseek_v4_pro_variant.cpp) +target_include_directories(test_deepseek_v4_pro_variant PRIVATE ${CMAKE_SOURCE_DIR}/src) + # Laguna-S-2.1 W1/W2 scaffolding gate: LagunaForCausalLM RESOLVES through the # registry + ParseLagunaParams descends the shipped config (nested dual-rope + # variable Q-head + ungrouped sigmoid-noaux MoE). No 73 GB checkpoint / GPU — the diff --git a/tests/vllm/models/test_deepseek_v4_pro_variant.cpp b/tests/vllm/models/test_deepseek_v4_pro_variant.cpp new file mode 100644 index 000000000..beb63c75b --- /dev/null +++ b/tests/vllm/models/test_deepseek_v4_pro_variant.cpp @@ -0,0 +1,295 @@ +// DeepSeek-V4-Pro SHAPE-GENERALITY gate (issue #504). +// +// `deepseek-ai/DeepSeek-V4-Pro` is `DeepseekV4ForCausalLM` / `model_type: +// deepseek_v4` — the SAME architecture as DeepSeek-V4-Flash, which this project +// already supports. A key-by-key diff of the two shipped `config.json` files has +// ZERO new and ZERO removed keys: every difference is a scaled value. This gate +// proves our config descent is genuinely shape-generic rather than Flash-tuned, +// which is the whole claim behind "no architecture work is owed for Pro". +// +// It proves that WITHOUT a checkpoint, a download, or a GPU — the 805 GiB Pro +// weights do not fit one GB10 (119 GiB), so a load/forward gate is memory- +// infeasible here (see the row spec). Nothing below claims Pro runs. +// +// ─── GROUNDED AGAINST UPSTREAM (pin 555967922, vLLM 0.26.0.dev0) ───────────── +// OURS <- UPSTREAM (vllm/) +// ParseDeepseekV4Params <- models/deepseek_v4/nvidia/model.py:535-679 +// + attention.py:193-211 — every dimension is +// read off `config`; there is NO Pro-specific +// path upstream, the same package serves both. +// p.has_compressor(l) <- attention.py:334 `if self.compress_ratio > 1` +// (== ratio != 0) over attention.py:209 +// `max(1, config.compress_ratios[layer_id])`. +// Ratio 0 becomes 1 upstream, and 1 > 1 is +// false, so ratio 0 means NO compressor on +// both sides — the polarity matches exactly. +// p.has_indexer(l) <- compressor.py:171 `assert compress_ratio in +// (== ratio == 4) [4, 128]` + compressor.py:181 (the ratio-4 +// arm is the overlapped Lightning-Indexer one, +// compressor.py:247 `self.overlap = ratio == 4`). +// index_topk <- attention.py:708 `self.topk_tokens = +// config.index_topk` (512 Flash / 1024 Pro). +// +// ─── GROUNDED AGAINST THE REAL CHECKPOINTS ────────────────────────────────── +// Normalizing layer and expert indices out of both real +// `model.safetensors.index.json` files yields 98 distinct tensor-name patterns on +// each side with ZERO unique to either. Predicting compressor/indexer layers from +// `compress_ratios` alone reproduces both checkpoints exactly: +// Flash (43L): predicted 41 compressor / 21 indexer == actual 41 / 21 +// Pro (61L): predicted 61 compressor / 30 indexer == actual 61 / 30 +// Those two pairs are the load-bearing assertions below. Pro's layers 0-1 carry +// the 4-tensor ratio-128 compressor group where Flash has none, which is the one +// STRUCTURAL difference between the two configs and is asserted directly. +#include "vllm/model_executor/models/deepseek_v4.h" + +#include + +#include + +#include +#include +#include + +using vllm::DeepseekV4Params; +using vllm::HfConfig; +using vllm::ParseDeepseekV4Params; + +namespace { + +// The real `compress_ratios` arrays, verbatim from the shipped configs +// (fetched 2026-08-12). Both are `num_hidden_layers + 1` long: the trailing +// entry is the MTP/nextn layer, which carries no compressor. +const std::vector& ProCompressRatios() { + static const std::vector v = { + 128, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, + 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, + 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 0}; + return v; +} + +const std::vector& FlashCompressRatios() { + static const std::vector v = { + 0, 0, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, + 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, 128, 4, + 128, 4, 0}; + return v; +} + +// One config builder parameterized by the values that DIFFER between the two +// checkpoints. Everything passed positionally here is a key whose value the +// Flash/Pro diff showed changing; every key hardcoded in the body is one the +// diff showed IDENTICAL on both. Keeping that split explicit is what makes this +// a generality gate rather than two copies of a fixture. +HfConfig V4Config(int64_t hidden_size, int64_t num_hidden_layers, + int64_t num_attention_heads, int64_t n_routed_experts, + int64_t moe_intermediate_size, int64_t q_lora_rank, int64_t o_groups, + int64_t index_topk, double routed_scaling_factor, + const std::vector& compress_ratios) { + HfConfig c; + c.architectures = {"DeepseekV4ForCausalLM"}; + c.hidden_size = hidden_size; + c.num_hidden_layers = num_hidden_layers; + c.vocab_size = 129280; + c.num_attention_heads = num_attention_heads; + c.num_key_value_heads = 1; + c.head_dim = 512; + c.rms_norm_eps = 1e-6; + c.max_position_embeddings = 1048576; + c.raw = { + {"hidden_size", hidden_size}, + {"num_hidden_layers", num_hidden_layers}, + {"num_attention_heads", num_attention_heads}, + {"n_routed_experts", n_routed_experts}, + {"moe_intermediate_size", moe_intermediate_size}, + {"q_lora_rank", q_lora_rank}, + {"o_groups", o_groups}, + {"index_topk", index_topk}, + {"routed_scaling_factor", routed_scaling_factor}, + {"compress_ratios", compress_ratios}, + // --- identical on Flash and Pro --- + {"vocab_size", 129280}, + {"num_key_value_heads", 1}, + {"head_dim", 512}, + {"qk_rope_head_dim", 64}, + {"o_lora_rank", 1024}, + {"sliding_window", 128}, + {"rms_norm_eps", 1e-6}, + {"max_position_embeddings", 1048576}, + {"num_nextn_predict_layers", 1}, + {"num_experts_per_tok", 6}, + {"n_shared_experts", 1}, + {"norm_topk_prob", true}, + {"swiglu_limit", 10.0}, + {"scoring_func", "sqrtsoftplus"}, + {"topk_method", "noaux_tc"}, + {"num_hash_layers", 3}, + {"expert_dtype", "fp4"}, + {"hc_mult", 4}, + {"hc_sinkhorn_iters", 20}, + {"hc_eps", 1e-6}, + {"index_head_dim", 128}, + {"index_n_heads", 64}, + {"compress_rope_theta", 160000}, + {"rope_theta", 10000}, + {"tie_word_embeddings", false}, + }; + return c; +} + +HfConfig ProConfig() { + return V4Config(/*hidden_size=*/7168, /*num_hidden_layers=*/61, + /*num_attention_heads=*/128, /*n_routed_experts=*/384, + /*moe_intermediate_size=*/3072, /*q_lora_rank=*/1536, + /*o_groups=*/16, /*index_topk=*/1024, + /*routed_scaling_factor=*/2.5, ProCompressRatios()); +} + +HfConfig FlashConfig() { + return V4Config(/*hidden_size=*/4096, /*num_hidden_layers=*/43, + /*num_attention_heads=*/64, /*n_routed_experts=*/256, + /*moe_intermediate_size=*/2048, /*q_lora_rank=*/1024, + /*o_groups=*/8, /*index_topk=*/512, + /*routed_scaling_factor=*/1.5, FlashCompressRatios()); +} + +// Count the compressor/indexer layers the way the forward does — through the +// params' own predicates, over [0, num_hidden_layers) only, so the trailing +// MTP entry is excluded exactly as upstream excludes it (attention.py:208). +struct DsaLayerCounts { + int64_t compressor = 0; + int64_t indexer = 0; +}; + +DsaLayerCounts CountDsaLayers(const DeepseekV4Params& p) { + DsaLayerCounts n; + for (int64_t l = 0; l < p.num_hidden_layers; ++l) { + if (p.has_compressor(l)) ++n.compressor; + if (p.has_indexer(l)) ++n.indexer; + } + return n; +} + +} // namespace + +TEST_CASE("deepseek-v4-pro: the real Pro config DESCENDS with no new keys") { + const DeepseekV4Params p = ParseDeepseekV4Params(ProConfig()); + + // Every value the Flash/Pro diff showed CHANGING must arrive scaled, not + // clamped to a Flash default. + CHECK(p.hidden_size == 7168); + CHECK(p.num_hidden_layers == 61); + CHECK(p.num_attention_heads == 128); + CHECK(p.n_routed_experts == 384); + CHECK(p.moe_intermediate_size == 3072); + CHECK(p.q_lora_rank == 1536); + CHECK(p.o_groups == 16); + CHECK(p.index_topk == 1024); + CHECK(p.routed_scaling_factor == doctest::Approx(2.5).scale(0.0)); + + // Every value the diff showed IDENTICAL must be unchanged from Flash. + CHECK(p.head_dim == 512); + CHECK(p.qk_rope_head_dim == 64); + CHECK(p.o_lora_rank == 1024); + CHECK(p.num_experts_per_tok == 6); + CHECK(p.n_shared_experts == 1); + CHECK(p.num_hash_layers == 3); + CHECK(p.sliding_window == 128); + CHECK(p.hc_mult == 4); + CHECK(p.hc_sinkhorn_iters == 20); + CHECK(p.index_head_dim == 128); + CHECK(p.index_n_heads == 64); + CHECK(p.scoring_func == "sqrtsoftplus"); + CHECK(p.topk_method == "noaux_tc"); + CHECK(p.expert_dtype == "fp4"); + CHECK(p.vocab_size == 129280); +} + +TEST_CASE("deepseek-v4-pro: derived MLA/LoRA geometry scales off the config") { + const DeepseekV4Params pro = ParseDeepseekV4Params(ProConfig()); + const DeepseekV4Params flash = ParseDeepseekV4Params(FlashConfig()); + + // NoPE width is head_dim - rope on both (448 = 512 - 64), the geometry the + // `head_dim == 512` scope assertion pins. + CHECK(pro.head_dim - pro.qk_rope_head_dim == 448); + CHECK(flash.head_dim - flash.qk_rope_head_dim == 448); + + // Grouped OUTPUT LoRA: heads-per-group is 8 on BOTH (64/8 and 128/16), so the + // wo_a bmm batch shape is invariant across the two checkpoints even though + // both operands doubled. + CHECK(pro.num_attention_heads / pro.o_groups == 8); + CHECK(flash.num_attention_heads / flash.o_groups == 8); + + // The wo_b input width (`zdim` in the forward) scales with o_groups only. + CHECK(pro.o_groups * pro.o_lora_rank == 16384); + CHECK(flash.o_groups * flash.o_lora_rank == 8192); +} + +TEST_CASE("deepseek-v4-pro: compress_ratios reproduce BOTH real checkpoints") { + const DsaLayerCounts pro = CountDsaLayers(ParseDeepseekV4Params(ProConfig())); + const DsaLayerCounts flash = CountDsaLayers(ParseDeepseekV4Params(FlashConfig())); + + // Counted from the real model.safetensors.index.json of each checkpoint by + // grouping compressor/indexer tensors per layer (see the header note). + CHECK(pro.compressor == 61); + CHECK(pro.indexer == 30); + CHECK(flash.compressor == 41); + CHECK(flash.indexer == 21); + + // Pro has MORE compressor layers than it has non-MTP layers in Flash, and the + // two counts must not be equal — a Flash-hardcoded layer count would collapse + // these into the same number. + CHECK(pro.compressor != flash.compressor); + CHECK(pro.indexer != flash.indexer); +} + +TEST_CASE("deepseek-v4-pro: the layer-0/1 compressor delta is the ONE structural diff") { + const DeepseekV4Params pro = ParseDeepseekV4Params(ProConfig()); + const DeepseekV4Params flash = ParseDeepseekV4Params(FlashConfig()); + + // Pro's compress_ratios start [128, 128, ...] where Flash starts [0, 0, ...]. + // Pro therefore gains a ratio-128 compressor on layers 0 and 1 (confirmed in + // the real Pro checkpoint: 4 compressor tensors on each of those layers, where + // Flash has none). Neither is an indexer layer, since indexer == ratio 4. + for (int64_t l = 0; l < 2; ++l) { + CHECK(pro.compress_ratio(l) == 128); + CHECK(pro.has_compressor(l)); + CHECK_FALSE(pro.has_indexer(l)); + + CHECK(flash.compress_ratio(l) == 0); + CHECK_FALSE(flash.has_compressor(l)); + CHECK_FALSE(flash.has_indexer(l)); + } + + // From layer 2 on, both alternate 4/128 identically, so the compressor and + // indexer roles agree layer-for-layer over Flash's whole depth. + for (int64_t l = 2; l < flash.num_hidden_layers; ++l) { + CHECK(pro.compress_ratio(l) == flash.compress_ratio(l)); + CHECK(pro.has_indexer(l) == flash.has_indexer(l)); + } + + // Only ratios 0, 4 and 128 ever appear — upstream's compressor asserts + // `compress_ratio in [4, 128]` (compressor.py:171) after mapping 0 to "absent", + // so any other value would be unrepresentable on both sides. + for (int64_t l = 0; l < pro.num_hidden_layers; ++l) { + const int64_t r = pro.compress_ratio(l); + CHECK((r == 0 || r == 4 || r == 128)); + } +} + +TEST_CASE("deepseek-v4-pro: the trailing MTP compress_ratios entry is excluded") { + const DeepseekV4Params pro = ParseDeepseekV4Params(ProConfig()); + const DeepseekV4Params flash = ParseDeepseekV4Params(FlashConfig()); + + // Both arrays are num_hidden_layers + 1 long; the extra entry is the nextn/MTP + // layer and is 0. Upstream guards the same off-by-one with an explicit + // `if layer_id < config.num_hidden_layers` (attention.py:208-211). + REQUIRE(static_cast(pro.compress_ratios.size()) == pro.num_hidden_layers + 1); + REQUIRE(static_cast(flash.compress_ratios.size()) == + flash.num_hidden_layers + 1); + CHECK(pro.compress_ratios.back() == 0); + CHECK(flash.compress_ratios.back() == 0); + + // num_nextn_predict_layers == 1 on both, which is what that entry accounts for. + CHECK(pro.num_nextn_predict_layers == 1); + CHECK(flash.num_nextn_predict_layers == 1); +}