Skip to content

perf(ds4): optimize q5 fused verifier on AMD - #569

Merged
davide221 merged 12 commits into
mainfrom
codex/ds4-q5-verifier
Aug 13, 2026
Merged

perf(ds4): optimize q5 fused verifier on AMD#569
davide221 merged 12 commits into
mainfrom
codex/ds4-q5-verifier

Conversation

@davide221

@davide221 davide221 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add the AMD q=5 fused DSpark verifier path
  • support exact partial-rejection rollback across both compressor boundaries
  • keep five-row dense projections on MMVQ and use the qualified gfx1201 x4+1 ROCmFP4 path
  • add deterministic context-sweep tooling and unit coverage
  • pin the JSON fallback to an immutable source archive so builds avoid the flaky release-asset route

This is the first PR in the heterogeneous DeepSeek 4 performance stack.

Validation

Tested on an R9700 target with Strix Halo expert offload, four routed experts, q=5, and deterministic 128-token completions.

Context Previous median Fused q=5 median
2K 64.212 tok/s 67.957 tok/s
4K 62.428 tok/s 65.927 tok/s
8K 59.267 tok/s 62.544 tok/s
16K 53.750 tok/s 56.335 tok/s
2K after 16K 63.804 tok/s 67.458 tok/s

All 25 measured responses matched the expected SHA-256. The DeepSeek 4 GPU unit suite passed.

Reproduction

Run harness/qualification/deepseek4/qualify_ds4_q5_amd.sh with TARGET_MODEL, DRAFT_MODEL, and HOTNESS_CSV. The qualifier records configuration, response hashes, timings, ROCm state, and memory use.

@davide221 davide221 changed the title perf(ds4): prototype q5 fused verifier on AMD perf(ds4): optimize q5 fused verifier on AMD Aug 1, 2026
@davide221
davide221 force-pushed the codex/ds4-16k-amd-opt branch from be20268 to 0208630 Compare August 3, 2026 21:26
@davide221
davide221 force-pushed the codex/ds4-q5-verifier branch from 5417326 to e24ad65 Compare August 4, 2026 10:17
@davide221
davide221 changed the base branch from codex/ds4-16k-amd-opt to main August 4, 2026 10:17
@davide221
davide221 force-pushed the codex/ds4-q5-verifier branch from e24ad65 to 4ce3b82 Compare August 12, 2026 15:01
@davide221
davide221 marked this pull request as ready for review August 12, 2026 15:03

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread harness/qualification/deepseek4/qualify_ds4_q5_amd.sh
Comment thread server/deps/llama.cpp/ggml/src/ggml-cuda/fattn.cu
Comment thread server/src/deepseek4/deepseek4_fused_verify.inc
Comment thread server/src/deepseek4/deepseek4_backend.cpp Outdated
Comment thread server/deps/llama.cpp/ggml/src/ggml.c
Comment thread server/deps/llama.cpp/ggml/src/ggml-cuda/top-k.cu Outdated
Comment thread harness/benchmarks/deepseek4/ds4_publication_decode_client.py
Comment thread server/scripts/ds4_publication_decode_client.py Outdated
Comment thread server/scripts/ds4_publication_decode_client.py Outdated
Comment thread server/deps/llama.cpp/ggml/src/ggml-cuda/fattn.cu

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread harness/benchmarks/deepseek4/ds4_publication_decode_client.py
Comment thread server/src/deepseek4/deepseek4_backend.cpp Outdated
Comment thread harness/qualification/deepseek4/qualify_ds4_q5_amd.sh
Comment thread server/scripts/qualify_ds4_q5_amd.sh Outdated
Comment thread server/scripts/qualify_ds4_q5_amd.sh Outdated
Comment thread server/src/deepseek4/deepseek4_backend.cpp
Comment thread server/deps/llama.cpp/ggml/src/ggml-cuda/ds4-indexer.cu Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread server/scripts/qualify_ds4_q5_amd.sh Outdated
@davide221
davide221 force-pushed the codex/ds4-q5-verifier branch from b5d3515 to 3244b9c Compare August 12, 2026 19:39
@davide221
davide221 merged commit ab42351 into main Aug 13, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant