Skip to content

perf(dflash): add opt-in exact prefill bands - #590

Draft
cheese-cakee wants to merge 10 commits into
Luce-Org:mainfrom
cheese-cakee:codex/perf-ds4-exact-prefill-aug10-v2
Draft

perf(dflash): add opt-in exact prefill bands#590
cheese-cakee wants to merge 10 commits into
Luce-Org:mainfrom
cheese-cakee:codex/perf-ds4-exact-prefill-aug10-v2

Conversation

@cheese-cakee

@cheese-cakee cheese-cakee commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

This remains a hardware-gated draft for the DeepSeek V4 exact-prefill objective. It does not claim the 1.5x target, production-model correctness, or merge readiness.

  • Adds default-off compressor-safe exact prefill bands at widths 2 through 4 via DFLASH_DS4_EXACT_PREFILL_BANDS=1.
  • Keeps exact attention tokenwise and preserves default/off, explicit --chunk 1, dense, sparse, decode, and speculative-verification behavior.
  • Separates execution-topology requirements from host vocabulary-logits readback, suppressing unused interior readbacks without exposing stale logits.
  • Preserves q1 raw-ring reduction order after SWA wrap and q1 hybrid-FFN numerical order for q4 exact bands.
  • Executes the temporary exact-band routing graph eagerly to avoid stale native replay while retaining persistent owner-graph replay.
  • Keeps final prompt, exact snapshot, and real external-consumer logits intact.

--chunk 1 remains the explicit escape hatch.

Current head

3e98b468fee82dbc441c18c2b6073db3f35eeae0

This is a real merge of the reviewed production remediation chain with upstream main at 4cf58a04c9f6b7c117a54c09ab1cb004d13175c9. It retains current-main ROCTX, indexer, sparse-decode, 0731, and shared fused-width behavior.

Only seven files differ from current main:

  • server/docs/DS4.md
  • server/docs/ENVIRONMENT.md
  • server/src/deepseek4/deepseek4_backend.cpp
  • server/src/deepseek4/deepseek4_fused_verify.inc
  • server/src/deepseek4/deepseek4_graph.cpp
  • server/src/deepseek4/deepseek4_internal.h
  • server/tests/test_deepseek4_unit.cpp

Diagnostic trace producers, comparator/package tooling, evidence files, and local research documents are not included.

Verification

Current-head local negative control:

  • CUDA 12.6, RTX 4050 (sm_89), Release, CUDA graphs enabled.
  • dflash_server and test_deepseek4_unit build successfully from exact offline Block-Sparse-Attention/CUTLASS identities.
  • test_deepseek4_unit exits 0, including output-intent, stale-logit, raw-ring, q4 FFN-order, nested graph-override, and graph-generation-guard regressions.
  • HIP-only tests and the two-GPU test are skipped; CUDA is not gfx1151 proof.
  • git diff upstream/main..HEAD --check passes.
  • Fresh independent final review found no actionable source issue after the current-main conflict resolution and fused-width max/max+1 correction.
  • GitHub reports the head mergeable; Cubic passes on the exact head.

Earlier Lucebox5 runs on the pre-merge remediation identities passed affected HIP/gfx1151 suites and bounded real-model comparators, and exposed/fixed several previously masked correctness defects. Those results are historical evidence for their exact binaries; they do not replace a fresh final-head production differential.

Performance

No qualifying performance result is available. No speedup or 1.5x claim is made.

The acceptance comparison remains the frozen matched four-cell run on Lucebox5:

  1. Heterogeneous baseline.
  2. Heterogeneous corrected candidate.
  3. Strix-only baseline.
  4. Strix-only corrected candidate.

The model, quantization, prompt, cache state, topology, clocks, power state, build inputs, correctness semantics, warmups, and sample order remain pinned. Timing stays closed until the fresh exact differential, loader smoke, official-source oracle, and correctness matrix pass.

Remaining gates

  • Fresh exact-head HIP/gfx1151 affected suites.
  • Bounded repeated reset-q4/layer-32/position-2879 comparator proof.
  • One fresh eight-cell exact differential through the unchanged comparator.
  • Six-build loader smoke and official-source oracle on both topologies.
  • Exact 12-case correctness matrix.
  • Frozen 20-launch/60-row timing run.
  • Exact-head CI and maintainer review.

The older ROCMFP2 model can establish only that model's correctness and speed delta; it does not prove 0731 support or performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant