Skip to content

prism-v6: dspark Path A, converge on upstream dflash with log-SNR conditioning and drafter-own embeddings - #114

Open
bri-prism wants to merge 1 commit into
prism-v6from
prism-v6-dspark-dflash
Open

prism-v6: dspark Path A, converge on upstream dflash with log-SNR conditioning and drafter-own embeddings#114
bri-prism wants to merge 1 commit into
prism-v6from
prism-v6-dspark-dflash

Conversation

@bri-prism

@bri-prism bri-prism commented Aug 11, 2026

Copy link
Copy Markdown

What

Path A of the dspark re-port: instead of re-porting our standalone dspark arch, converge on upstream's merged dflash implementation and re-apply the two drafter-side deltas it lacks.

  1. GIDD log-SNR conditioning (LogSnrEmbed). GIDD-trained drafters ship a log_snr_embed module: a sinusoidal featurization of a per-position log-SNR value through a 2-layer SiLU MLP, added to the draft noise embedding before the backbone (anchor position of each block at max_log_snr, masked positions at min_log_snr). The feature matrix is a pure function of quantities known at graph-build time, so it is precomputed host-side and staged through a new llm_graph_input_dspark_logsnr input. New optional metadata dflash.log_snr_conditioning plus dflash.min_log_snr/max_log_snr (required and validated once the flag is set); new tensors log_snr_fc1/fc2.{weight,bias}, required when the flag is set so a broken conversion fails loudly instead of silently running unconditioned.
  2. Optional drafter-own token_embd/output. Loaded as TENSOR_NOT_REQUIRED; the decoder graph already prefers model-own tensors and only borrows the target's via ctx_other when absent. Carrying the drafter's own full-precision embeddings and head protects accept rate when the target is heavily quantized, and lets such drafters initialize without ctx_other during memory fitting.

Drafters without either feature convert and load exactly as before.

Why

Keeping a standalone arch means carrying a parallel implementation of the same math forever. Upstream's dflash is in-graph, multi-seq, and block-size-sweepable; measured head-to-heads showed our accept advantage came from these two drafter-side deltas, not the algorithm, so this re-applies just the deltas.

How verified

  • Builds clean; loader accepts a converted GIDD drafter (79 tensors including the 4 log_snr tensors and own embd/head) and rejects one whose metadata flag is set without the tensors
  • End-to-end llama-speculative-simple --spec-type draft-dspark run on Metal: graph builds, spec loop executes, coherent output
  • Accept-rate validation (llama-server probe, M5 Pro, GIDD drafter x binary Q1_0 target, greedy): 51.7% accept without conditioning (upstream-equivalent stripped drafter) vs 82.6% accept, mean len 3.43 with log-SNR conditioning active. An embd-less conditioned variant scores the identical 82.6%, so the accept gain is entirely log-SNR. Upstream master measures 41.6% with the stripped drafter on H100, bit-identical at the DSpark merge commit and current master: no upstream regression, and the base path in this branch is healthy (stripped drafter scores the same 51.7% here as on the prism-v6 tip).
  • Do NOT evaluate this with llama-speculative-simple: it never sets ctx_other on the draft context, so embd-less drafters segfault and embd-carrying drafters silently draft without target features (2-6% accept). Use llama-server or the framework path. Details on the fork status board.

Rebased onto prism-v6 tip. Remaining follow-ups tracked on the status board: gguf-py/HF-converter mapping for fresh checkpoints, window-fix fold-in, CUDA speedup numbers.

@khosravipasha
khosravipasha force-pushed the prism-v6-dspark-dflash branch from 0d0265c to ab81cb6 Compare August 11, 2026 01:39
@bri-prism
bri-prism marked this pull request as ready for review August 11, 2026 03:35
Two drafter-side deltas our fork's standalone dspark arch carried, re-applied
on the upstream dflash implementation (Path A of the dspark re-port):

1. GIDD log-SNR conditioning (LogSnrEmbed): some GIDD-trained drafters ship a
   log_snr_embed module, a sinusoidal featurization of a per-position log-SNR
   value through a 2-layer SiLU MLP, added to the draft noise embedding before
   the backbone. Anchor position of each block at max_log_snr, masked
   positions at min_log_snr; the feature matrix is a pure function of
   quantities known at graph-build time, so it is precomputed host-side and
   staged through a new llm_graph_input_dspark_logsnr input.
   New optional metadata: dflash.log_snr_conditioning (bool) plus
   dflash.min_log_snr/max_log_snr (required, validated finite and max > min
   once the flag is set); new tensors log_snr_fc1/fc2.{weight,bias}, REQUIRED
   when the flag is set so a broken conversion fails loudly instead of
   silently running unconditioned.

2. Optional drafter-own token_embd/output: loaded as TENSOR_NOT_REQUIRED; the
   decoder graph already prefers model-own tensors and only borrows the
   target's via ctx_other when absent. Carrying the drafter's own full-precision
   embeddings/head protects accept rate when the target is heavily quantized,
   and makes the drafter loadable without ctx_other during memory fitting.

Drafters without either feature convert and load exactly as before.
@bri-prism
bri-prism force-pushed the prism-v6-dspark-dflash branch from ab81cb6 to 2746240 Compare August 11, 2026 04:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant