Muse Glimmer GGUF k-quant generates INCOHERENT text (" is is is ...") where llama.cpp on the same file is coherent

[localai-bot]

Found while fixing #347 (the tokenizer.ggml.pre = "llama4" gap); filed
separately per AGENTS.md rather than folded into that change.

What happens

With #347's tokenizer fix in place the 16.76 GB Muse Glimmer k-quant now runs
end to end and emits tokens, but they are degenerate:

build-cpu/examples/vllm-cli \
  --model /mnt/nas_share/checkpoints/muse-glimmer-30b-gguf/muse-glimmer-30B-kquant-17gb.gguf \
  --prompt "The capital of France is" --max-tokens 12 --temperature 0 --device cpu

vllm-cli: run=1/1 finish_reason=length prompt_tokens=5 completion_tokens=12
 is is is is is is is is is is is is

It is not the file, the tokenizer, the load, or the backend

Isolated four ways, all on the same CPU-only box (20 cores, 84 GB, no GPU),
same GGUF (muse-glimmer-30B-kquant-17gb.gguf, 16,756,681,056 bytes, rev
2fb01e4e6f):

• the file is fine — llama.cpp master 153d324bcf, same prompt, --temp 0:

  llama-completion -m muse-glimmer-30B-kquant-17gb.gguf \
      -p "The capital of France is" -n 12 --temp 0 -no-cnv -t 8
  The capital of France is Paris. It is the most populous city in France and
  

• the tokenizer is fine — prompt_tokens=5 is the correct GPT-4o
  tokenization of that prompt, and examples/tokenize on this GGUF matches HF
  tokenizers on 57/57 entries of
  tests/parity/goldens/tokenizer_muse_glimmer/corpus.txt (zero diff).

• the load is fine — test_muse_glimmer_gguf is 12/12 (428 assertions) with
  VLLM_MUSE_GGUF, and 12/12 (1064 assertions) with VLLM_MUSE_GGUF_LOAD, i.e.
  with the whole 16.76 GB materialised.

• the CPU backend is fine in general — vllm-cli on opt-125m-bf16-st, same
  box and same build, generates " the capital of the French Republic.".

So the defect is in the Muse Glimmer forward over GGUF weights — numerics or
wiring — and not upstream of it.

Why it matters

.agents/specs/muse-glimmer.md §10.6 recorded "no forward was run on GGUF
weights, so there is no e2e". That is now "a forward runs and is wrong", which
is a different and newly actionable claim. #333's quant-matched llama.cpp
comparison is still blocked, for this reason instead of the tokenizer one: the
GGUF is the only artifact both engines can hold.

Suggested first cuts

• Compare our bf16 safetensors arm on the same prompt (does the model impl
  generate coherently at all, at full depth?). §10.6 gated text at reduced depth
  4/52 only, so full-depth text has never been checked in either format.
• Per-tensor A/B of the k-quant dequant against llama.cpp's own dequant for the
  sandwich norms (spec §10.2 stores them PRE-OFFSET) and the gate_up_proj
  keep-quant block concat (§10.2 note on Q4_K superblock concatenation).
• First-step logits against llama.cpp's on the same 5 prompt ids.

No speed claim is made anywhere above; every number is a correctness
observation on a CPU-only box.