Bug Report: kvarn6 KV-quantization causes massive performance regression on ROCm (RX 7900 XTX)

[raufaser]

Issue: Using KV-quantization kvarn6 results in extremely slow performance compared to q8_0 on AMD ROCm hardware.

Performance Comparison

Both tests used the same prompt (10,410 tokens) and identical settings except for KV quantization.

Note: The kvarn6 run was canceled early due to excessive runtime.

Metric	q8_0	kvarn6	Regression
Prompt Processing	821.19 tokens/sec	53.79 tokens/sec	~15.3x slower
Text Generation	29.52 tokens/sec	1.34 tokens/sec	~22x slower

This is a severe performance regression that makes kvarn6 practically unusable.

Steps to Reproduce

1. Use llama.cpp build 11192 (ee8754c57) with ROCm backend on Windows
2. Load a large model (tested with Qwen3.6-27B-Fable-Fusion-Q4_K_M)
3. Run with full offload (-ngl 99) and flash attention (-fa 1)
4. Compare these two configurations:

q8_0 (baseline):

llama-server.exe -m <model> -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0 -c 40000

kvarn6 (regression):

llama-server.exe -m <model> -ngl 99 -fa 1 -ctk kvarn6 -ctv kvarn6 -c 40000

System Information

• GPU: AMD Radeon RX 7900 XTX (24 GB VRAM)
• CPU: AMD Ryzen 9 7900X 12-Core
• RAM: 64 GB
• OS: Windows
• Backend: ROCm (HIP), NO_VMM = 1
• Build: 11192 (ee8754c) with Clang 20.1.8 for Windows x86_64
• Model: Qwen3.6-27B-Fable-Fusion (Q4_K_M, 65 layers, 27.32B params, 17.22 GiB)

Relevant Log Excerpts

q8_0 performance:

prompt processing, n_tokens = 10410, progress = 1.00, t = 11.93 s / 872.56 tokens per second
prompt eval time = 12681.67 ms / 10414 tokens (1.22 ms per token, 821.19 tokens per second)
eval time = 73468.79 ms / 2169 tokens (33.87 ms per token, 29.52 tokens per second)

kvarn6 performance (run canceled after ~5.5 minutes):

prompt processing, n_tokens = 10410, progress = 1.00, t = 193.54 s / 53.79 tokens per second
n_decoded = 144, tg = 1.34 t/s

Expected Behavior

kvarn6 should provide competitive or better performance than q8_0 while using less VRAM, or at least not be an order of magnitude slower.

Actual Behavior

kvarn6 is 15-22x slower than q8_0 on ROCm hardware, making it impractical for use.

Additional Notes

• This appears to be ROCm-specific; the kvarn6 implementation may not be optimized for AMD GPUs
• The issue affects both prompt processing and token generation equally severely
• The kvarn6 run was manually canceled due to excessive runtime; actual degradation may be even more severe over longer generations
• Full log level 5 logs for both q8_0 and kvarn6 runs are attached to this issue

log_kvarn6_lv5.txt
log_q8_0_lv5.txt