ggml-cuda : optimize mmvq nwarps for Pascal DP4A

[animeshsri14]

Overview

Adds a dedicated MMVQ parameter table for Pascal DP4A GPUs (compute capability 6.1 / 6.2 -GTX 10xx, P40, P4) and tunes the warp count for single-token decode.

Previously these cards fell through to MMVQ_PARAMETERS_GENERIC. This PR introduces MMVQ_PARAMETERS_PASCAL_DP4A, selected for NVIDIA archs in [DP4A, VOLTA) on both the device and host table-id paths, and for ncols_dst == 1 (single-token decode) uses 2 warps instead of 4. Pascal decode is memory-bandwidth-bound on these small SMs, so the extra warps add scheduling/occupancy pressure without helping throughput. Batched decode and prefill keep the generic behaviour and are unchanged.

Measured +2.5% to +5.4% single-token decode across quants (geomean +3.07%) on a GTX 1060 6GB, with no measurable prefill regression.

Additional information

Benchmarked rigorously, both coverage and depth with a self-contained script that builds HEAD~1 (baseline) and HEAD (this PR) into separate binaries and runs llama-bench alternating between them across 6 iterations. Consumer Pascal can't lock clocks (nvidia-smi -lgc), so I report the paired per-iteration delta, which cancels thermal/boost drift; t is the paired t-statistic of the per-iteration Δ%. GTX 1060 6GB (CC 6.1), Llama-3.2-3B-Instruct.

Decode (tg) - the mmvq path this PR changes (-p 0 -n 128 -ngl 99 -r 10)

quant	baseline t/s	patched t/s	mean Δ	paired σ%	t	pairs up
Q4_0	49.41	51.68	+4.59%	0.45%	25.00	6/6
Q4_K_M	43.49	45.85	+5.44%	0.38%	34.74	6/6
Q5_K_M	39.92	41.22	+3.24%	0.33%	23.80	6/6
Q8_0	32.62	33.43	+2.48%	0.53%	11.56	6/6
Q6_K	34.10	33.99	-0.32%	0.56%	-1.41	2/6

Geomean +3.07%. Q6_K is within noise. An earlier independent 10-iteration run reproduced these to within ~0.2% (geomean +3.08%).

Prefill (pp) - MMQ path, no-regression control (expected ~0%) - two prompt lengths and two -ub batch sizes; all rows flat:

quant	config	baseline t/s	patched t/s	mean Δ	paired σ%	t	pairs up
Q4_0	pp512	950.77	950.19	-0.06%	0.52%	-0.27	3/6
Q4_K_M	pp512	825.24	838.80	+1.64%	4.25%	1.03	3/6
Q5_K_M	pp512	812.26	811.70	-0.07%	0.19%	-0.88	2/6
Q8_0	pp512	873.82	874.03	+0.02%	0.20%	0.30	3/6
Q6_K	pp512	770.35	770.46	+0.01%	0.23%	0.15	4/6
Q4_0	pp512_ub128	946.72	946.95	+0.02%	0.18%	0.34	3/6
Q4_K_M	pp512_ub128	801.61	799.35	-0.28%	0.90%	-0.94	2/6
Q5_K_M	pp512_ub128	807.51	806.92	-0.07%	0.16%	-1.09	2/6
Q8_0	pp512_ub128	865.61	867.55	+0.22%	0.17%	3.26	5/6
Q6_K	pp512_ub128	765.39	765.17	-0.03%	0.08%	-0.83	3/6
Q4_0	pp2048	835.78	833.94	-0.22%	0.95%	-0.56	2/6
Q4_K_M	pp2048	748.45	748.56	+0.02%	0.61%	0.07	3/6
Q5_K_M	pp2048	725.78	725.93	+0.02%	0.54%	0.10	3/6
Q8_0	pp2048	773.93	773.75	-0.02%	0.71%	-0.07	3/6
Q6_K	pp2048	692.05	692.68	+0.09%	0.56%	0.41	4/6

(The Q4_K_M pp512 +1.64% is noise - σ=4.25%, t=1.03, and the same quant's other prefill configs are flat.)

Benchmark script (builds both commits itself; point MODELS/MODEL_DIR at your GGUFs and run ./bench-pascal-mmvq.sh): https://pastebin.com/he3WzqjG

Requirements

• I have read and agree with the contributing guidelines.
• AI usage disclosure: Yes, codex only helped write the benchmarking script, the entire diff itself is handwritten. The source change is my own, from conception to verification and benchmarking methodology.

[ggml-gh-bot]

Hi @animeshsri14, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[JohannesGaessler]

Why is this PR still a draft? Do you intend to add more changes?

[animeshsri14]

@JohannesGaessler Finalized, just wanted to strengthen the benchmark section for easier review.

[animeshsri14]

@JohannesGaessler This is ready for review now benchmarks are in the description showing the mmvq nwarps gains on Pascal DP4A. Let me know if you'd like any additional data or changes. Thankyou.

[animeshsri14]

Drafting this for now not abandoning it, but I'd rather put my single open-PR slot toward a higher-impact change (fattn-tile config tuning, which benefits all NVIDIA architectures rather than Pascal only). Branch stays up; happy to proceed or resubmit this once that lands.