Skip to content

Prebuilt: pin the IQ1_XS, IQ1_XXS and IQ1_XXXS quant types - #92

Merged
danielhanchen merged 1 commit into
prset-drop-merged-26841from
prset-pin-iq1-narrow
Aug 11, 2026
Merged

Prebuilt: pin the IQ1_XS, IQ1_XXS and IQ1_XXXS quant types#92
danielhanchen merged 1 commit into
prset-drop-merged-26841from
prset-pin-iq1-narrow

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Adds the three sub-IQ1_S quantization types to the nightly mix: IQ1_XS at 1.4375 bpw, IQ1_XXS at 1.3125 and IQ1_XXXS at 1.1875.

Stacked on #90, so it targets that branch rather than master. #90 drops the merged 26841 pin, which currently fails the merge before anything after it is reached. Merge #90 first and this retargets to master on its own.

Pinning #91, not #61

The code is #61. The pin points at #91 because #61 branches off master past the squash sync 53fd974b4, and master shares history with upstream only up to 2026-06-10. resolve merges pins onto the upstream base tag, so #61 arrives 765 commits stale and conflicts across tools/ui/, vendor/cpp-httplib/ and vendor/sheredom/, files it never touches. #91 is the same commit on a b10356 base, the same arrangement #70 uses. #61 stays open for design review.

Resolve replay on b10356

including ggml-org/llama.cpp#24423 @ daca8075d871483545dd85d58ce11970b304b541
including ggml-org/llama.cpp#25731 @ 0a9841fa63edce1cd252ed6efea713715abb0cf2
including unslothai/llama.cpp#70 @ 06d2326acbf515b10f8d6abeada09123d361acde
including unslothai/llama.cpp#91 @ c86ed269986f2dced6325c5c58bda966a2e2ead1
warning: unslothai/llama.cpp#70 needed an additive merge; every conflict was a pure add/add
MERGED OK
  tag          b10356-mix-1c79028
  ggml tree    92d6a3e7335fd106486f466ab50dbdbd75d09971 (version 0.19.0)
  build number 10356

Only #70 needs additive_merge.py, and only for pure add/add hunks, same as today. The cmake/build-info.cmake stamping step still finds its three anchor lines in b10356.

Build and tests

Full mix built with CUDA sm_100 on a B200. The tree resolve produced hashes to cdc8f9490e4e33bc0dc87c94842455f6a8e5e189, identical to the tree these numbers come from.

check result
build completes, no errors
ctest 59 of 60; the 60th is test-backend-ops hitting the 1500s ctest timeout, run separately below
test-backend-ops -o MUL_MAT -b CUDA0 1219/1219, 2/2 backends
test-backend-ops -o MUL_MAT_ID -b CUDA0 874/874, 2/2 backends
quantize and generate, all three new types works on Llama-3.2-1B with an imatrix

Sizes come out in the order the bpw table predicts: IQ1_S 393551840, IQ1_XS 380182496, IQ1_XXS 366813152, IQ1_XXXS 353443808.

Nothing else in the mix regresses

Built the same tree without the new commit and requantized from the same imatrix with both binaries. Byte-identical output:

IQ1_S    IDENTICAL  02dc2c2c1a63817dfb3998acc272923d8f68ed1ac10d06dc00e87b137c2c4482
IQ2_XXS  IDENTICAL  3a461ee80e793e93ab28b742da8513a9bd2342c5270f403a9398a7871efd657c
Q4_K_M   IDENTICAL  574d6de2b69c35bcb539dd210adee181b8eba1b7ae8864f525ef20b6ef83c2b3
Q8_0     IDENTICAL  05085ca8b9697115aeb2be01ee32873d476dec5b060abaaf2367b394fae82392

DiffusionGemma, Inkling, Kimi-K3 and Muse Glimmer all still land. Every arch registers in llama-arch.cpp, test-llama-archs passes, the four llama-diffusion-* binaries link, and the libraries export what they should (libllama.so: diffusion_gemma 17, inkling 24, kimi_k3 23, muse_glimmer 17; libmtmd.so: kimivl 7, muse_glimmer 14). Muse Glimmer comes from the base tag now that its pin is gone, at upstream's final state.

Adds IQ1_XS at 1.4375 bpw, IQ1_XXS at 1.3125 and IQ1_XXXS at 1.1875 to the
nightly. Pins #91 rather than #61: same commit content, but on an upstream
base, so it merges onto the base tag instead of dragging two months of
squash-synced master in behind it.

Replayed resolve on b10356 with this set: all four pins merge, tag
b10356-mix-1c79028, and the resulting tree hash matches the tree that was
built and tested (cdc8f949).
@danielhanchen
danielhanchen merged commit c0ab496 into prset-drop-merged-26841 Aug 11, 2026
3 checks passed
danielhanchen added a commit that referenced this pull request Aug 11, 2026
#92 merged into its stacked base branch rather than master, so the pin
landed on prset-drop-merged-26841 and never reached the file the nightly
reads. Same one-line addition, on top of master.

#91 is merged now, into iq1-narrow-upstream-base rather than master, so the
tree resolve builds still does not contain it and the pin is still what puts
it in. A merged required pin keeps being mixed in, with a warning, which is
the intended path here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant