Fix llama-server router mode removes "UD-" prefix from auto-discovere…

[adityabagchi24]

Overview

Fix an issue in llama-server router mode where auto-discovered Unsloth models lose the UD- prefix in their generated model IDs.

Previously, a model downloaded with:

llama-server -hf unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL

would be auto-discovered in router mode as:

unsloth/gemma-4-12B-it-qat-GGUF:Q4_K_XL

instead of the expected:

unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL

This caused the generated model ID, alias, and hf-repo value to differ from the original repository identifier even though the GGUF filename correctly contained the UD- prefix.

The fix preserves the full quantization suffix when constructing model identifiers for auto-discovered Hugging Face models, ensuring Unsloth models retain their correct names.

Fixes #24690.

Additional information

The issue was reproducible with Unsloth GGUF models using quantization variants such as UD-Q4_K_XL. Router mode incorrectly stripped the UD- portion during model discovery, resulting in mismatched model IDs and launch arguments.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES – OpenCode was used to assist with code development and implementation. ChatGPT was used to help draft the PR description. All code changes, testing, verification, and final submission were reviewed by me.

[angt]

This is not a bug, the real tag is Q4_K_XL, there is no - allowed in the GGUF format convention.
The code is made to allow that usage, as many people use it.

[OPS-NeoRetro]

This is not a bug, the real tag is Q4_K_XL, there is no - allowed in the GGUF format convention. The code is made to allow that usage, as many people use it.

@angt how? Unsloth uses UD- prefixes to mark files quantized with their quantization recipe. Imagine that one day, Unsloth makes a model with both UD and non-UD quants for the same quant type, like Qwen4-27B-Thinking-it-GGUF:UD-Q4_K_M and Qwen4-27B-Thinking-it-GGUF:Q4_K_M. If llama-server doesn't list UD quants in model API queries, it becomes messy once Unsloth does something like that.

[angt]

That's why they use _XL to not make the situation you described.

[itn3rd77]

@angt Thank you for the explanation. But with that implementation in place I will always have two entries for the same model in the API response /v1/models. As it is impossible to have the name/id matching from the cache and in the preset. Is that true or have I missed something?

Example

[unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL]

; load model on startup
load-on-startup = true

; model
hf = unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL

; reasoning/thinking 
reasoning = off

; fine tuning model
cache-ram = 0
ctx-size = 896
parallel = 1
top-k = 64
top-p = 0.95

spec-type = ngram-mod,ngram-map-k4v

spec-ngram-mod-n-match = 12
spec-ngram-mod-n-min = 24
spec-ngram-mod-n-max = 48

spec-ngram-map-k4v-size-n = 8
spec-ngram-map-k4v-size-m = 16
spec-ngram-map-k4v-min-hits = 2

This results in a two models from a API call to /v1/models:

1. unsloth/gemma-4-E4B-it-qat-GGUF:Q4_K_XL (UD- removed)
2. unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL

From an end user perspective this does not help two have two models displayed in a frontend.

[angt]

I guess we need to fix the places where UD- is kept in the tag

[adityabagchi24]

@angt please check this commit once as this did fix it

[angt]

UD- should not be considered as part of the tag (UD_XXX could, by the way), from the llama.cpp format spec I’m aware of, so presets, tools, etc., should align to that... if we start hardcoding special rules for UD-, it's going to be a mess. And Q4_K_XL works perfectly fine.

[brd-ai]

@itn3rd77 stripping UD- from your model key should already solve it, since you are specifying the hf repo anyway

[unsloth/gemma-4-E4B-it-qat-GGUF:Q4_K_XL]

...

; model
hf = unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL

...