common: refactor model handling

[ngxson]

Overview

Refactor model handling (-m, -hf, etc) into a centralized common_models_handler object

It exposes 2 main calls:

• fetch_meta(): check if the input option like -hf or -dr is valid; in case of HF repo: get the list of files
• apply(): download and apply the model local path to common_params

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: most of the code is human-written, AI only used for validation

[angt]

This is not related to hf-cache.h to me, but to the selection mechanism and model management

[ngxson]

hmm ok I moved it back to download.h in df822d2

[ggerganov]

Suggested change

	struct hf_plan {
	hf_cache::hf_file primary;
	hf_cache::hf_files model_files;
	hf_cache::hf_file mmproj;
	hf_cache::hf_file mtp;
	hf_cache::hf_file preset; // if set, only this file is downloaded
	};
	hf_plan get_hf_plan(const common_params_model & model, const common_download_opts & opts);
	struct common_download_hf_plan {
	hf_cache::hf_file primary;
	hf_cache::hf_files model_files;
	hf_cache::hf_file mmproj;
	hf_cache::hf_file mtp;
	hf_cache::hf_file preset; // if set, only this file is downloaded
	};
	common_download_hf_plan common_download_get_hf_plan(const common_params_model & model, const common_download_opts & opts);

[ggerganov]

Not needed

[ggerganov]

This struct seems a bit unnecessary - likely can remain a function. The private methods can be static functions in the cpp file.

[ngxson]

make sense, I refactored it again:

struct common_models_handler {
    common_download_hf_plan plan;
    common_download_opts opts;
};

common_models_handler common_models_handler_init(const common_params & params, llama_example curr_ex);
bool common_models_handler_is_preset_repo(const common_models_handler & handler);
void common_models_handler_apply(common_models_handler & handler, common_params & params, common_download_callback * callback = nullptr);

Usage:

• common_models_handler_init will fetch metadata and construct the hf_plan / opts
• common_models_handler_is_preset_repo is to check if the hf-repo is preset.ini-only
• common_models_handler_apply will download the files and modify params to point to the downloaded location

[ggerganov]

Since all the usages are like:

{
    auto handler = common_models_handler_init(...);
    if (need_apply) {
        common_models_handler_apply(handler);
    }
}

It does not seem necessary to introduce the common_models_handler. A single function such as the following seems enough?

void common_params_update(
        common_params & params,
        llama_example ex,
        bool apply,
        common_download_callback * callback);

[ngxson]

Your idea is similar to the existing common_params_handle_models before this refactoring, but the main point of the 2-step init/apply is for server code, it looks somewhat like this:

models_handler = common_models_handler_init(params, LLAMA_EXAMPLE_SERVER);
if (common_models_handler_is_preset_repo(models_handler)) {
    // apply the preset and start the server in router mode
    common_models_handler_apply(models_handler, params);
}

// do other endpoint setup
ctx_http.get ("/models/sse", progress_download_report);
// ... then later on

if (not_preset) {
    // non-roter + non-preset: download model, with progress callback via `progress_download_report`
    common_models_handler_apply(models_handler, params, callback);
}

We could also get rid of common_models_handler but that will require re-fetching the list of files for each call. My idea is somewhat like this:

• The list of files to be downloaded will be stored in common_models_handler
• It can also be useful for future, for example, if we want to show to the user list of files (+ size), then ask for confirmation before downloading

[ngxson]

Also, note that if we only call common_models_handler_init without _apply, that means "validate if the HF model exist", and indeed the logic is being used in POST /models endpoint is now benefit from this

[ggerganov]

After this change the following config no longer loads:

https://huggingface.co/ggerganov/preset-128gb/blob/main/preset.ini#L122-L137

This is the error:

[52799] 0.03.758.552 I srv    load_model: loading draft model 'mtp-Qwen3.6-27B-Q4_0.gguf'
[52799] 0.03.758.602 E gguf_init_from_file: failed to open GGUF file 'mtp-Qwen3.6-27B-Q4_0.gguf' (No such file or directory)
[52799] 0.03.758.640 E llama_model_load: error loading model: llama_model_loader: failed to load model from mtp-Qwen3.6-27B-Q4_0.gguf
[52799] 0.03.758.645 E llama_model_load_from_file_impl: failed to load model
[52799] 0.03.758.645 E srv    load_model: failed to load draft model, 'mtp-Qwen3.6-27B-Q4_0.gguf'
[52799] 0.03.758.651 I srv    operator(): operator(): cleaning up before exit...
[52799] 0.03.759.154 E srv  llama_server: exiting due to model loading error
[52799] 0.03.759.471 I ggml_metal_free: deallocating
0.09.008.129 I srv    operator(): instance name=qwen3.6-27b-hf-think-mtp exited with status 1

Before the change it loads the draft model like this:

[53064] 0.04.864.512 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[53064] 0.05.063.933 I srv    load_model: loading draft model '~/.cache/huggingface/hub/models--ggml-org--Qwen3.6-27B-GGUF/snapshots/4194c3ffc51e8c43a5d438470db134921e4cb577/mtp-Qwen3.6-27B-Q4_0.gguf'

[adrianhoehne]

After this change, this config broke. I start the server generally with the --offline argument.

2927] 0.00.331.823 I srv    load_model: loading model 'unsloth/Qwen3-Coder-Next-GGUF'
[52927] 0.00.396.249 E llama_model_load: error loading model: illegal split file idx: 2 (file: /home/adrian/.cache/huggingface/hub/models--unsloth--Qwen3-Coder-Next-GGUF/snapshots/ce09c67b53bc8739eef83fe67b2f5d293c270632/UD-Q6_K/Qwen3-Coder-Next-UD-Q6_K-00003-of-00003.gguf), model must be loaded with the first split
[52927] 0.00.396.270 E llama_model_load_from_file_impl: failed to load model

Config:

[qwen3-coder-next-q6k-rtx-hot-autolearn]
hf-repo = unsloth/Qwen3-Coder-Next-GGUF
hf-file = UD-Q6_K/Qwen3-Coder-Next-UD-Q6_K-00001-of-00003.gguf
device = CUDA0
main-gpu = 0
temp = 0.2
reasoning = off
tags = qwen3next,coder-next,hot-cache,primary-rtx,autolearn

cpu-moe = true
no-mmproj-offload = true