Skip to content

ui: model status and load progress via /models/sse feed - #24878

Merged
ServeurpersoCom merged 3 commits into
ggml-org:masterfrom
ServeurpersoCom:ui/loading-progress
Jun 22, 2026
Merged

ui: model status and load progress via /models/sse feed#24878
ServeurpersoCom merged 3 commits into
ggml-org:masterfrom
ServeurpersoCom:ui/loading-progress

Conversation

@ServeurpersoCom

@ServeurpersoCom ServeurpersoCom commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Overview

When you pick a model that is not loaded yet, you now see what is happening instead of a silent wait: the model row shows a live status and a progress bar as it loads, and if you just send a message to a cold model, the same progress shows right in the chat while it warms up. It updates itself in real time, so the interface always reflects the real state of your models.

Additional information

Video

LoadingProgress.mp4

Implementation

Replace the post-operation status polling with a single persistent /models/sse feed that becomes the source of truth for model state in router mode. The feed drives live row status, staged load progress and load/unload completion, surfaced both in the model selector and inline in chat while a cold model warms up. The stage labels and the tail share that each phase owns are centralized as constants, so the per-stage reports unify into one global percentage: text_model fills the bar while each later phase, spec then mmproj, takes a configurable trailing slice (set to 10%), tune the share and the names in one place. It is one reconnecting reader with one awaiter per model, no per-operation streams. Dispatch is event-kind based so it extends cleanly: the upcoming dedicated download process slots in by adding a downloading status and download_* events on the same feed, with its own byte-based progress alongside the stage-based load progress, no rework of the backbone.

Built on the server-side feed work by @ngxson : real-time load progress via /models/sse (#24828), mtmd load progress callback (#24865) and the spec model fix with the stages list (#24870); the dedicated download process (#24834) plugs into the same feed.

Ref: #24822

Requirements

@ServeurpersoCom

Copy link
Copy Markdown
Contributor Author

We can configure the displayed text and the percentage for draft/mmproj inside model-loading.ts :

/**
 * Labels shown while a model loads, keyed by the stage reported on /models/sse.
 */
export const MODEL_LOAD_STAGE_LABELS: Record<ApiModelLoadStage, string> = {
        text_model: 'Loading weights',
        spec_model: 'Loading draft',
        mmproj_model: 'Loading projector'
};

/**
 * Share of the bar reserved for each load phase after text_model.
 * text_model fills the rest, so a plain model reaches 100% on its own.
 */
export const MODEL_LOAD_TAIL_SHARE = 0.1;

And it will be easy to make it work for downloading and single model mode when the backend does.

Comment thread tools/ui/src/lib/stores/models.svelte.ts Outdated

@allozaur allozaur left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonderful work. One architectural nitpick and we good to go @ServeurpersoCom

Comment thread tools/ui/src/lib/stores/models.svelte.ts Outdated
@ServeurpersoCom
ServeurpersoCom merged commit 099b579 into ggml-org:master Jun 22, 2026
6 checks passed
papamoose pushed a commit to papamoose/llama.cpp that referenced this pull request Jun 27, 2026
* ui: model status and load progress via /models/sse feed

* ui: centralize SSE wire-format delimiters into shared constants for the chat and /models/sse parsers

* ui: type /models/sse event names as a ServerModelsSseEventType enum

Address review from allozaur
adrianhoehne pushed a commit to adrianhoehne/llama.cpp that referenced this pull request Jul 5, 2026
* ui: model status and load progress via /models/sse feed

* ui: centralize SSE wire-format delimiters into shared constants for the chat and /models/sse parsers

* ui: type /models/sse event names as a ServerModelsSseEventType enum

Address review from allozaur
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants