ggml-backend-meta: fix device and buffer type names - #25688
Open
Frozenlock wants to merge 1 commit into
Open
Conversation
|
Hi @Frozenlock, thanks for your contribution! Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:
Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below. |
The constructors iterated the moved-from constructor parameter instead of the member of the same name, so every meta device and buffer type was named "Meta()". With a single meta device this is cosmetic, but code that distinguishes buffer types by name (e.g. the KV cache context map) breaks as soon as more than one meta device exists.
Frozenlock
force-pushed
the
fix-meta-naming
branch
from
July 15, 2026 09:13
368466a to
6939317
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Fixes the names of meta devices and meta buffer types: the constructors iterate the moved-from constructor parameter instead of the member of the same name, so every meta device and buffer type is named "Meta()".
With a single meta device this is cosmetic (the current -sm tensor path), but any code that distinguishes buffer types by name breaks as soon as more than one meta device exists — e.g. the KV cache context map in llama-kv-cache.cpp collapses the buffer types of distinct meta devices into one entry, allocating all layers' caches on the first device.
Verified: with two meta devices, names render as
Meta(CUDA0,CUDA1)/Meta(CUDA2,CUDA3)and the per-layer buffer-type maps no longer collapse.Additional information
Found while experimenting with tensor parallelism for MLA architectures on a 12x RTX 3090 rig (multiple meta devices in one process). Related follow-up work will be proposed separately.
Requirements
[1] I am NOT a cpp programmer, so I'm not in a position to evaluate the quality as I would with my main programming languages. That's my way on contributing back to this project. Given the contributing guidelines, I'll understand if you don't accept the PR.
Edit:
Results from the follow-up work (for context)
On 12x RTX 3090 (PCIe Gen4), GLM-5.2 744B-A40B at UD-IQ2_M:
-sm layer(baseline)"spec" =
--spec-type ngram-map-k4v,draft-mtp(the GLM MTP head, ported from #24868 toGLM-DSA). TP output verified byte-identical to
-sm layeron GLM-4.7-Flash before anyspeculative decoding was added.
Branches:
mla-tensor-parallel
hybrid-tp-pp
mla-mtp