Skip to content

cmake: use glob to collect src/models sources - #22005

Merged
ngxson merged 2 commits into
ggml-org:masterfrom
ngxson:xsn/cmake_model_glob
Apr 16, 2026
Merged

cmake: use glob to collect src/models sources#22005
ngxson merged 2 commits into
ggml-org:masterfrom
ngxson:xsn/cmake_model_glob

Conversation

@ngxson

@ngxson ngxson commented Apr 16, 2026

Copy link
Copy Markdown
Collaborator

Overview

The goal is to make #22004 a bit easier

Requirements

@ngxson
ngxson requested review from CISC and pwilkin April 16, 2026 16:25
@ngxson
ngxson requested a review from ggerganov as a code owner April 16, 2026 16:25
@ngxson ngxson mentioned this pull request Apr 16, 2026
Merged
6 tasks
@ngxson
ngxson merged commit 089dd41 into ggml-org:master Apr 16, 2026
11 of 50 checks passed
@ngladitz

ngladitz commented Apr 17, 2026

Copy link
Copy Markdown

Just saw this by chance. CMake explicitly discourages GLOB for picking up sources files (see the Note in the file(GLOB) documentation). If you feel you must you might want to at least add CONFIGURE_DEPENDS.

Explicit source file listings work like manifests that make builds more reproducible.
With GLOB in general it can pick up e.g. leftover files / temporary copies and CMake won't implicitly know when files are added or removed (CONFIGURE_DEPENDS helps with that at least).

cnsiva pushed a commit to saas-home/llama.cpp that referenced this pull request Apr 17, 2026
@ngxson @cnsiva
cmake: use glob to collect src/models sources (ggml-org#22005)
a663926
samuraieng pushed a commit to samuraieng/llama.cpp that referenced this pull request Apr 19, 2026
@ngxson @samuraieng
cmake: use glob to collect src/models sources (ggml-org#22005)
4d359cb
mengqin pushed a commit to mengqin/llama.cpp that referenced this pull request Apr 20, 2026
@ngxson @mengqin
cmake: use glob to collect src/models sources (ggml-org#22005)
4ce7ce3
ArberSephirotheca pushed a commit to ArberSephirotheca/llama.cpp that referenced this pull request Apr 21, 2026
@ngxson @ArberSephirotheca
cmake: use glob to collect src/models sources (ggml-org#22005)
f8c6f1f
TheTom added a commit to TheTom/llama-cpp-turboquant that referenced this pull request Apr 22, 2026
@TheTom @claude
Cherry-pick upstream CMake glob for model sources
b3401ea 
Cherry-picks ggml-org#22005 — replaces manual model file
listing with glob autodiscovery. New model source files are picked up
automatically without editing CMakeLists.txt.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TheTom added a commit to TheTom/llama-cpp-turboquant that referenced this pull request Apr 22, 2026
@TheTom @claude
Cherry-pick upstream CMake glob for model sources
f64e879 
Cherry-picks ggml-org#22005 — replaces manual model file
listing with glob autodiscovery. New model source files are picked up
automatically without editing CMakeLists.txt.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Apr 23, 2026
@ngxson @arthw
cmake: use glob to collect src/models sources (ggml-org#22005)
3a52fb8
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@ngxson
cmake: use glob to collect src/models sources (ggml-org#22005)
2342c4b
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ngxson
cmake: use glob to collect src/models sources (ggml-org#22005)
af11dde
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ngxson
cmake: use glob to collect src/models sources (ggml-org#22005)
1f155bb
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
@ngxson
cmake: use glob to collect src/models sources (ggml-org#22005)
860afec
MrLordCat referenced this pull request in MrLordCat/llama.cpp-with-GUI Jul 16, 2026
@ngxson
cmake: use glob to collect src/models sources (#22005)
c19dbb3
naamfung pushed a commit to naamfung/laamaafung that referenced this pull request Jul 20, 2026
@TheTom @claude
Cherry-pick upstream CMake glob for model sources
2adb2c5 
Cherry-picks ggml-org/llama.cpp#22005 — replaces manual model file
listing with glob autodiscovery. New model source files are picked up
automatically without editing CMakeLists.txt.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
naamfung pushed a commit to naamfung/laamaafung that referenced this pull request Jul 20, 2026
@TheTom @claude
Cherry-pick upstream CMake glob for model sources
c0d6976 
Cherry-picks ggml-org/llama.cpp#22005 — replaces manual model file
listing with glob autodiscovery. New model source files are picked up
automatically without editing CMakeLists.txt.

Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
kainlan added a commit to kainlan/llama.cpp-intel-optimizations that referenced this pull request Aug 1, 2026
@kainlan @claude
fix(build): four tracked src/*.cpp were absent from the library build
1fb188c 
llama-kv-block.cpp, llama-pp-scheduler.cpp, llama-moe-profile.cpp and
llama-tensor-class.cpp existed on disk and tracked in git, but were absent
from src/CMakeLists.txt's add_library(llama ...) list -- compiled into
nothing, so any symbol from them fails at LINK time in whatever test
happens to need it, naming the wrong culprit (llama.cpp-4hvq blamed test
rot; the module was simply not built).

Verdict, per file: all four are accidental-drop, not deliberate removal.

- llama-kv-block.cpp (added by eff6449, "llama: Add unified KV cache
  mode for continuous batching"), llama-tensor-class.cpp (added by
  1bb021a, "llama: add tensor-name classifier for placement priority"),
  and llama-pp-scheduler.cpp (added by 7c0a455, "SYCL: Add missing
  headers required for build") were all present in src/CMakeLists.txt
  until merge commit ff333a5 ("Merge remote-tracking branch
  'upstream/master' into feature/sycl-coalescing"). That merge pulled in
  upstream's 089dd41 ("cmake: use glob to collect src/models sources",
  ggml-org#22005), which replaced the explicit models/*.cpp list with
  file(GLOB LLAMA_MODELS_SOURCES "models/*.cpp"). Conflict resolution on
  src/CMakeLists.txt took the incoming block wholesale and silently
  dropped these three fork-local top-level entries along with it -- the
  glob's reach stops at models/*.cpp, so it never covered them. Verified
  by content-checking src/CMakeLists.txt at both parents of every merge
  on the first-parent chain between each file's addition and HEAD (git
  log's default path simplification hides merges from a plain -S
  pickaxe search over a pathspec, so this required bisecting the actual
  --first-parent chain and diffing each merge against both parents
  directly).

- llama-moe-profile.cpp (added by 71075d6, "src: Add missing
  llama-moe-profile files") is a different case: that commit added only
  the .cpp/.h pair and never touched src/CMakeLists.txt at all, so there
  is no removal commit to blame -- it was never wired into the build in
  the first place.

None of the four have any live external caller: nothing else under src/
or tools/ includes their headers, so restoring them to the build has no
blast radius beyond the modules themselves.

The strongest evidence for accidental-drop over deliberate-removal is
that llama-moe-profile.cpp no longer compiled once actually built:

    llama-moe-profile.cpp:301:26: error: reference to non-static member
    function must be called
        profile.init(hparams.n_layer, hparams.n_expert, ...)

llama_hparams::n_layer went from a plain field to a method (n_layer(),
"number of effective layers, excludes nextn layers") sometime after this
file was written in Dec 2025 -- unrelated upstream API drift that nobody
could have caught, because the module has not been part of any build for
roughly seven months. A module silently rotting outside the build while
its dependencies move on is a much stronger signal than a missing CMake
line alone: deliberately-retired code does not go on drifting against an
API nobody meant it to track. Fixed with a single call-site change
(hparams.n_layer -> hparams.n_layer()); no other n_layer use in the file
touches hparams, and no other line was changed.

Also fixes a masking side effect: tests/CMakeLists.txt's test-tensor-class
registration was compiling src/llama-tensor-class.cpp directly into the
test binary as a workaround for this exact gap. That workaround did double
harm -- it hid the missing-from-build defect from anyone reading that
test, and it becomes an ODR hazard the moment the module returns to
libllama (two definitions of llama_tensor_classify et al. reaching the
same executable). Removed now that the module links from the library like
every other test; verified test-tensor-class still passes linking against
libllama instead of its own private copy of the source.

tests/test-src-cmake-coverage.py is the RED/GREEN gate for the four-file
gap, registered in tests/CMakeLists.txt. Adapted from the original
proposal: a naive git-ls-files-vs-CMakeLists.txt substring check also
false-positives on ~140 files under src/models/, which have been globbed
in via file(GLOB ...) since 089dd41 rather than listed by name. Added a
sibling test_models_glob_is_present to guard that exemption mechanism
directly instead of hand-listing ~140 names in EXCLUDED, which would rot
on the next model addition.

Verified: RED confirmed on the unmodified tree (failed listing exactly
the four files above). GREEN after the fix (3 passed). Clean
./scripts/sycl-build.sh exits 0. ctest --test-dir build -N: 180 -> 181
(the one new gate; test-tensor-class was already registered). Mistral
Q4_0 completion gate on level_zero:1 outputs the expected
"1, 2, 3, 4, 5, 6, 7, 8, 9, 10" with GGML_SYCL:BOOL=ON and
libggml-sycl/libsycl linked into the binary.

llama.cpp-habh

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes
@ggerganov ggerganov ggerganov approved these changes
@pwilkin pwilkin pwilkin approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ngxson @ngladitz @CISC @ggerganov @pwilkin