chore(server): bump llama.cpp to b10238 / current image digests - #217
Merged
Conversation
Upstream moved well past every pin we carry. This bumps all three of
them and fixes the bundle breakage the new layout exposed.
Pins:
- Dockerfile.cuda 7b3d7834 -> fd68d130 (:server-cuda)
- Dockerfile 6bc9134e -> 9e60dd36 (:server, multi-arch)
- Makefile LLAMA_VERSION b8914 -> b10238 (+ checksum row)
fetch-llama.sh shipped a hand-maintained list of dylibs. b10238 moved
each tool's logic into its own libllama-<tool>-impl.dylib, so the list
produced a bundle whose llama-server aborted at dyld load time:
Library not loaded: @rpath/libllama-server-impl.dylib
Ship the whole dylib set instead — the same lesson the Dockerfiles
already encode with `COPY /app/*.so*`. The other tools' impl dylibs
cost ~1 MB of a ~52 MB bundle. Standalone binaries are still dropped.
A post-copy otool check now fails the fetch when any @rpath dependency
of llama-server is missing, so the next layout change surfaces at
fetch time instead of on an operator's machine.
Also fix the weekly pin-freshness workflow's duplicate detection: the
issue title contains "(" and ":", which GitHub's search parser reads as
a qualifier, so `--search "$title in:title"` never matched and every run
filed a fresh duplicate (#170 and #205 are the same reminder). Match the
exact title against the open-issue list instead.
Verified:
- CPU image builds locally (arm64) and /app/llama-server runs;
docker scout: 0 CRITICAL / 0 HIGH / 0 MEDIUM / 0 LOW, 73 MB
- macOS arm64 bundle: llama-server --version -> b10238, all @rpath
dependencies resolve
- CUDA image: /app/llama-server + libllama-server-impl.so present in
the new digest and the `COPY /app/*.so*` glob still captures them;
a full CUDA build + scan still needs the amd64 builder (host was
offline) and must run before the next server release
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps every llama.cpp pin in the repo and fixes the bundle breakage the new upstream layout exposed.
Closes #205, closes #170 (same reminder, filed twice — see the workflow fix below). Supersedes #181, whose digests were already stale.
Pins
server/Dockerfile.cuda7b3d7834fd68d130(:server-cuda)server/Dockerfile6bc9134e9e60dd36(:server, multi-arch)server/MakefileLLAMA_VERSION=b8914b10238(+ checksum row)The real bug:
fetch-llama.shshipped a fixed file listb10238 moved each tool's logic into its own
libllama-<tool>-impl.dylib. The hand-maintainedSHIP=(...)list did not know about it, so bumpingLLAMA_VERSIONalone produced a bundle whosellama-serveraborted at dyld load:This is the same failure that took prod down once via the CUDA image, which is why
Dockerfile/Dockerfile.cudaalreadyCOPY /app/*.so*wholesale. The fetch script now does the same:llama-server+ every*.dylib. Standalone binaries (llama-cli,llama-bench,ggml-rpc-server,mtmd-*, …) are still dropped. Cost: ~1 MB of other tools' impl dylibs in a ~52 MB bundle.A post-copy
otool -Lcheck now fails the fetch if any@rpathdependency ofllama-serveris missing from the destination — so the next upstream layout change surfaces at fetch time, not on an operator's machine.Workflow fix: duplicate reminder issues
The weekly pin-freshness job deduped with
gh issue list --search "$title in:title". The title contains(and:, so GitHub's search parser readschore(server/cuda):as a qualifier and the query matched nothing — every run filed a fresh duplicate. #170 and #205 are the same reminder. Now it matches the exact title against the open-issue list.Verification
/app/llama-serverruns (version: 10236).docker scout cves: 0C / 0H / 0M / 0L, 229 packages, 73 MB.llama-server --version→b10238, every@rpathdependency resolves./app/llama-server+libllama-server-impl.soare present and theCOPY /app/*.so*glob captures them;lddneeds only libs the distroless/cc-debian13 base + the copied CUDA/gomp set already provide.Important
A full
make scout-cuda(build + scan) has not run — it needs the native amd64 builder on the RTX box, which was offline. This must pass before the next server release tag.🤖 Generated with Claude Code