Prebuilt: carry TML Inkling on top of MiniMax-M3 for the nightly PR set - #40
Prebuilt: carry TML Inkling on top of MiniMax-M3 for the nightly PR set#40danielhanchen wants to merge 5 commits into
Conversation
Hybrid attention model: 55 sliding-window plus 11 global layers, banded content-dependent relative position bias instead of RoPE, per-layer short convolution state, fine-grained MoE (256 experts top-6 plus 2 shared), attention log-scaling past 128K, 1M context. Includes the GGML_OP_FLASH_ATTN_EXT_BANDED operator (CPU and CUDA, fused into the MMA flash attention kernel with an fp16 accumulator overflow guard), HF to GGUF conversion, chat template with typed content block parsing (interleaved thinking, narration and tool calls), mmproj vision and audio support, and backend op tests at production shapes.
# Conflicts: # ggml/src/ggml-cuda/mmq.cuh # src/llama-model-saver.cpp # src/llama-vocab.h
Refreshing the ggml-org#25731 pin was not enough on its own. It and ggml-org#24523 conflict with each other in common/chat.cpp and src/llama-arch.h: both append a new llm_arch value immediately before LLM_ARCH_UNKNOWN and both add a chat parser plus a detection block in the same regions. Whichever entry is applied second conflicts, and reordering does not help. #40 carries ggml-org#24523 with ggml-org#25731 merged into it and the conflict resolved, keeping every function and dispatch entry from both sides. It also adapts Inkling to the renamed thinking_end_tags API, so it builds against b10133 and newer. Replaying the resolver sequence on b10133 with these pins: OK ggml-org ggml-org#24423 @ c3fb972 OK ggml-org ggml-org#24523 @ a58a7fa OK unslothai #40 @ fee66f6 Drop the #40 entry and repin ggml-org#25731 upstream once either PR lands.
Upstream 910196f renamed common_chat_params::thinking_end_tag to thinking_end_tags and changed it from std::string to a vector, so data.thinking_end_tag = END_MESSAGE no longer compiles.
…or the nightly prebuild
Both PRs add a chat parser and a template-detection block in the same
region of common/chat.cpp, immediately before "namespace workaround {",
so they cannot be merged into the prebuild tree independently:
whichever is applied second conflicts, regardless of ordering.
This branch carries the resolution. Both sides are purely additive in
common/chat.cpp against the merge base, so it keeps both
common_chat_params_init_minimax_m3 and common_chat_params_init_inkling
along with both detection blocks.
src/llama-arch.h no longer needs resolving: ggml-org#24523 now places
LLM_ARCH_MINIMAX_M3 next to LLM_ARCH_MINIMAX_M2 rather than at the tail
of the enum, so it no longer collides with LLM_ARCH_INKLING.
The thinking_end_tags adaptation is no longer carried here either; it
has landed on the Inkling branch itself.
a58a7fa to
baee0f5
Compare
fee66f6 to
233cedb
Compare
ggml-org ggml-org#24523 now places LLM_ARCH_MINIMAX_M3 next to LLM_ARCH_MINIMAX_M2 instead of at the tail of the enum, matching how llama-arch.cpp already groups the name table. That removes the src/llama-arch.h half of the collision with ggml-org#25731, which appends LLM_ARCH_INKLING at the tail. ggml-org ggml-org#25731 has picked up current master and the thinking_end_tags rename, so it now builds on its own against b10133 and newer. Both head SHAs moved, so the pins and the carrier branch are refreshed. unslothai #40 now only resolves what is left: both PRs still add a chat parser and a detection block in the same region of common/chat.cpp, immediately before "namespace workaround {". There is no principled alternative anchor there, since chat.cpp has no MiniMax-M2 parser to sit beside, so that half stays with the carrier. Replaying the resolver sequence on b10133: OK ggml-org ggml-org#24423 @ c3fb972 OK ggml-org ggml-org#24523 @ baee0f5 OK unslothai #40 @ 233cedb
#39 landed the first pin refresh only, so master still points at commits that have since moved on and that conflict with each other. ggml-org ggml-org#24523 was force-pushed: it now places LLM_ARCH_MINIMAX_M3 next to LLM_ARCH_MINIMAX_M2 rather than at the tail of the enum, matching how llama-arch.cpp already groups the name table. That removes the src/llama-arch.h half of its collision with ggml-org#25731, which appends LLM_ARCH_INKLING at the tail. New head baee0f5. ggml-org ggml-org#25731 has picked up current master and the thinking_end_tags rename, so it now builds on its own against b10133 and newer. Its entry is replaced by unslothai #40, which carries what is still unresolvable between the two: both add a chat parser and a detection block in the same region of common/chat.cpp, immediately before "namespace workaround {". chat.cpp has no MiniMax-M2 parser to sit beside, so there is no principled alternative anchor for that half. Replaying the resolver sequence on b10133: OK ggml-org ggml-org#24423 @ c3fb972 OK ggml-org ggml-org#24523 @ baee0f5 OK unslothai #40 @ 233cedb Drop the #40 entry and repin ggml-org#25731 upstream once either PR lands. Co-authored-by: Daniel Han <unslothai@gmail.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 233cedbe9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| model_arch = gguf.MODEL_ARCH.INKLING | ||
| undo_permute = False | ||
|
|
||
| _SKIP_PREFIXES = ("model.visual.", "model.audio.", "model.mtp.") |
There was a problem hiding this comment.
For checkpoints whose tower tensors use the top-level visual.* or audio.* layout, the text converter does not filter those tensors even though InklingMmprojModel.filter_tensors explicitly supports the same layout. They consequently reach the fallback map_tensor_name() call and fail with Can not map tensor, preventing creation of the language GGUF; include the bare tower prefixes in this skip list.
Useful? React with 👍 / 👎.
| // turns may interleave narration, thinking and calls; parse block-by-block (tool block | ||
| // first) since a whole-body choice would let the text branch swallow tool blocks into | ||
| // visible content | ||
| auto mixed_body = p.one_or_more(p.choice({ tool_block, reasoning_block, text_block })); |
There was a problem hiding this comment.
When tools use the default/auto tool choice and parallel_tool_calls is false, this unconditional one_or_more still permits multiple tool_blocks, so both the grammar and parser can accept and return several calls despite the request disabling parallel calls. Apply the same single-call constraint used by tool_calls to the mixed-content branch.
Useful? React with 👍 / 👎.
| GGML_OP_FILL, | ||
|
|
||
| GGML_OP_FLASH_ATTN_EXT, | ||
| GGML_OP_FLASH_ATTN_EXT_BANDED, |
There was a problem hiding this comment.
When an RPC client and server mix protocol patch versions 4.0.3 and 4.0.4, inserting this value in the middle of ggml_op renumbers every following operation even though negotiate_hello() accepts the connection without comparing the patch version. Because RPC serializes the numeric op value, an older peer can decode operations such as GGML_OP_SSM_CONV as a different operation, producing an invalid graph or incorrect results; append the new value without shifting existing IDs or make the handshake reject incompatible peers.
Useful? React with 👍 / 👎.
2 participants
Companion to #39. Resolves the last conflict blocking the nightly full release build.
Why this is needed
ggml-org#24523 (MiniMax-M3) and ggml-org#25731 (TML Inkling) cannot both be merged into the prebuild tree independently. Both append a new
llm_archvalue immediately beforeLLM_ARCH_UNKNOWN, and both add a chat parser plus a template-detection block in the same regions ofcommon/chat.cpp.Whichever entry is applied second conflicts, and reordering does not help:
Conflicting files both ways:
common/chat.cpp,src/llama-arch.h.What this branch is
ggml-org/llama.cpp#24523head (a58a7fa6) withggml-org/llama.cpp#25731head (453c4381) merged into it and the conflict resolved. Both sides are purely additive incommon/chat.cppagainst the merge base (b10107), so the resolution keeps everything from both sides:src/llama-arch.h: bothLLM_ARCH_MINIMAX_M3andLLM_ARCH_INKLINGcommon/chat.cpp: bothcommon_chat_params_init_minimax_m3andcommon_chat_params_init_inkling, and both detection blocksAdditional fix
Inkling had the same stale chat API usage that MiniMax-M3 did. Upstream 910196f renamed
common_chat_params::thinking_end_tagtothinking_end_tagsand made it a vector, so this no longer compiled:That means ggml-org#25731 as it stands does not build against
b10133or newer either, independently of the conflict.Verification
Built
llama,llama-common,test-llama-archsandtest-chatagainst the merged tree with zero errors.test-chat: all tests passedtest-llama-archs: exit 0,minimax-m3OK at 0.00e+00 (inklingreports SKIP on this backend, as several other archs do)Merge sequence on
b10133with this branch pinned as the third entry is clean.Lifetime
Temporary. Once either upstream PR lands, the other can be rebased and pinned directly again, and this branch should be dropped from
scripts/unsloth/pr-set.json. Per the pr-set doc, keep this PR open rather than merging it into fork master, since the nightly tree is the upstream tag plus pins.