Preliminary MiniMax-M3 support

[danielhanchen]

Prelim support for MiniMax-M3.

• Tried to re-use as much as possible from other archs
• Sparse attention is not yet supported

[cpumaxx]

I can confirm this is working on my machine, although model warming code doesn't seem to trigger, even with explicit flags.
I can see you are still actively uploading various sized ggufs, but are you going to produce mmproj files as well?

[GmasteR]

tool calls not working.
in logs:
30.24.074.114 W srv operator(): got exception: {"error":{"code":500,"message":"Failed to parse input at pos 113: <]minimax[>[<tool_call>\n]<]minimax[>[<invoke name="bash">]<]minimax[>[date]<]minimax[>[]<]minimax[>[\n]<]minimax[>[</tool_call>","type":"server_error"}}

command: llama-server --host 0.0.0.0 -hf unsloth/MiniMax-M3-GGUF:UD-Q4_K_XL --alias minimax-m3 --jinja --cache-type-k q4_0 --cache-type-v q4_0 --cache-prompt --cache-reuse 256 --cache-ram 40960 --metrics --log-prefix -np 2 --ctx-size 262144 --kv-unified --fit on

[GmasteR]

tool calls not working. in logs: 30.24.074.114 W srv operator(): got exception: {"error":{"code":500,"message":"Failed to parse input at pos 113: <]minimax[>[<tool_call>\n]<]minimax[>[]<]minimax[>[date]<]minimax[>[]<]minimax[>[\n]<]minimax[>[</tool_call>","type":"server_error"}}

command: llama-server --host 0.0.0.0 -hf unsloth/MiniMax-M3-GGUF:UD-Q4_K_XL --alias minimax-m3 --jinja --cache-type-k q4_0 --cache-type-v q4_0 --cache-prompt --cache-reuse 256 --cache-ram 40960 --metrics --log-prefix -np 2 --ctx-size 262144 --kv-unified --fit on

Tool calling broken with the bundled M3 template — left-recursive PEG grammar
Build b9615 (2846a38), unsloth/MiniMax-M3-GGUF:UD-Q4_K_XL, 4×RTX 5090.
Any request containing tools fails. Startup logs:

load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
unsupported grammar, left recursion detected for nonterminal at index 29

and the server 500s on parse:

Failed to parse input at pos 103: <]minimax[>[<tool_call> ... </tool_call>

Plain chat (no tools) is fully correct — inference, clean stop, and reasoning_content (mm:think split) all work. So this is isolated to the tool-call grammar built from the bundled M3 template.

Fix that works: run with --chat-template-file models/templates/MiniMax-M2.jinja (same minimax:tool_call syntax, already PEG-compatible). The left recursion error disappears and tool calls now parse into proper structured tool_calls. Only change needed was / → mm:think/</mm:think> for M3 reasoning.
So the bundled M3 template's tool-call block produces a left-recursive grammar the autoparser can't build. Reusing the M2 tool-call rendering (or fixing the M3 template) should resolve it. The special_eos_id ∉ special_eog_ids mismatch probably wants a look too.

Ideal fix: ship a corrected models/templates/MiniMax-M3.jinja (M2 tool-call block + mm:think tags) and wire it as the default for the M3 arch, so -hf users get working tool calls without --chat-template-file

[RodriMora]

tool calls not working. in logs: 30.24.074.114 W srv operator(): got exception: {"error":{"code":500,"message":"Failed to parse input at pos 113: <]minimax[>[<tool_call>\n]<]minimax[>[]<]minimax[>[date]<]minimax[>[]<]minimax[>[\n]<]minimax[>[</tool_call>","type":"server_error"}}

command: llama-server --host 0.0.0.0 -hf unsloth/MiniMax-M3-GGUF:UD-Q4_K_XL --alias minimax-m3 --jinja --cache-type-k q4_0 --cache-type-v q4_0 --cache-prompt --cache-reuse 256 --cache-ram 40960 --metrics --log-prefix -np 2 --ctx-size 262144 --kv-unified --fit on

Im having similar results. Normal chat work fine, but tool calls errors out:

Error Details: Error code: 500 - {'error': {'code': 500, 'message': 'Failed to parse input at pos 557: <]minimax[>[<tool_call>\n]<]minimax[>[<invoke name="get_current_weather">]<]minimax[>[<location>London, UK]<]minimax[>[</location>]<]minimax[>[<unit>celsius]<]minimax[>[</unit>]<]minimax[>[</invoke>\n]<]minimax[>[<invoke name="calculator">]<]minimax[>[<expression>85 * 13]<]minimax[>[</expression>]<]minimax[>[</invoke>\n]<]minimax[>[</tool_call>', 'type': 'server_error'}} (Type: server_error)

[Sentdex]

For the time being, I am using a modified M2 template for M3: https://gist.github.com/Sentdex/35d78bfcf6558a0629ce21d9b5863c2f

and then:

--chat-template-file ~/llama.cpp-m3/models/templates/MiniMax-M3-temp.jinja when serving.

This will solve the major breaking-woes, but your harness still needs to contend with Minimax's tool-call markup.

[csabakecskemeti]

FYI
On Transformers 5.12.0 native MiniMax-M3 support and explicitly converts moe_layer_freq: [0,0,0,1,...] → mlp_layer_types: ['dense','dense','dense','sparse',...]

conversion/minimax.py need the following change
moe_layer_freq = self.find_hparam(["moe_layer_freq"]) -> moe_layer_freq = self.find_hparam(["moe_layer_freq", "mlp_layer_types"])

and in the for v in moe_layer_freq
also accept v == "dense"

[verdelyi]

Tool calling fix (FYI — feel free to use or discard)

I had Claude (Opus 4.8) dig into the Failed to parse input at pos N: ]<]minimax[>[<tool_call>... 500 that @GmasteR and @RodriMora reported, and it produced a working parser patch, so I figured I'd post it here in case it's useful.

Root cause: there's no specialized parser for M3, so it falls through to the differential autoparser, which can't handle M3's format for two reasons:

1. The namespace special token ]<]minimax[>[ (single vocab token, 200058) prefixes every tool-call tag, and its literal characters ] < [ > collide with the delimiters the autoparser uses — so it mis-segments the token (you can see it infer a mangled tool_section_start: '<]minimax[>' in --verbose).
2. M3 changed params from M2.7's <parameter name="k">v</parameter> to param-name-as-tag (<file_path>v</file_path>).

(@Sentdex's modified-M2-template workaround stops the crash but, as he noted, doesn't actually parse the markup into tool_calls.)

Fix: a dedicated common_chat_params_init_minimax_m3 in common/chat.cpp, modeled on deepseek_v3_2 — treats the ns-token as one opaque preserved token (sidestepping the char collision) and adds a param-as-tag arg rule, plus a template-detection branch. The grammar constrains generation to the format the model already emits natively, so it's in-distribution.

Edit / follow-up: also handles a reasoning-tag leak — M3 emits a bare </mm:think> (no opening tag) on non-thinking turns, e.g. between edits after tool results, which otherwise leaks into content. The patch makes the opening <mm:think> optional so the bare close is consumed. (Diff below updated accordingly.)

Verified against this PR's build (V100, UD-IQ3_XXS) — all return clean OpenAI tool_calls:

• single tool call, parallel calls, typed/boolean/enum params
• streaming (incremental argument deltas)
• full round-trip (assistant tool_calls + tool result fed back → coherent continuation)
• tools-present-but-no-call → finish_reason: stop, plain content
• post-tool-result continuation no longer leaks </mm:think>

Tested working through qwen-code as an agentic backend.

Patch (against `common/chat.cpp`, ~217 lines)

diff --git a/common/chat.cpp b/common/chat.cpp
index 9639af9ca..7e40d1c04 100644
--- a/common/chat.cpp
+++ b/common/chat.cpp
@@ -1979,6 +1979,194 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
     return data;
 }
 
+static common_chat_params common_chat_params_init_minimax_m3(const common_chat_template &    tmpl,
+                                                             const autoparser::generation_params & inputs) {
+    common_chat_params data;
+
+    data.prompt             = common_chat_template_direct_apply_impl(tmpl, inputs);
+    data.generation_prompt  = common_chat_template_generation_prompt_impl(tmpl, inputs);
+    data.format             = COMMON_CHAT_FORMAT_PEG_NATIVE;
+    data.supports_thinking  = true;
+    data.thinking_start_tag = "<mm:think>";
+    data.thinking_end_tag   = "</mm:think>";
+
+    // MiniMax-M3 wraps tool calls in XML where every tag is prefixed by the namespace
+    // special token "]<]minimax[>[" (a single vocab token). The wrapper is <tool_call>,
+    // each call is <invoke name="...">, and each parameter uses the parameter NAME as the
+    // tag (e.g. <file_path>...</file_path>) rather than <parameter name="...">.
+    const std::string NS           = "]<]minimax[>[";
+    const std::string THINK_START  = "<mm:think>";
+    const std::string THINK_END    = "</mm:think>";
+    const std::string FC_START     = NS + "<tool_call>";
+    const std::string FC_END       = NS + "</tool_call>";
+    const std::string INVOKE_END   = NS + "</invoke>";
+
+    data.preserved_tokens   = {
+        NS,
+        "<tool_call>",
+        "</tool_call>",
+        THINK_START,
+        THINK_END,
+    };
+
+    auto has_tools           = inputs.tools.is_array() && !inputs.tools.empty();
+    auto has_response_format = !inputs.json_schema.is_null() && inputs.json_schema.is_object();
+    auto extract_reasoning   = inputs.reasoning_format != COMMON_REASONING_FORMAT_NONE;
+    auto include_grammar     = has_response_format || (has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE);
+
+    const std::string GEN_PROMPT = data.generation_prompt;
+
+    if (inputs.has_continuation()) {
+        const auto & msg = inputs.continue_msg;
+
+        data.generation_prompt = GEN_PROMPT + THINK_START + msg.reasoning_content;
+        if (inputs.continue_final_message == COMMON_CHAT_CONTINUATION_CONTENT) {
+            data.generation_prompt += THINK_END + msg.render_content();
+        }
+
+        data.prompt += data.generation_prompt;
+    }
+
+    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
+        auto generation_prompt = p.literal(GEN_PROMPT);
+        auto end = p.end();
+
+        auto reasoning = p.eps();
+        // M3 emits a bare </mm:think> (no opening tag) on non-thinking turns
+        // (e.g. after tool results), so the opening tag must be optional.
+        if (extract_reasoning && inputs.enable_thinking) {
+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.reasoning(p.until(THINK_END)) + THINK_END);
+        } else if (extract_reasoning) {
+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.until(THINK_END) + p.literal(THINK_END));
+        }
+
+        if (has_response_format) {
+            auto response_format = p.rule("response-format",
+                p.literal("```json") + p.space() +
+                p.content(p.schema(p.json(), "response-format-schema", inputs.json_schema)) +
+                p.space() + p.literal("```"));
+            return generation_prompt + reasoning + response_format + end;
+        }
+
+        if (!has_tools || inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_NONE) {
+            return generation_prompt + reasoning + p.content(p.rest()) + end;
+        }
+
+        auto tool_choice = p.choice();
+        foreach_function(inputs.tools, [&](const json & tool) {
+            const auto & function = tool.at("function");
+            std::string  name     = function.at("name");
+            auto params   = function.contains("parameters") ? function.at("parameters") : json::object();
+            const auto & props    = params.contains("properties") ? params.at("properties") : json::object();
+
+            std::set<std::string> required;
+            if (params.contains("required")) {
+                params.at("required").get_to(required);
+            }
+
+            auto schema_info = common_schema_info();
+            schema_info.resolve_refs(params);
+
+            std::vector<common_peg_parser> required_parsers;
+            std::vector<common_peg_parser> optional_parsers;
+            for (const auto & [param_name, param_schema] : props.items()) {
+                bool is_required = required.find(param_name) != required.end();
+                bool is_string   = schema_info.resolves_to_string(param_schema);
+
+                const std::string p_close = NS + "</" + param_name + ">";
+
+                auto arg = p.tool_arg(
+                    p.tool_arg_open(
+                        p.literal(NS + "<") +
+                        p.tool_arg_name(p.literal(param_name)) +
+                        p.literal(">")) +
+                    (is_string
+                         ? p.tool_arg_string_value(p.until(p_close))
+                         : p.tool_arg_json_value(p.schema(p.json(),
+                                                          "tool-" + name + "-arg-" + param_name + "-schema",
+                                                          param_schema, false))) +
+                    p.tool_arg_close(p.literal(p_close)));
+
+                auto named_arg = p.rule("tool-" + name + "-arg-" + param_name, arg);
+                if (is_required) {
+                    required_parsers.push_back(named_arg);
+                } else {
+                    optional_parsers.push_back(named_arg);
+                }
+            }
+
+            common_peg_parser args_seq = p.eps();
+            for (size_t i = 0; i < required_parsers.size(); i++) {
+                if (i > 0) {
+                    args_seq = args_seq + p.space();
+                }
+                args_seq = args_seq + required_parsers[i];
+            }
+
+            if (!optional_parsers.empty()) {
+                common_peg_parser any_opt = p.choice();
+                for (const auto & opt : optional_parsers) {
+                    any_opt |= opt;
+                }
+                args_seq = args_seq + p.repeat(p.space() + any_opt, 0, -1);
+            }
+
+            common_peg_parser invoke_body = args_seq;
+            auto func_parser = p.tool(
+                p.tool_open(p.literal(NS + "<invoke name=\"") +
+                            p.tool_name(p.literal(name)) + p.literal("\">")) +
+                p.space() + invoke_body + p.space() +
+                p.tool_close(p.literal(INVOKE_END)));
+
+            tool_choice |= p.rule("tool-" + name, func_parser);
+        });
+
+        auto require_tools = inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED;
+
+        common_peg_parser tool_calls = p.eps();
+        if (inputs.parallel_tool_calls) {
+            tool_calls = p.trigger_rule("tool-call",
+                p.literal(FC_START) + p.space() + tool_choice +
+                p.zero_or_more(p.space() + tool_choice) + p.space() + p.literal(FC_END));
+        } else {
+            tool_calls = p.trigger_rule("tool-call",
+                p.literal(FC_START) + p.space() + tool_choice + p.space() + p.literal(FC_END));
+        }
+
+        if (!require_tools) {
+            tool_calls = p.optional(tool_calls);
+        }
+
+        auto content_before_tools = p.content(p.until(FC_START));
+        return generation_prompt + reasoning + content_before_tools + tool_calls + end;
+    });
+
+    data.parser = parser.save();
+
+    if (include_grammar) {
+        data.grammar_lazy = !(has_response_format || (has_tools && inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED));
+        data.grammar      = build_grammar([&](const common_grammar_builder & builder) {
+            foreach_function(inputs.tools, [&](const json & tool) {
+                const auto & function = tool.at("function");
+                auto         schema   = function.contains("parameters") ? function.at("parameters") : json::object();
+                builder.resolve_refs(schema);
+            });
+            if (has_response_format) {
+                auto schema = inputs.json_schema;
+                builder.resolve_refs(schema);
+            }
+            parser.build_grammar(builder, data.grammar_lazy);
+        });
+
+        data.grammar_triggers = {
+            { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, FC_START },
+        };
+    }
+
+    return data;
+}
+
+
 namespace workaround {
 
 static void map_developer_role_to_system(json & messages) {
@@ -2247,6 +2435,17 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
         return common_chat_params_init_gigachat_v3(tmpl, params);
     }
 
+    // MiniMax-M3 format detection: the namespace special token "]<]minimax[>[" prefixes
+    // every tool-call XML tag (<tool_call>/<invoke>/<param>), with the parameter NAME used
+    // as the tag. The generic autoparser cannot segment this token (its literal characters
+    // ]<[> collide with markup delimiters), so a dedicated parser is required.
+    if (src.find("]<]minimax[>[") != std::string::npos &&
+        src.find("<tool_call>") != std::string::npos &&
+        src.find("<invoke name=") != std::string::npos) {
+        LOG_DBG("Using specialized template: MiniMax-M3\n");
+        return common_chat_params_init_minimax_m3(tmpl, params);
+    }
+
     // DeepSeek V3.2 format detection: template defines dsml_token and uses it for tool calls.
     // The template source contains the token as a variable assignment, not as a literal in markup.
     if (src.find("dsml_token") != std::string::npos &&

[danielhanchen]

Hey folks sorry on the delay - will look into the issues today!

[danielhanchen]

Added tool calling parses as suggested + a transformers conversion fix + others

[RodriMora]

Added tool calling parses as suggested + a transformers conversion fix + others

Tool calling working fine with the mayor coding harness (opencode & pi).
Model seems coherent. I havan't run any benchmark yet though.

[USBhost]

I tested the MXFP4_MOE from unsloth and it's working fine in chat. Have not tested tool calls.

[Fringe210]

hey! Is there anyone working on the Sparse attention?

[cpumaxx]

What's left to fix on this PR before merging? I've been using it steady for over a week now with no issues.

[heapsoftware]

Need MiniMax Sparse Attention

[erm14254]

What's left to fix on this PR before merging? I've been using it steady for over a week now with no issues.

Vision (no mmproj creation possible) and Sparse attention, I think that's it, tool calling are working on my end.

[verdelyi]

Vision / video (mmproj) support — findings from an independent exploration (FYI, not a PR)

Following on from the tool-calling fix above: since the text side works, I had Claude (Opus 4.8) take a run at the dropped vision tower / mmproj, iterating on a 2× RTX 6000 Ada box against this PR's UD-IQ3_XXS. It now runs end-to-end on images and video, so I'm posting the reverse-engineering here in case it's useful for whoever does the official mmproj — the architecture mapping is the expensive part and it compresses to a few paragraphs.

I'm posting this as findings, not a PR — the mtmd module explicitly doesn't take predominantly AI-generated PRs, and this is AI-assisted work validated on one quant. Treat it as a map, not merge-ready code.

What M3-VL actually is (architecturally):

• Geometry = Qwen2.5-VL: Conv3d patch embed (split into 2× Conv2d), temporal_patch_size=2, 2×2 spatial merge, dynamic resolution. So it rides the existing Qwen2.5-VL scaffolding (build_inp_with_temporal_merge, token-count geometry, temporal merge) almost for free.
• Encoder = plain CLIP, not Qwen: LayerNorm (not RMSNorm), non-gated GELU MLP (fc1→gelu→fc2), full attention (no window attention), and pre_ln only (no post_ln).
• Projector = project-then-merge (this is the real departure from Qwen's concat-then-MLP): each patch is first projected to the text dim (mm.1→gelu→mm.2), then 2×2 neighbours are concatenated and run through a second GELU MLP (patch_merge.1→gelu→patch_merge.2).

The one trap that cost almost all the debugging time — 3-axis vision RoPE. M3 was trained with 3-axis (T/H/W) M-RoPE, not Qwen2.5-VL's 2-axis (H,W). Concretely: 2*(d_head/2) rotary dims split evenly across T/H/W (axis_dim pairs each, remainder pass-through), per HF MiniMaxM3VL3DRotaryEmbedding; positions laid out [t, h, w, e] with t=e=0 for stills. If you inherit Qwen's 2-axis sections {d/4,d/4,d/4,d/4} + positions [h,w,h,w], you get a train/test rope mismatch that (a) scrambles spatial detail on stills — centered subjects survive, distributed texture (landscapes) comes back smeared — and (b) feeds out-of-range position indices into ggml_rope_multi, which on the longer video decode path surfaces as an async CUDA error: unspecified launch failure mid-generation. Same root cause, two very different-looking symptoms. Fix is sections {ax,ax,ax,(d/2)-3*ax} with ax=(d_head/2)/3 (= {13,13,13,1} for d_head=80).

Footprint: ~270 lines across 11 files — most of it is the ViT graph (~145 lines) + clip.cpp wiring (~50); the rest is converter (Conv3d→2×Conv2d split, tensor map), enums, and tensor names. Small because it leans on the Qwen path.

Validated:

• HF transformers 5.12 reference tower built from the extracted weights loads 0 missing / 0 unexpected; patch grid → token count matches clip.cpp exactly; HF projector forward is line-identical to the clip.cpp graph.
• Images: cat/dog/food/landscapes all described accurately (landscapes only after the rope fix).
• Video works (no separate fix needed — the rope fix resolved the crash too). Tested on a slideshow, a synthetic motion clip (it perceives inter-frame motion, not just single frames), and a real webcam-gesture screencap where it read the Japanese on-screen text, tracked a countdown timer and score counter across frames, and identified the hand gestures — i.e. genuine OCR + temporal understanding.

Caveats (so nobody over-trusts this): built/tested only on the Unsloth UD-IQ3_XXS quant; validated numerically at the tensor-load + projector level and functionally, but not yet a bit-level embedding cosine diff vs HF; one residual I'm aware of — the ggml VISION theta_scale uses n_dims=d/2=40 as its frequency-normalisation denominator vs HF's axis_dim=26 (a smooth frequency stretch, not the gross scramble — output is accurate, but it's the obvious thing to tighten if you want exactness).

Operational notes for anyone reproducing on similar hardware: the tower is tiny, but the IQ3 text model auto-fits to fill both GPUs, leaving no VRAM for the vision encoder → run it CPU-side (--no-mmproj-offload); also --jinja + --image-min-tokens 1024 for accuracy on stills. On CPU the vision encode is ~2 s per merged frame-pair, so long/hi-res video gets slow — fine for short clips, wants an on-GPU tower for anything longer.

Full diff below (against this PR's branch, ~270 lines) — feel free to use as-is, use as a starting point, or discard. Sharing, not pushing a PR; given the mtmd no-AI-PR policy I'm leaving the decision to you. Most of it is plumbing (enums, tensor names, converter); the only real logic is the ViT graph in models/minimax_m3vl.cpp.

Patch (11 files, +271/−2)

diff --git a/conversion/__init__.py b/conversion/__init__.py
index d1582d3bf..7cc879610 100644
--- a/conversion/__init__.py
+++ b/conversion/__init__.py
@@ -247,6 +247,7 @@ TEXT_MODEL_MAP: dict[str, str] = {
 
 
 MMPROJ_MODEL_MAP: dict[str, str] = {
+    "MiniMaxM3SparseForConditionalGeneration": "minimax",
     "AudioFlamingo3ForConditionalGeneration": "ultravox",
     "CogVLMForCausalLM": "cogvlm",
     "DeepseekOCR2ForCausalLM": "deepseek",
diff --git a/conversion/minimax.py b/conversion/minimax.py
index d190e613d..238a0130d 100644
--- a/conversion/minimax.py
+++ b/conversion/minimax.py
@@ -1,13 +1,13 @@
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import Callable, Iterable, TYPE_CHECKING
 
 import torch
 
 if TYPE_CHECKING:
     from torch import Tensor
 
-from .base import ModelBase, TextModel, gguf
+from .base import MmprojModel, ModelBase, TextModel, gguf
 
 
 @ModelBase.register("MiniMaxM2ForCausalLM")
@@ -128,3 +128,37 @@ class MiniMaxM3Model(TextModel):
             return
 
         yield from super().modify_tensors(data_torch, name, bid)
+
+
+@ModelBase.register("MiniMaxM3SparseForConditionalGeneration")
+class MiniMaxM3VisionModel(MmprojModel):
+    # MiniMax-M3-VL vision tower: CLIP-style ViT (Conv3d patch embed, pre-LN, 3D M-RoPE,
+    # gelu MLP) + a per-patch GELU projector (multi_modal_projector) + a 2x2 patch-merge
+    # GELU MLP (patch_merge_mlp) into the text embedding dim. Most encoder tensors use
+    # standard CLIP names that already map in tensor_mapping.py; only patch_merge_mlp and
+    # the Conv3d split need special handling.
+
+    def set_gguf_parameters(self):
+        super().set_gguf_parameters()
+        assert self.hparams_vision is not None
+        self.gguf_writer.add_clip_projector_type(gguf.VisionProjectorType.MINIMAXM3VL)
+        self.gguf_writer.add_vision_attention_layernorm_eps(self.hparams_vision.get("layer_norm_eps", 1e-5))
+
+    @classmethod
+    def filter_tensors(cls, item: tuple[str, Callable[[], Tensor]]) -> tuple[str, Callable[[], Tensor]] | None:
+        name, _ = item
+        if not name.startswith(("vision_tower.", "multi_modal_projector.", "patch_merge_mlp.")):
+            return None
+        return super().filter_tensors(item)
+
+    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
+        if "embeddings.patch_embedding.weight" in name:
+            # split Conv3d [out, in, kt=2, kh, kw] into two Conv2d slices (Qwen-style;
+            # ggml has no Conv3d). clip.cpp sums the two temporal-slice convolutions.
+            _, _, kt, _, _ = data_torch.shape
+            assert kt == 2, "expected temporal_patch_size == 2"
+            patch = gguf.TENSOR_NAMES[gguf.MODEL_TENSOR.V_ENC_EMBD_PATCH]
+            yield (patch + ".weight", data_torch[:, :, 0, ...])
+            yield (patch + ".weight.1", data_torch[:, :, 1, ...])
+        else:
+            yield from super().modify_tensors(data_torch, name, bid)
diff --git a/gguf-py/gguf/constants.py b/gguf-py/gguf/constants.py
index 234c7fc43..c80a1ebdc 100644
--- a/gguf-py/gguf/constants.py
+++ b/gguf-py/gguf/constants.py
@@ -778,6 +778,8 @@ class MODEL_TENSOR(IntEnum):
     V_MM_SOFT_EMB_NORM   = auto() # gemma3
     V_MM_EMBEDDING       = auto() # gemma3n
     V_MM_HARD_EMB_NORM   = auto() # gemma3n
+    V_MM_PATCH_MERGE_1   = auto() # minimax-m3-vl
+    V_MM_PATCH_MERGE_2   = auto() # minimax-m3-vl
     V_ENC_CONV_STEM      = auto() # gemma3n
     V_ENC_CONV_STEM_NORM = auto() # gemma3n
     V_ENC_MSFA_EXP       = auto() # gemma3n
@@ -1333,6 +1335,8 @@ TENSOR_NAMES: dict[MODEL_TENSOR, str] = {
     MODEL_TENSOR.V_MM_SOFT_EMB_NORM:        "mm.soft_emb_norm",         # gemma3n
     MODEL_TENSOR.V_MM_EMBEDDING:            "mm.embedding",             # gemma3n
     MODEL_TENSOR.V_MM_HARD_EMB_NORM:        "mm.hard_emb_norm",         # gemma3n
+    MODEL_TENSOR.V_MM_PATCH_MERGE_1:        "mm.patch_merge.1",         # minimax-m3-vl
+    MODEL_TENSOR.V_MM_PATCH_MERGE_2:        "mm.patch_merge.2",         # minimax-m3-vl
     MODEL_TENSOR.V_ENC_CONV_STEM:           "v.conv_stem.conv",         # gemma3n
     MODEL_TENSOR.V_ENC_CONV_STEM_NORM:      "v.conv_stem.bn",           # gemma3n
     MODEL_TENSOR.V_ENC_MSFA_EXP:            "v.msfa.ffn.pw_exp.conv",   # gemma3n
@@ -1538,6 +1542,8 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
         MODEL_TENSOR.V_LAYER_OUT_SCALE,
         MODEL_TENSOR.V_PRE_NORM,
         MODEL_TENSOR.V_POST_NORM,
+        MODEL_TENSOR.V_MM_PATCH_MERGE_1,
+        MODEL_TENSOR.V_MM_PATCH_MERGE_2,
         MODEL_TENSOR.V_MM_POST_NORM,
         MODEL_TENSOR.V_MM_INP_PROJ,
         MODEL_TENSOR.V_MM_INP_NORM,
@@ -4552,6 +4558,7 @@ class VisionProjectorType:
     EXAONE4_5 = "exaone4_5"
     QWEN3VL = "qwen3vl_merger"
     STEP3VL = "step3vl"
+    MINIMAXM3VL = "minimaxm3vl"
     ULTRAVOX = "ultravox"
     INTERNVL = "internvl"
     QWEN2A = "qwen2a" # audio
diff --git a/gguf-py/gguf/tensor_mapping.py b/gguf-py/gguf/tensor_mapping.py
index 5f1e28818..58ecfe9f2 100644
--- a/gguf-py/gguf/tensor_mapping.py
+++ b/gguf-py/gguf/tensor_mapping.py
@@ -1406,6 +1406,14 @@ class TensorNameMap:
             "model.mm_projector.peg.peg.{bid}",
         ),
 
+        MODEL_TENSOR.V_MM_PATCH_MERGE_1: (
+            "patch_merge_mlp.linear_1", # minimax-m3-vl
+        ),
+
+        MODEL_TENSOR.V_MM_PATCH_MERGE_2: (
+            "patch_merge_mlp.linear_2", # minimax-m3-vl
+        ),
+
         MODEL_TENSOR.V_ENC_EMBD_CLS: (
             "vision_tower.vision_model.embeddings.class_embedding",
             "model.vision_tower.embeddings.cls_token", # Intern-S1
diff --git a/tools/mtmd/CMakeLists.txt b/tools/mtmd/CMakeLists.txt
index 09b62357f..5588c50e5 100644
--- a/tools/mtmd/CMakeLists.txt
+++ b/tools/mtmd/CMakeLists.txt
@@ -39,6 +39,7 @@ add_library(mtmd
             models/minicpmv.cpp
             models/paddleocr.cpp
             models/pixtral.cpp
+            models/minimax_m3vl.cpp
             models/qwen2vl.cpp
             models/qwen3vl.cpp
             models/mimovl.cpp
diff --git a/tools/mtmd/clip-impl.h b/tools/mtmd/clip-impl.h
index b104f3736..61c2e75ba 100644
--- a/tools/mtmd/clip-impl.h
+++ b/tools/mtmd/clip-impl.h
@@ -123,6 +123,7 @@
 #define TN_MM_SOFT_EMB_N   "mm.soft_emb_norm.weight"    // gemma3
 #define TN_MM_PROJECTOR    "mm.model.fc.%s"             // idefics3, deepseekocr
 #define TN_MM_PATCH_MERGER "mm.patch_merger.%s"         // mistral small 3.1, glm4v
+#define TN_MM_PATCH_MERGE  "mm.patch_merge.%d.%s"        // minimax-m3-vl (2x2 spatial merge MLP)
 #define TN_TOK_IMG_BREAK   "v.token_embd.img_break"     // pixtral
 #define TN_TOK_GLM_BOI     "adapter.boi"                // glm-edge (these embeddings are not in text model)
 #define TN_TOK_GLM_EOI     "adapter.eoi"                // glm-edge (these embeddings are not in text model)
@@ -363,6 +364,7 @@ enum projector_type {
     PROJECTOR_TYPE_GRANITE_SPEECH,
     PROJECTOR_TYPE_MIMOVL,
     PROJECTOR_TYPE_GRANITE4_VISION,
+    PROJECTOR_TYPE_MINIMAX_M3_VL,
     PROJECTOR_TYPE_UNKNOWN,
 };
 
@@ -375,6 +377,7 @@ static std::map<projector_type, std::string> PROJECTOR_TYPE_NAMES = {
     { PROJECTOR_TYPE_QWEN2VL,   "qwen2vl_merger"},
     { PROJECTOR_TYPE_QWEN25VL,  "qwen2.5vl_merger"},
     { PROJECTOR_TYPE_QWEN3VL,   "qwen3vl_merger"},
+    { PROJECTOR_TYPE_MINIMAX_M3_VL, "minimaxm3vl"},
     { PROJECTOR_TYPE_STEP3VL,   "step3vl"},
     { PROJECTOR_TYPE_GEMMA3,    "gemma3"},
     { PROJECTOR_TYPE_GEMMA3NV,  "gemma3nv"},
diff --git a/tools/mtmd/clip-model.h b/tools/mtmd/clip-model.h
index 48796b630..f0c44b258 100644
--- a/tools/mtmd/clip-model.h
+++ b/tools/mtmd/clip-model.h
@@ -400,6 +400,12 @@ struct clip_model {
     ggml_tensor * mm_2_w = nullptr;
     ggml_tensor * mm_2_b = nullptr;
 
+    // minimax-m3-vl 2x2 patch-merge MLP (applied after the per-patch projector)
+    ggml_tensor * mm_patch_merge_0_w = nullptr;
+    ggml_tensor * mm_patch_merge_0_b = nullptr;
+    ggml_tensor * mm_patch_merge_1_w = nullptr;
+    ggml_tensor * mm_patch_merge_1_b = nullptr;
+
     ggml_tensor * image_newline = nullptr;
     ggml_tensor * view_seperator = nullptr;
 
diff --git a/tools/mtmd/clip.cpp b/tools/mtmd/clip.cpp
index 208486fd1..0fbbdcd8c 100644
--- a/tools/mtmd/clip.cpp
+++ b/tools/mtmd/clip.cpp
@@ -907,6 +907,10 @@ static std::unique_ptr<clip_graph> clip_get_graph_builder(clip_ctx * ctx, const
             {
                 builder = std::make_unique<clip_graph_qwen3vl>(ctx, img);
             } break;
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
+            {
+                builder = std::make_unique<clip_graph_minimax_m3vl>(ctx, img);
+            } break;
         case PROJECTOR_TYPE_EXAONE4_5:
             {
                 builder = std::make_unique<clip_graph_exaone4_5>(ctx, img);
@@ -1441,6 +1445,7 @@ struct clip_model_loader {
                 case PROJECTOR_TYPE_QWEN2VL:
                 case PROJECTOR_TYPE_QWEN25VL:
                 case PROJECTOR_TYPE_QWEN3VL:
+                case PROJECTOR_TYPE_MINIMAX_M3_VL:
                     {
                         hparams.n_merge = 2; // default value for Qwen 2 and 2.5
                         hparams.image_resize_algo = RESIZE_ALGO_BILINEAR;
@@ -2040,6 +2045,19 @@ struct clip_model_loader {
                     model.mm_1_w = get_tensor(string_format(TN_LLAVA_PROJ, 2, "weight"));
                     model.mm_1_b = get_tensor(string_format(TN_LLAVA_PROJ, 2, "bias"));
                 } break;
+            case PROJECTOR_TYPE_MINIMAX_M3_VL:
+                {
+                    // per-patch projector (multi_modal_projector): mm.1 -> mm.2
+                    model.mm_0_w = get_tensor(string_format(TN_LLAVA_PROJ, 1, "weight"));
+                    model.mm_0_b = get_tensor(string_format(TN_LLAVA_PROJ, 1, "bias"));
+                    model.mm_1_w = get_tensor(string_format(TN_LLAVA_PROJ, 2, "weight"));
+                    model.mm_1_b = get_tensor(string_format(TN_LLAVA_PROJ, 2, "bias"));
+                    // 2x2 patch-merge MLP (patch_merge_mlp): mm.patch_merge.1 -> .2
+                    model.mm_patch_merge_0_w = get_tensor(string_format(TN_MM_PATCH_MERGE, 1, "weight"));
+                    model.mm_patch_merge_0_b = get_tensor(string_format(TN_MM_PATCH_MERGE, 1, "bias"));
+                    model.mm_patch_merge_1_w = get_tensor(string_format(TN_MM_PATCH_MERGE, 2, "weight"));
+                    model.mm_patch_merge_1_b = get_tensor(string_format(TN_MM_PATCH_MERGE, 2, "bias"));
+                } break;
             case PROJECTOR_TYPE_MIMOVL:
                 {
                     model.mm_0_w = get_tensor(string_format(TN_LLAVA_PROJ, 0, "weight"));
@@ -3213,6 +3231,7 @@ int clip_n_output_tokens_x(const struct clip_ctx * ctx, struct clip_image_f32 *
         case PROJECTOR_TYPE_QWEN2VL:
         case PROJECTOR_TYPE_QWEN25VL:
         case PROJECTOR_TYPE_QWEN3VL:
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
         case PROJECTOR_TYPE_EXAONE4_5:
         case PROJECTOR_TYPE_MIMOVL:
         case PROJECTOR_TYPE_GLM4V:
@@ -3235,6 +3254,7 @@ int clip_n_output_tokens_y(const struct clip_ctx * ctx, struct clip_image_f32 *
         case PROJECTOR_TYPE_QWEN2VL:
         case PROJECTOR_TYPE_QWEN25VL:
         case PROJECTOR_TYPE_QWEN3VL:
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
         case PROJECTOR_TYPE_EXAONE4_5:
         case PROJECTOR_TYPE_MIMOVL:
         case PROJECTOR_TYPE_GLM4V:
@@ -3315,6 +3335,7 @@ int clip_n_output_tokens(const struct clip_ctx * ctx, struct clip_image_f32 * im
         case PROJECTOR_TYPE_QWEN2VL:
         case PROJECTOR_TYPE_QWEN25VL:
         case PROJECTOR_TYPE_QWEN3VL:
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
         case PROJECTOR_TYPE_EXAONE4_5:
         case PROJECTOR_TYPE_MIMOVL:
         case PROJECTOR_TYPE_GLM4V:
@@ -3756,6 +3777,31 @@ bool clip_image_batch_encode(clip_ctx * ctx, const int n_threads, const clip_ima
                 set_input_i32("merger_ds_idx_2", m_ds_2);
                 set_input_i32("merger_ds_idx_3", m_ds_3);
             } break;
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
+            {
+                // 3-axis (T/H/W) vision RoPE: position channels are [t, h, w, e].
+                // For a still image t=0 and the pass-through channel e=0; only h,w vary.
+                // Patches are emitted in 2x2 merge-window order (matches the projector merge).
+                const int merge_ratio = hparams.n_merge;
+                const int pw = image_size_width  / patch_size;
+                const int ph = image_size_height / patch_size;
+                std::vector<int> positions(n_pos * 4, 0);
+                int ptr = 0;
+                for (int y = 0; y < ph; y += merge_ratio) {
+                    for (int x = 0; x < pw; x += merge_ratio) {
+                        for (int dy = 0; dy < merge_ratio; dy++) {
+                            for (int dx = 0; dx < merge_ratio; dx++) {
+                                positions[                  ptr] = 0;     // t
+                                positions[    num_patches + ptr] = y + dy; // h
+                                positions[2 * num_patches + ptr] = x + dx; // w
+                                positions[3 * num_patches + ptr] = 0;     // e (pass-through)
+                                ptr++;
+                            }
+                        }
+                    }
+                }
+                set_input_i32("positions", positions);
+            } break;
         case PROJECTOR_TYPE_QWEN2VL:
         case PROJECTOR_TYPE_QWEN3VL:
         case PROJECTOR_TYPE_GLM4V:
@@ -4523,6 +4569,8 @@ int clip_n_mmproj_embd(const struct clip_ctx * ctx) {
         case PROJECTOR_TYPE_JANUS_PRO:
         case PROJECTOR_TYPE_YOUTUVL:
             return ctx->model.mm_1_b->ne[0];
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
+            return ctx->model.mm_patch_merge_1_b->ne[0];
         case PROJECTOR_TYPE_QWEN3VL:
             // main path + deepstack paths
             return ctx->model.mm_1_b->ne[0] * (1 + ctx->model.n_deepstack_layers);
@@ -4606,6 +4654,7 @@ int clip_model_n_temporal_merge(const struct clip_ctx * ctx) {
         case PROJECTOR_TYPE_QWEN2VL:
         case PROJECTOR_TYPE_QWEN25VL:
         case PROJECTOR_TYPE_QWEN3VL:
+        case PROJECTOR_TYPE_MINIMAX_M3_VL:
             return 2;
         default:
             return 1;
diff --git a/tools/mtmd/models/minimax_m3vl.cpp b/tools/mtmd/models/minimax_m3vl.cpp
new file mode 100644
index 000000000..c0ebe31fa
--- /dev/null
+++ b/tools/mtmd/models/minimax_m3vl.cpp
@@ -0,0 +1,145 @@
+#include "models.h"
+
+// MiniMax-M3-VL vision tower.
+//
+// Geometry is identical to Qwen2.5-VL (Conv3d patch embed split into 2 Conv2d, 3D
+// M-RoPE, 2x2 spatial merge, temporal_patch_size=2), so we reuse
+// clip_graph_qwen2vl::build_inp_with_temporal_merge() and the same position layout.
+//
+// The encoder differs: standard CLIP transformer blocks (LayerNorm, NOT RMSNorm;
+// non-gated GELU MLP fc1->gelu->fc2), full attention (no window attention), pre_ln
+// only (no post_ln).
+//
+// The projector differs from Qwen's concat-then-MLP: MiniMax projects EACH patch to
+// the text dim first (mm.1 -> gelu -> mm.2), THEN concatenates 2x2 neighbours and runs
+// a second GELU MLP (mm.patch_merge.1 -> gelu -> mm.patch_merge.2).
+ggml_cgraph * clip_graph_minimax_m3vl::build() {
+    GGML_ASSERT(model.patch_bias == nullptr);
+    GGML_ASSERT(model.class_embedding == nullptr);
+    GGML_ASSERT(model.position_embeddings == nullptr); // 3D M-RoPE, no learned pos-emb
+
+    const int batch_size       = 1;
+    const int n_pos            = n_patches;
+    const int num_position_ids = n_pos * 4; // m-rope requires 4 dim per position
+
+    // CLIP-style LayerNorm (weight + bias), unlike Qwen2.5-VL's RMSNorm
+    const norm_type norm_t = NORM_TYPE_NORMAL;
+
+    // MiniMax-M3 uses 3-axis (T/H/W) vision RoPE: 2*(d_head/2) rotary dims split evenly
+    // across T,H,W (each axis_dim pairs), remaining dims pass through. Unlike Qwen2.5-VL's
+    // 2-axis {d/4,d/4,d/4,d/4}. positions are laid out [t, h, w, e] with t=e=0 for images.
+    const int ax = (d_head / 2) / 3;                 // pairs per axis (13 for d_head=80)
+    int mrope_sections[4] = {ax, ax, ax, (d_head / 2) - 3 * ax};
+
+    ggml_tensor * inp = build_inp_with_temporal_merge();
+
+    // reorder patches into 2x2 spatial-merge windows (same as Qwen2VL), so that 4
+    // consecutive patches form a 2x2 block for the merge step in the projector
+    {
+        inp = ggml_permute(ctx0, inp, 1, 2, 0, 3);  // [w, h, c, b] -> [c, w, h, b]
+        inp = ggml_cont_4d(ctx0, inp, n_embd * 2, n_patches_x / 2, n_patches_y, batch_size);
+        inp = ggml_reshape_4d(ctx0, inp, n_embd * 2, n_patches_x / 2, 2, batch_size * (n_patches_y / 2));
+        inp = ggml_permute(ctx0, inp, 0, 2, 1, 3);
+        inp = ggml_cont_3d(ctx0, inp, n_embd, n_patches_x * n_patches_y, batch_size);
+    }
+
+    ggml_tensor * inpL = inp;
+
+    ggml_tensor * positions = ggml_new_tensor_1d(ctx0, GGML_TYPE_I32, num_position_ids);
+    ggml_set_name(positions, "positions");
+    ggml_set_input(positions);
+
+    // pre-layernorm
+    if (model.pre_ln_w) {
+        inpL = build_norm(inpL, model.pre_ln_w, model.pre_ln_b, norm_t, eps, -1);
+        cb(inpL, "pre_ln", -1);
+    }
+
+    // encoder layers (full attention throughout)
+    for (int il = 0; il < n_layer; il++) {
+        const auto & layer = model.layers[il];
+
+        ggml_tensor * cur = inpL; // residual = inpL
+
+        // layernorm1
+        cur = build_norm(cur, layer.ln_1_w, layer.ln_1_b, norm_t, eps, il);
+        cb(cur, "ln1", il);
+
+        // self-attention
+        {
+            ggml_tensor * Qcur = ggml_add(ctx0, build_mm(layer.q_w, cur), layer.q_b);
+            ggml_tensor * Kcur = ggml_add(ctx0, build_mm(layer.k_w, cur), layer.k_b);
+            ggml_tensor * Vcur = ggml_add(ctx0, build_mm(layer.v_w, cur), layer.v_b);
+
+            Qcur = ggml_reshape_3d(ctx0, Qcur, d_head, n_head, n_patches);
+            Kcur = ggml_reshape_3d(ctx0, Kcur, d_head, n_head, n_patches);
+            Vcur = ggml_reshape_3d(ctx0, Vcur, d_head, n_head, n_patches);
+
+            cb(Qcur, "Qcur", il);
+            cb(Kcur, "Kcur", il);
+            cb(Vcur, "Vcur", il);
+
+            // 3D M-RoPE (same as Qwen2.5-VL vision)
+            Qcur = ggml_rope_multi(ctx0, Qcur, positions, nullptr,
+                d_head/2, mrope_sections, GGML_ROPE_TYPE_VISION, 32768, 10000, 1, 0, 1, 32, 1);
+            Kcur = ggml_rope_multi(ctx0, Kcur, positions, nullptr,
+                d_head/2, mrope_sections, GGML_ROPE_TYPE_VISION, 32768, 10000, 1, 0, 1, 32, 1);
+
+            cb(Qcur, "Qcur_rope", il);
+            cb(Kcur, "Kcur_rope", il);
+
+            cur = build_attn(layer.o_w, layer.o_b, Qcur, Kcur, Vcur, nullptr, kq_scale, il);
+            cb(cur, "attn_out", il);
+        }
+
+        // residual 1
+        cur = ggml_add(ctx0, cur, inpL);
+        inpL = cur;
+        cb(cur, "ffn_inp", il);
+
+        // layernorm2
+        cur = build_norm(cur, layer.ln_2_w, layer.ln_2_b, norm_t, eps, il);
+        cb(cur, "ffn_inp_normed", il);
+
+        // non-gated GELU MLP (fc1 -> gelu -> fc2)
+        cur = build_ffn(cur,
+            layer.ff_up_w,   layer.ff_up_b,
+            nullptr,         nullptr,
+            layer.ff_down_w, layer.ff_down_b,
+            FFN_GELU, il);
+        cb(cur, "ffn_out", il);
+
+        // residual 2
+        cur = ggml_add(ctx0, inpL, cur);
+        cb(cur, "layer_out", il);
+        inpL = cur;
+    }
+
+    // (no post-layernorm for MiniMax-M3-VL)
+
+    // ---- projector: project-then-merge ----
+    ggml_tensor * embeddings = inpL; // [n_embd, n_pos, batch]
+
+    // per-patch projection to text dim: mm.1 -> gelu -> mm.2
+    embeddings = build_ffn(embeddings,
+        model.mm_0_w, model.mm_0_b,
+        nullptr,      nullptr,
+        model.mm_1_w, model.mm_1_b,
+        FFN_GELU, -1);
+    cb(embeddings, "proj_per_patch", -1);
+
+    // concat 2x2 neighbours: [proj_dim, n_pos, b] -> [proj_dim*4, n_pos/4, b]
+    embeddings = ggml_reshape_3d(ctx0, embeddings, hparams.projection_dim * 4, n_pos / 4, batch_size);
+
+    // patch-merge MLP: mm.patch_merge.1 -> gelu -> mm.patch_merge.2
+    embeddings = build_ffn(embeddings,
+        model.mm_patch_merge_0_w, model.mm_patch_merge_0_b,
+        nullptr,                  nullptr,
+        model.mm_patch_merge_1_w, model.mm_patch_merge_1_b,
+        FFN_GELU, -1);
+    cb(embeddings, "proj_merged", -1);
+
+    ggml_build_forward_expand(gf, embeddings);
+
+    return gf;
+}
diff --git a/tools/mtmd/models/models.h b/tools/mtmd/models/models.h
index 3a15f7682..b13dd6484 100644
--- a/tools/mtmd/models/models.h
+++ b/tools/mtmd/models/models.h
@@ -40,6 +40,14 @@ struct clip_graph_qwen3vl : clip_graph_qwen2vl {
     ggml_cgraph * build() override;
 };
 
+// MiniMax-M3-VL: Qwen2.5-VL-style geometry (Conv3d patch embed, 3D M-RoPE, 2x2 merge)
+// but a CLIP-style encoder (LayerNorm, non-gated GELU MLP, full attention) and a
+// project-then-merge projector. Reuses qwen2vl's build_inp_with_temporal_merge().
+struct clip_graph_minimax_m3vl : clip_graph_qwen2vl {
+    clip_graph_minimax_m3vl(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph_qwen2vl(ctx, img) {}
+    ggml_cgraph * build() override;
+};
+
 struct clip_graph_mimovl : clip_graph {
     clip_graph_mimovl(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph(ctx, img) {}
     ggml_cgraph * build() override;
diff --git a/tools/mtmd/mtmd.cpp b/tools/mtmd/mtmd.cpp
index 8e839ef8f..27e01c160 100644
--- a/tools/mtmd/mtmd.cpp
+++ b/tools/mtmd/mtmd.cpp
@@ -460,6 +460,13 @@ struct mtmd_context {
                     img_end = "<|vision_end|>";
                     image_preproc = std::make_unique<mtmd_image_preprocessor_dyn_size>(ctx_v);
                 } break;
+            case PROJECTOR_TYPE_MINIMAX_M3_VL:
+                {
+                    // ]<]start of image[>[ ... (image embeddings) ... ]<]end of image[>[
+                    img_beg = "]<]start of image[>[";
+                    img_end = "]<]end of image[>[";
+                    image_preproc = std::make_unique<mtmd_image_preprocessor_dyn_size>(ctx_v);
+                } break;
             case PROJECTOR_TYPE_YOUTUVL:
                 {
                     // <|vision_start|> ... (image embeddings) ... <|vision_end|>

[evilJazz]

Is this PR not moving forward because it is still in draft state? IMO all the other features could be added at a later stage. Not having support for M3 in mainline is quite baffling TBH.

[erm14254]

Is there any particularly compelling reason for why this PR hasn't been merged yet?

I assume because it is still being worked on or something?

[stew675]

Is there any particularly compelling reason for why this PR hasn't been merged yet?

I assume because it is still being worked on or something?

It already has one approval, and Daniel just seems to be fixing up merge conflicts at this point whenever mainline is updated and breaks a clean merge. If there's something else to be done, no one seems to have said what that is. It just seems to be getting ignored and that's kind of disappointing.

[erm14254]

If there's something else to be done, no one seems to have said what that is.

From what I understand this PR still has no support for Sparse Attention nor Vision Projector, no? Though that could easily be overlooked since timkhronos is already covering this in his two other PRs and if all 3 gets merged into mainline llama.cpp you would get MiniMax-M3 support with pretty much everything?

[danielhanchen]

The PR should be fine @pwilkin probs if you can re-review

I checked CI - it seems to be flaky CI tests - but I can't reinvoke so I might do a 0 diff commit to force CI to run

[pwilkin]

@danielhanchen it's fine from my side but you need someone from the core/model maintainers to review it, I'm only responsible for the parser parts :)

[Volunteer-1]

I tried to merge and build this. It fails:
llama.cpp\common\chat.cpp(2133,10): error C2039: 'thinking_end_tag': is not a member of 'common_chat_params' [llama.cpp\build\common\llama-common.vcxproj]
llama.cpp\common\chat.h(269,8):
see declaration of 'common_chat_params'

[danielhanchen]

@ngxson Would it be possible for you to take a look into this - appreciate it!

[danielhanchen]

I fixed merge conflicts for the other PRs I made (Inkling + DiffusionGemma) as well since they can't merge together as well

[stew675]

Just confirming that this all now works at my end. Unsloth MiniMax-M3 @ IQ3_XS (~200GiB weights) across 2 x 128GB Strix Halo's using Vulkan and pipeline (layer) parallelism. Running at about 10t/s, but given that we're missing the improvements of the other two PR's that are also waiting, then that all seems about right. I think hitting 15-20t/s would definitely be achievable once the other PRs also go in. Top work @danielhanchen

[pwilkin]

@danielhanchen so I talked to the core team and they've decided to approve @timkhronos branch because it's more feature complete. Sorry that it turned out this way - if you could catch me on Discord at some point, I'd like to talk about how to avoid misunderstandings and duplicated work like this in the future.

[danielhanchen]

@pwilkin No worries - I see the other PR anyways built on top of this haha so all good!

[erm14254]

if you could catch me on Discord at some point, I'd like to talk about how to avoid misunderstandings and duplicated work like this in the future.

What!? How can you say something like "misunderstandings and duplicated work like this", when the commit history of both of the merged PRs makes it clear that both were built upon this PR, not the other way around: