Skip to content

[Mirror] Add Gemma3n multimodal support with MobileNetV5 vision encoder #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ngxson wants to merge 22 commits into
base: master
Choose a base branch
from
Open

[Mirror] Add Gemma3n multimodal support with MobileNetV5 vision encoder #64

Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
3e4c8f8
Add Gemma3nVisionModel - MobileNetV5 vision encoder convertor to conv…
simrnsingh Dec 19, 2025
ad5ed98
Add mobilenetv5 impl
simrnsingh Dec 20, 2025
f577054
Fix comments, remove unused vars
simrnsingh Dec 20, 2025
4589d3e
Fix permute and remove transpose of projection weights
simrnsingh Dec 21, 2025
28d39cb
Merge branch 'master' into feat-gemma3n-vision
simrnsingh Dec 21, 2025
47423a2
Fix comments, remove debugging prints from hf_to_gguf
simrnsingh Dec 21, 2025
67801e5
1. Hard-code image_mean = 0 and image_std = 1
simrnsingh Dec 21, 2025
04947c7
1. Move mobilenetv5 helpers declarations to `clip_graph_mobilenetv5` …
simrnsingh Dec 21, 2025
86618c7
Remove obsolete comments
simrnsingh Dec 22, 2025
e2835e9
- convert_hf_to_gguf.py & constants.py & tensor_mapping.py: Use expli…
simrnsingh Dec 26, 2025
632e29f
- Rename tensors to v.conv..., v.blk..., v.msfa... to better align wi…
simrnsingh Dec 26, 2025
d37c22b
Fix stem conv bias name
simrnsingh Dec 26, 2025
58667f5
Remove explicit handling of bias term for stem conv
simrnsingh Dec 27, 2025
47b7dd1
- Change order of addition in "project_per_layer_inputs" to support b…
simrnsingh Dec 27, 2025
465e888
Merge branch 'master' into feat-gemma3n-vision
ngxson Jan 9, 2026
eea5881
clean up conversion script
ngxson Jan 9, 2026
bfbb315
fix code style
ngxson Jan 9, 2026
395d2d4
also preserve audio tensors
ngxson Jan 9, 2026
6a68b35
trailing space
ngxson Jan 9, 2026
e842b93
split arch A and V
ngxson Jan 9, 2026
8f6dbbe
rm unused gemma3 func
ngxson Jan 9, 2026
60c23c9
fix alignment
ngxson Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix stem conv bias name
  • Loading branch information
@simrnsingh
simrnsingh committed Dec 26, 2025
commit d37c22b2c5dd3e551e0e18e1061c2d89d1e8f8ff
2 changes: 1 addition & 1 deletion gguf-py/gguf/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1072,7 +1072,7 @@ class MODEL_TENSOR(IntEnum):
MODEL_TENSOR.V_MM_EMBEDDING: "mm.embedding", # gemma3n
MODEL_TENSOR.V_MM_HARD_EMB_NORM: "mm.hard_emb_norm", # gemma3n
MODEL_TENSOR.V_ENC_CONV_STEM: "v.conv_stem.conv", # gemma3n
MODEL_TENSOR.V_ENC_CONV_STEM_BIAS: "v.conv_stem.conv_bias", # gemma3n
MODEL_TENSOR.V_ENC_CONV_STEM_BIAS: "v.conv_stem.conv.bias", # gemma3n
MODEL_TENSOR.V_ENC_CONV_STEM_NORM: "v.conv_stem.bn", # gemma3n
MODEL_TENSOR.V_ENC_MSFA_EXP: "v.msfa.ffn.pw_exp.conv", # gemma3n
MODEL_TENSOR.V_ENC_MSFA_EXP_NORM: "v.msfa.ffn.pw_exp.bn", # gemma3n
Expand Down