Skip to content

Commit 60defb4

Browse files
LollipopliuxiaomingJustinTong0323
authored andcommitted
fix: correct weight loading prefix mapping for Qwen3-VL (sgl-project#18024)
Co-authored-by: liuxiaoming <liuxiaoming@modelbest.cn> Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
1 parent 72765f3 commit 60defb4

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

python/sglang/srt/models/qwen3_vl.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -959,7 +959,13 @@ def load_weights(self, weights: Iterable[Tuple[str, torch.Tensor]]):
959959
name = name.replace(r"model.language_model.", r"model.")
960960
layer_id = get_layer_id(name)
961961

962-
if self.pp_group.is_last_rank and "model.embed_tokens.weight" in name:
962+
# Only copy embed_tokens to lm_head when tie_word_embeddings=True
963+
# For models with tie_word_embeddings=False (e.g. 8B), lm_head has independent weights
964+
if (
965+
self.pp_group.is_last_rank
966+
and "model.embed_tokens.weight" in name
967+
and self.config.tie_word_embeddings
968+
):
963969
if "lm_head.weight" in params_dict:
964970
lm_head_param = params_dict["lm_head.weight"]
965971
weight_loader = getattr(

0 commit comments

Comments
 (0)