Skip to content

Support model initialization without loading pretrained weights - #9821

Open
LiXinYuECNU wants to merge 1 commit into
modelscope:mainfrom
LiXinYuECNU:main
Open

Support model initialization without loading pretrained weights#9821
LiXinYuECNU wants to merge 1 commit into
modelscope:mainfrom
LiXinYuECNU:main

Conversation

@LiXinYuECNU

Copy link
Copy Markdown

Summary

  • add load_model and return_dummy_model model arguments to control model instantiation and pretrained-weight loading
  • allow constructing a randomly initialized model directly from config.json, including support for device_map=meta
  • add swift export --to_model_summary to inspect model architecture, parameter statistics, tokenizer, and template information without loading pretrained weights
  • add Chinese and English documentation plus runnable examples

Motivation

Some workflows need the model architecture and processor but do not need pretrained weights. Examples include training a model from random initialization, debugging tokenization/templates, and inspecting large model architectures without allocating real parameter memory or downloading weight files.

With this change:

  • --load_model false prepares only the config and processor
  • --return_dummy_model true builds the complete model from its config with randomly initialized parameters
  • --device_map meta keeps the dummy model on the meta device for architecture inspection without real memory allocation

User impact

Users can initialize models for training without loading pretrained checkpoints, or inspect model/tokenizer metadata through the export pipeline. Existing behavior remains unchanged because load_model=true and return_dummy_model=false are still the defaults.

Validation

  • Python syntax validation passed for all modified Python files
  • Ruff checks passed for the modified logic files
  • git diff --check passed
  • shell syntax checks passed for both new examples

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@LiXinYuECNU
LiXinYuECNU marked this pull request as ready for review July 31, 2026 08:09
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant