[bug] Crash when loading pretrained model

[jlamypoirier]

🐞 Describe the Bug

Had this when running the tutorial, probably came from #55

2024-12-12 18:26:09,461 [Rank 0] Traceback (most recent call last):
  File "/app/fast_llm/tools/cli.py", line 29, in fast_llm
    Runnable.parse_and_run(unparsed)
  File "/app/fast_llm/engine/config_utils/runnable.py", line 28, in parse_and_run
    config.validate()
  File "/app/fast_llm/config.py", line 328, in validate
    self._validate()
  File "/app/fast_llm/engine/training/config.py", line 349, in _validate
    super()._validate()
  File "/app/fast_llm/engine/multi_stage/config.py", line 360, in _validate
    self.model = self.model.from_pretrained(self.pretrained, default=self.model)
  File "/app/fast_llm/engine/multi_stage/config.py", line 257, in from_pretrained
    metadata = cls.load_metadata(pretrained)
  File "/app/fast_llm/engine/multi_stage/config.py", line 295, in load_metadata
    metadata = config.format.get_handler_class().load_metadata(config)
  File "/app/fast_llm/engine/checkpoint/external.py", line 172, in load_metadata
    imported_model_config = cls._import_config(cls._load_config(config.path), True)
  File "/app/fast_llm/engine/checkpoint/external.py", line 217, in _import_config
    value = None if converter.export_name is None else get_nested_dict_value(config, converter.export_name)
  File "/app/fast_llm/utils.py", line 243, in get_nested_dict_value
    d = d[key]
TypeError: 'NoneType' object is not subscriptable

🔄 Steps to Reproduce

Run the tutorial

🎯 Expected Behavior

No crash

[Dmmoim]

Same here

[vaibhav016]

@jlamypoirier

I also faced this issue, and i changed the model card to mistral. Can you try this

torchrun --standalone --nnodes 1 --nproc_per_node=4 --no_python fast-llm train gpt --config mistral_4_node_benchmark.yaml

mistral_4_node_benchmark.yaml is taken from examples here https://github.com/ServiceNow/Fast-LLM/blob/main/examples/mistral-4-node-benchmark.yaml

Kindly adjust the nodes as per your configs. This works for me.

[jlamypoirier]

@jlamypoirier

I also faced this issue, and i changed the model card to mistral. Can you try this

The bug is in the config conversion from Hugging Face format, and the example config doesn't use it so should indeed work properly. I'm working on a fix for the conversion, it should be ready later today.

[jlamypoirier]

The problem is with rope_scaling, the new version expects a dict and no longer supports a plain null. Only llama models are affected, other should convert correctly.