关于cast_trainable_params_to_fp32导致lora predict发生错误

[wanghuii1]

Reminder

• I have read the README and searched the existing issues.

System Info

LLaMA Factory, version 0.7.2.dev0

Reproduction

问题描述：
在lora sft训练中，如果扩充special token，则adapter_config会将modules_to_save 设置成input和output embedding层。
训练完成后，导入lora推理时，会在
https://github.com/hiyouga/LLaMA-Factory/blob/f8d8690bf4c2981f3151b4ccf07daeb4f3cd38a9/src/llamafactory/model/adapter.py#L173C1-L173C17

将input 和output embedding层设置成trainable，后续会在
https://github.com/hiyouga/LLaMA-Factory/blob/f8d8690bf4c2981f3151b4ccf07daeb4f3cd38a9/src/llamafactory/model/adapter.py#L242
将embedding层设置成fp32，这个是和模型其他参数fp16不一致的，导致推理报参数类型不一致出错

Expected behavior

扩充special token的情况下，训练lora后能正常predict

Others

No response

[wanghuii1]

建议在
https://github.com/hiyouga/LLaMA-Factory/blob/f8d8690bf4c2981f3151b4ccf07daeb4f3cd38a9/src/llamafactory/model/adapter.py#L242
修改为
if cast_trainable_params_to_fp32 and is_trainable:

[hiyouga]

推理命令是什么？

[wanghuii1]

python src/train.py examples/lora_single_gpu/llama3_lora_predict.yaml
后续我尝试了
CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch
--config_file examples/accelerate/single_config.yaml
src/train.py examples/lora_single_gpu/llama3_lora_predict.yaml
虽然input embedding（fp32）和proj weight（fp16）仍然不一致，但是没有报错，可以正常计算

[hiyouga]

已修复