function call 微调后部署问题

[pugnazhaotianqi]

Reminder

• I have read the README and searched the existing issues.

System Info

• llamafactory version: 0.8.4.dev0
• Platform: Linux-6.8.0-40-generic-x86_64-with-glibc2.35
• Python version: 3.10.12
• PyTorch version: 2.3.0a0+ebedce2 (GPU)
• Transformers version: 4.43.4
• Datasets version: 2.20.0
• Accelerate version: 0.32.0
• PEFT version: 0.12.0
• TRL version: 0.9.6
• GPU type: NVIDIA H20
• DeepSpeed version: 0.14.5

Reproduction

使用glaive fuction call数据集微调llama3.1后使用vllm部署调用tools时生成的prompt和原始llama3.1 8b的prompt不一致，tools不会正确组装到system prompt中

Expected behavior

No response

Others

No response

[hiyouga]

fixed