Support OpenAI and Anthropic message processors

[lumincui]

What changed

• normalize OpenAI assistant.tool_calls into SWIFT tool_call messages
• normalize Anthropic tool_use and tool_result content blocks into SWIFT canonical roles
• add explicit OpenAIMessagesPreprocessor and AnthropicMessagesPreprocessor classes
• auto-detect provider tool-call formats in MessagesPreprocessor
• document both accepted formats in English and Chinese

Why

MessagesPreprocessor._check_messages keeps only role, content, loss, and loss_scale. OpenAI-style tool calls stored in assistant.tool_calls were therefore silently removed during dataset preprocessing even though templates can encode tool calls. This can produce training examples that retain assistant narration but lose the actual tool-call targets.

Anthropic tool-use traces similarly need their content blocks expanded into SWIFT's canonical assistant, tool_call, and tool_response roles before template encoding.

Impact

OpenAI and Anthropic agent traces can now be passed through the normal dataset preprocessing path without an external conversion script. Parallel OpenAI tool calls, string or object arguments, Anthropic text/tool interleaving, nested text tool results, and loss metadata are covered.

The regression test also decodes supervised labels and asserts that the tool name is present, rather than only checking that template encoding succeeds.

Validation

• pre-commit run --all-files
• python -m unittest tests.general.test_data_preprocess.TestProviderMessagesPreprocess tests.general.test_data_preprocess.TestRejectedMessagesPreprocess tests.general.test_data_preprocess.TestDataPreprocess.test_tool_message

Related to #8327 and #9084.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[lumincui]

Thanks @ezeli and @HGY19854338008 for raising #8327 and #9084 and providing concrete OpenAI tool-call examples — they were very helpful when shaping this change.

This PR adds normalization for OpenAI assistant.tool_calls and Anthropic tool-use content blocks while continuing to use the existing MessagesPreprocessor pipeline. @ezeli and @HGY19854338008, if you have time, it would be great if you could check whether the OpenAI behavior matches what you had in mind.

@Jintao-Huang @hjh0119 @tastelikefeet, when convenient, we would also appreciate your guidance on whether MessagesPreprocessor is the right integration point for these provider formats. Happy to adjust the design based on your feedback. Thank you!

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[tastelikefeet]

Thanks! I will check this code asap

[lumincui]

Hi @tastelikefeet, thank you again for the helpful review. The requested multimodal handling, documentation examples, and regression coverage have now been added, and the focused tests and pre-commit checks pass.

When convenient, would you mind taking another look? We would really appreciate having this merged into main when it is ready, as our system is currently carrying these changes as a downstream patch version. We are happy to make any further adjustments you think are needed. Thank you!