Enhance multi-modal for vllm model by chenyushuo · Pull Request #547 · agentscope-ai/Trinity-RFT

chenyushuo · 2026-05-22T09:16:31Z

Description

As the title says.

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

2. enhance logger in checkpoint manager

pan-x-c · 2026-05-25T09:00:20Z

Don't add large data files into git repository
Write a function to automatically generate a data file would be better

…ance_mm_for_vllm_model

chenyushuo · 2026-05-26T05:29:17Z

/unittest-module-common

chenyushuo · 2026-05-26T05:29:24Z

/unittest-module-buffer

chenyushuo · 2026-05-26T05:29:34Z

/unittest-module-explorer

github-actions · 2026-05-26T06:28:48Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
82	80	1	1	0	0	31m 26s

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/common/vllm_test.py::TestTinkerAPI::test_tinker_api	The test failed in the call phase

Skipped

Tests	Status
tests/common/vllm_test.py::TestTinkerAsyncAPIServer::test_api_async	skipped ⏭️

Tests

Test Name	Status	Duration
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	15.9s
tests/common/config_test.py::TestConfig::test_chat_template_path	✅	98ms
tests/common/config_test.py::TestConfig::test_config_flatten	✅	37ms
tests/common/config_test.py::TestConfig::test_continue_from_checkpoint_is_valid	✅	522ms
tests/common/config_test.py::TestConfig::test_default_workflow	✅	97ms
tests/common/config_test.py::TestConfig::test_inference_model_base_port_falls_back_when_unavailable	✅	2ms
tests/common/config_test.py::TestConfig::test_inference_model_base_port_uses_engine_id	✅	1ms
tests/common/config_test.py::TestConfig::test_inference_model_random_port_can_use_port_reserved_by_api_server	✅	1ms
tests/common/config_test.py::TestConfig::test_inference_model_random_port_ignores_base_port	✅	1ms
tests/common/config_test.py::TestConfig::test_inference_model_without_base_port_uses_ephemeral_port	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	359ms
tests/common/config_test.py::TestConfig::test_max_token_len_per_gpu_set_correctly	✅	97ms
tests/common/config_test.py::TestConfig::test_multinode_inference_is_rejected_for_non_vllm_sglang_engines	✅	37ms
tests/common/config_test.py::TestConfig::test_multinode_vllm_config_is_valid	✅	36ms
tests/common/config_test.py::TestConfig::test_multinode_vllm_requires_full_node_occupancy	✅	35ms
tests/common/config_test.py::TestConfig::test_multinode_vllm_requires_matching_nnodes_for_full_nodes	✅	37ms
tests/common/config_test.py::TestConfig::test_multinode_vllm_requires_nnodes_within_cluster_size	✅	36ms
tests/common/config_test.py::TestConfig::test_optimizer_config_propagation	✅	97ms
tests/common/config_test.py::TestConfig::test_update_config_from_ray_cluster	✅	1.7s
tests/common/experience_extraction_test.py::TestExperienceExtraction::test_convert_completion_output_extracts_sglang_routed_experts	✅	1ms
tests/common/experience_extraction_test.py::TestExperienceExtraction::test_convert_completion_output_ignores_invalid_routed_experts_shape	✅	1ms
tests/common/experience_test.py::TestEID::test_eid_properties	✅	1ms
tests/common/experience_test.py::TestExperience::test_action_mask_and_logprobs_type	✅	1ms
tests/common/experience_test.py::TestExperience::test_assertions	✅	1ms
tests/common/experience_test.py::TestExperience::test_build_experience_token_view_aligns_prompt_action_mask_and_logprobs	✅	1ms
tests/common/experience_test.py::TestExperience::test_deserialize_legacy_pickle_payload	✅	2ms
tests/common/experience_test.py::TestExperience::test_deserialize_single_rejects_batch_payload	✅	1ms
tests/common/experience_test.py::TestExperience::test_dpo_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_format_colored_tokens_uses_action_mask	✅	1ms
tests/common/experience_test.py::TestExperience::test_format_colored_tokens_uses_decoded_token_text	✅	1ms
tests/common/experience_test.py::TestExperience::test_hf_datasets_conversion	✅	16ms
tests/common/experience_test.py::TestExperience::test_multi_turn_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_print_colored_tokens_writes_to_file	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_deserialize	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_deserialize_with_routed_experts	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_many_deserialize_many	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_many_deserialize_many_with_routed_experts	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_many_with_shared_multimodal_tensor	✅	1ms
tests/common/experience_test.py::TestExperience::test_single_turn_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_to_dict	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/external_model_test.py::TestExternalModel::test_external_model_load	✅	825ms
tests/common/models/utils_test.py::TestTokenizer::test_action_mask	✅	638ms
tests/common/models/utils_test.py::TestTokenizer::test_action_mask_with_tools	✅	267ms
tests/common/models/utils_test.py::TestTokenizeAndMaskMessagesDefault::test_first_message_is_assistant	✅	1.1s
tests/common/models/utils_test.py::TestTokenizeAndMaskMessagesDefault::test_messages_empty	✅	715ms
tests/common/models/utils_test.py::TestTokenizeAndMaskMessagesDefault::test_mm_messages	✅	1.3s
tests/common/models/utils_test.py::TestTokenizeAndMaskMessagesDefault::test_no_assistant_messages	✅	721ms
tests/common/models/utils_test.py::TestTokenizeAndMaskMessagesDefault::test_normal_conversation_data	✅	1.0s
tests/common/sglang_test.py::TestSGLangOpenAIAPI_0::test_chat_completions	✅	2m 50s
tests/common/sglang_test.py::TestSGLangOpenAIAPI_1::test_chat_completions	✅	1m 6s
tests/common/sglang_test.py::TestSGLangOpenAIAPI_2::test_chat_completions	✅	55.7s
tests/common/sudoku_test.py::test_9x9_generator_produces_valid_solution	✅	2ms
tests/common/sudoku_test.py::test_9x9_generator_creates_holes	✅	1ms
tests/common/sudoku_test.py::test_9x9_solution_is_fully_filled	✅	1ms
tests/common/sudoku_test.py::test_judge_allows_incomplete_board	✅	1ms
tests/common/sudoku_test.py::test_judge_detects_row_violation	✅	1ms
tests/common/sudoku_test.py::test_judge_detects_column_violation	✅	1ms
tests/common/sudoku_test.py::test_judge_detects_block_violation	✅	1ms
tests/common/sudoku_test.py::test_4x4_generator_produces_valid_solution	✅	1ms
tests/common/sudoku_test.py::test_4x4_solution_is_fully_filled	✅	1ms
tests/common/sudoku_test.py::test_4x4_judge_detects_row_violation	✅	1ms
tests/common/sudoku_test.py::test_4x4_judge_detects_block_violation	✅	1ms
tests/common/vllm_test.py::ModelWrapperTest_0::test_generate	✅	1m 11s
tests/common/vllm_test.py::ModelWrapperTest_1::test_generate	✅	1m
tests/common/vllm_test.py::ModelWrapperTest_2::test_generate	✅	3m 19s
tests/common/vllm_test.py::TestModelLen_0::test_model_len	✅	55.0s
tests/common/vllm_test.py::TestModelLen_1::test_model_len	✅	54.8s
tests/common/vllm_test.py::TestModelLen_2::test_model_len	✅	54.8s
tests/common/vllm_test.py::TestModelLenWithoutPromptTruncation::test_model_len	✅	55.4s
tests/common/vllm_test.py::TestMessageProcess::test_no_prompt_truncation	✅	55.7s
tests/common/vllm_test.py::TestMessageProcess::test_truncation_status	✅	54.5s
tests/common/vllm_test.py::TestAPIServer::test_api	✅	42.7s
tests/common/vllm_test.py::TestQwen35APIServerReasoning::test_reasoning_content	✅	2m 30s
tests/common/vllm_test.py::TestQwen35APIServerMultiModal::test_multi_modal_content	✅	2m 35s
tests/common/vllm_test.py::TestLogprobs::test_logprobs_api	✅	35.3s
tests/common/vllm_test.py::TestAsyncAPIServer::test_api_async	✅	36.7s
tests/common/vllm_test.py::TestTinkerAsyncAPIServer::test_api_async	⏭️	1ms
tests/common/vllm_test.py::TestAPIServerToolCall_0_deepseek_r1::test_api_tool_calls	✅	2m
tests/common/vllm_test.py::TestAPIServerToolCall_1::test_api_tool_calls	✅	1m 59s
tests/common/vllm_test.py::TestSuperLongGeneration::test_generate	✅	2m 39s
tests/common/vllm_test.py::TestTinkerAPI::test_tinker_api	❌	1m 14s

Github Test Reporter by CTRF 💚

…ance_mm_for_vllm_model

chenyushuo · 2026-05-26T06:30:07Z

/unittest-module-buffer

github-actions · 2026-05-26T06:35:21Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
49	49	0	0	0	0	2m 39s

Github Test Reporter by CTRF 💚

chenyushuo added 5 commits May 21, 2026 16:55

1. enhance RewardSTDFilter

536de52

2. enhance logger in checkpoint manager

enhance mm for vllm model

12f0972

fix multi modal sft formatter

3572563

add multi_modal_inputs for history recording

968f5c2

refactor multi_modal_inputs extraction

09c5b9c

pan-x-c reviewed May 25, 2026

View reviewed changes

chenyushuo added 4 commits May 25, 2026 20:36

add unittest for mm api server

f31f7c5

Merge branch 'main' of github.com:modelscope/Trinity-RFT into dev/enh…

f4c70bf

…ance_mm_for_vllm_model

fix unittest

f761719

update docs

99c6ebe

chenyushuo changed the title ~~[WIP] Enhance multi-modal for vllm model~~ Enhance multi-modal for vllm model May 26, 2026

Merge branch 'main' of github.com:modelscope/Trinity-RFT into dev/enh…

7bf8f35

…ance_mm_for_vllm_model

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance multi-modal for vllm model#547

Enhance multi-modal for vllm model#547
chenyushuo wants to merge 10 commits into
agentscope-ai:mainfrom
chenyushuo:dev/enhance_mm_for_vllm_model

chenyushuo commented May 22, 2026

Uh oh!

pan-x-c May 25, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

chenyushuo commented May 22, 2026

Description

Checklist

Uh oh!

pan-x-c May 25, 2026

Choose a reason for hiding this comment

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

Summary

Failed Tests

Skipped

Tests

Uh oh!

chenyushuo commented May 26, 2026

Uh oh!

github-actions Bot commented May 26, 2026

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants