Skip to content

Add mm processor args for Qwen3-VL#4196

Merged
lvhan028 merged 6 commits intoInternLM:mainfrom
CUHKSZzxy:add-processor-args
Dec 10, 2025
Merged

Add mm processor args for Qwen3-VL#4196
lvhan028 merged 6 commits intoInternLM:mainfrom
CUHKSZzxy:add-processor-args

Conversation

@CUHKSZzxy
Copy link
Copy Markdown
Collaborator

@CUHKSZzxy CUHKSZzxy commented Dec 10, 2025

Motivation

User should be able to configure mm processor parameters for Qwen3-VL, e.g. min_pixels, max_pixels etc.
Therefore, we add a general mm processor kwargs argument for Qwen3-VL and possibly more VLMs.

https://github.com/QwenLM/Qwen3-VL?tab=readme-ov-file#new-qwen-vl-utils-usage

Usage

Adding mm_processor_kwargs into the message

curl http://0.0.0.0:23333/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-8B-Instruct",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Can you describe these images?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg"
                    }
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
                    }
                }
            ]
        }
    ],
    "add_vision_id": true,
    "mm_processor_kwargs": {
        "min_pixels": 50176,
        "max_pixels": 50176
    }
  }'

Others

Remove deprecated env DEEPEP_MAX_BATCH_SIZE, refers to

@CUHKSZzxy CUHKSZzxy requested a review from lvhan028 December 10, 2025 04:34
enable_thinking=request.enable_thinking,
add_vision_id=request.add_vision_id,
)
mm_processor_kwargs=request.mm_processor_kwargs)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add this option to "/generate" endpoint? cc @irexyc @RunningLeon

Comment on lines +44 to +49
async def preprocess(self,
messages: List[Dict],
mm_processor_kwargs: Optional[Dict[str, Any]] = None) -> List[Dict]:
"""Preprocess multimodal data in the messages."""
future = asyncio.get_event_loop().run_in_executor(self.executor, self.model.preprocess, messages)
future = asyncio.get_event_loop().run_in_executor(self.executor, self.model.preprocess, messages,
mm_processor_kwargs)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changed the number of arguments for self.model.preprocess, will it affect other models like internvl? https://github.com/CUHKSZzxy/lmdeploy/blob/44f0942078bfb5a98da12743f4529f4492636029/lmdeploy/vl/model/internvl.py#L193

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it does. pr-test CI failed

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add mm_processor_kwargs for all the rest vision models? or any other better solution

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's address the internvl and qwen-vl series model first

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's address the internvl and qwen-vl series model first

Have resolved this, by first checking if self.model.preprocess accpets mm_processor_kwargs or not. See the updated code.

@lvhan028 lvhan028 merged commit a34db18 into InternLM:main Dec 10, 2025
5 of 7 checks passed
@CUHKSZzxy CUHKSZzxy deleted the add-processor-args branch December 10, 2025 11:55
@CUHKSZzxy CUHKSZzxy restored the add-processor-args branch December 10, 2025 11:55
@CUHKSZzxy CUHKSZzxy deleted the add-processor-args branch December 10, 2025 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants