fix internvl flash long context acc

[CUHKSZzxy]

Motivation

The main branch code was tested with the following script, which uses large images that exceed the max_prefill_token_num limit, triggering the is_long_context condition in lmdeploy. As a result, the response will be repeated and meaningless.

Testing script

import os
from lmdeploy import pipeline, PytorchEngineConfig
from lmdeploy.vl import load_image

if __name__ == '__main__':
    os.environ['CUDA_VISIBLE_DEVICES'] = '4'

    # configurations
    tp = 1
    backend_config = PytorchEngineConfig(
        tp=tp,
        # max_prefill_token_num=10240
    )

    # init pipeline
    model_path = 'InternVL3_5-8B-Flash'
    pipe = pipeline(
        model_path,
        backend_config=backend_config,
        log_level='INFO'
    )

    # inference
    messages = [
        dict(role='user', content=[
            dict(type='text', text='<IMAGE_TOKEN>\n<IMAGE_TOKEN>\nDescribe the two images in detail.'),
            dict(type='image_url', image_url=dict(url='img1.jpeg')),
            dict(type='image_url', image_url=dict(url='img2.jpeg')),
        ])
    ]

    response = pipe(messages)
    print(response)

Performance & Acc

Test with VLMEvalKit, dataset: BLINK

Model	Flash Mode	Time	Acc
InternVL3.5-8B	/	~3m06s	52.66
InternVL3.5-8B-Flash	false	~3m06s	52.91
InternVL3.5-8B-Flash	true	~2m51s	51.44

[grimoire]

LGTM

[lvhan028]

May add the evaution result of internvl3.5-241b

[CUHKSZzxy]

May add the evaution result of internvl3.5-241b

Test with VLMEvalKit, dataset: BLINK

Model	Flash Mode	Time	Acc
InternVL3.5-241B-A28B	/	/	61.4
241B-A8B-Flash	false	~7m12s	61.3
241B-A8B-Flash	true	~7m02s	59.7

InternVL3.5-241B-A28B results are taken from the official InternVL3.5 report Table 5.
https://arxiv.org/pdf/2508.18265