feat: add pytorch_engine_qwen2_5vl_sm120#3750
Conversation
|
|
||
| num_warps = 4 | ||
| if _nv_cap[0] < 8: | ||
| if _nv_cap[0] >= 12: # Blackwell (sm_120 etc.) |
There was a problem hiding this comment.
Would you please follow the original style to enable Blackwell support when _nv_cap[0] < 13 and put this branch to the end of the if-elif-else block? I think this will help the community better understand the code.
|
And you can try to install pre-commit to help you fix lint errors. Thank you for your efforts! |
|
Thank you for your helpful review and suggestions 🙏 Please take a look when you have time — really appreciate your support! |
test.py
Outdated
| from lmdeploy import PytorchEngineConfig, pipeline | ||
| from lmdeploy.vl import load_image | ||
|
|
||
| backend_config = PytorchEngineConfig(session_len=16384) |
There was a problem hiding this comment.
Hi, @kolmogorov-quyet
We appreciate your contribution. Just a quick note—the lmdeploy/tests directory is intended for unit test cases rather than functional testing.
We've already integrated Qwen2.5-VL model testing into lmdeploy's functional test suite, so you can safely remove this test file
Motivation
CancelledErrorlogs fromasyncio.run_coroutine_threadsafewhen the future is cancelled..gitignoreentries.Modifications
builder/,lmdeploy/lib/, IDE caches, etc.lmdeploy/pytorch/kernels/cuda/flashattention.py_kernel_meta_sm128(), refactored meta-selection logic (~40 LoC).lmdeploy/serve/async_engine.pylambda f: f.result()with a safe callback:lambda f: None if f.cancelled() else f.result()tests/test.pyBackward Compatibility
No API breaks; existing engines and interfaces continue to work.
Use Cases
CancelledErrorspam.Checklist
pytest -q)