Mllama fast image processor - #37539
Conversation
|
cc @yonigozlan |
| expected_empty_tiles = [ | ||
| # sample 1 with 1 image 2x2 grid | ||
| [ | ||
| [False, False, False, False], | ||
| [True, True, True, True], # padding | ||
| ], | ||
| # sample 2 | ||
| [ | ||
| [False, False, False, True], # 3x1 | ||
| [False, True, True, True], # 1x1 | ||
| ], |
There was a problem hiding this comment.
There is a mismatch in shape between fast and slow for this section, where slow append another image as padding for sample 1 but fast return the same number of samples (3) because the images have been flatten to a list beforehand
yonigozlan
left a comment
yonigozlan
left a comment
There was a problem hiding this comment.
Hi @rootonchair ! Sorry for the delay. Indeed for backward compatibility and to work seamlessly with the MLlama model, this processor needs to process nested images. I just opened a PR that improves support for image processor handling nested images, you can have a look here. Once this PR is merged, you can do something similar for MLlama than it's done for Idefics2/3/SmolVlm in the PR.
Sure @yonigozlan, I will proceed when your PR is merged |
|
Hello @rootonchair ! The PR I mentioned was merged btw, if you want this PR another go! |
|
@yonigozlan that's great. I will get back to this PR |
3 participants
What does this PR do?
Related #36978
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.