XNNPack linear kernel for group size 16 / padding weights

[jackzhxng]

🚀 The feature, motivation and pitch

For models with hidden dims that are indivisible by 32. Padding up to 32 could also work.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @digantdesai @mcr229 @cbilgin

[digantdesai]

@jackzhxng do you have a model and/or a test case? Is there a timeline we want to enable this by?

[jackzhxng]

Gemma3 4B, so we can quantize the encoder. Would be good to have by GA because 8GB .pte at the moment makes it a bit hard to run on-device

[digantdesai]

So I assume AoT padding doesn't help with memory so we need to solve this at the kernel level, right?

[jackzhxng]

I think padding could work, that was a suggestion by @metascroy