WIP, reference modeling

[molbap]

What does this PR do?

WIP, too many divergences in code/arch for models that are very similar, the idea is to have a "dummy" model that is functional, based on a recent omni model, and then derive functional blocks from it through modular. Pushing this as a PR to commit to it and build this up

Currently VLMs+audio do not exploit enough modular IMO

   - qwen2_5_omni         (4006 lines in modeling, 4307 in modular)                                                            
   - gemma3n              (2391 lines in modeling, 2659 in modular)                                                            
   - d_fine               (2191 lines in modeling, 1213 in modular)                                                            
   - informer             (2162 lines in modeling, 978 in modular)                                                             
   - rt_detr_v2           (2001 lines in modeling, 628 in modular)                                                             
   - wav2vec2_conformer   (1934 lines in modeling, 730 in modular)                                                             
   - phi4_multimodal      (1897 lines in modeling, 1744 in modular)
   - instructblipvideo    (1792 lines in modeling, 657 in modular)                                                             
   - zamba2               (1742 lines in modeling, 1151 in modular)                                                            
   - qwen2_5_vl           (1734 lines in modeling, 1061 in modular)                                                            
   - plbart               (1722 lines in modeling, 663 in modular)                                                                                                                                                                                            
   - glm4v                (1707 lines in modeling, 1724 in modular)                                                            
   - falcon_h1            (1625 lines in modeling, 1377 in modular)                                                            
   - emu3                 (1624 lines in modeling, 1206 in modular)                                                            
   - wav2vec2_bert        (1517 lines in modeling, 1070 in modular)                                                            
   - bamba                (1511 lines in modeling, 1219 in modular)                                                            
   - sam_hq               (1497 lines in modeling, 653 in modular)                                                             
   - janus                (1422 lines in modeling, 1592 in modular)                                                            
   - modernbert           (1398 lines in modeling, 1527 in modular)                                                            
   - t5gemma              (1388 lines in modeling, 1257 in modular)                                                            
   - gemma3               (1301 lines in modeling, 1184 in modular)                                                            
   - siglip2              (1301 lines in modeling, 637 in modular)                                                             
   - aria                 (1300 lines in modeling, 1665 in modular)                                                            
   - minimax              (1142 lines in modeling, 602 in modular)                                                                                                                                                                                            
   - moonshine            (1096 lines in modeling, 920 in modular)                                                             
   - csm                  (1091 lines in modeling, 768 in modular)                 

• many do not have a modular file (yet)

   - seamless_m4t_v2      (4402 lines in modeling file)                                                                        
   - seamless_m4t         (4073 lines in modeling file)                                                                                                                                                                                                       
   - perceiver            (3406 lines in modeling file)                                                                        
   - speecht5             (3247 lines in modeling file)                                                                        
   - oneformer            (3202 lines in modeling file)                                                                        
   - bigbird_pegasus      (3031 lines in modeling file)                                                                        
   - big_bird             (2960 lines in modeling file)                                                                        
   - reformer             (2776 lines in modeling file)                                                                        
   - led                  (2536 lines in modeling file)                                                                        
   - moshi                (2512 lines in modeling file)
   - mask2former          (2476 lines in modeling file)
   - blip_2               (2459 lines in modeling file)
   - mt5                  (2457 lines in modeling file)
   - musicgen             (2439 lines in modeling file)
   - t5                   (2408 lines in modeling file)
   - xlnet                (2380 lines in modeling file)
   - wav2vec2             (2360 lines in modeling file)
   - tapas                (2355 lines in modeling file)
   - musicgen_melody      (2273 lines in modeling file)
   - auto                 (2241 lines in modeling file)
   - longformer           (2223 lines in modeling file)
   - longt5               (2196 lines in modeling file)
   - luke                 (2180 lines in modeling file)
   - autoformer           (2123 lines in modeling file)
   - patchtsmixer         (2122 lines in modeling file)
   - prophetnet           (2048 lines in modeling file)
   - flava                (2039 lines in modeling file)
   - udop                 (2006 lines in modeling file)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.