[#6186][feat] Introduce QKNormRoPEAttention module - #6830

Merged

Funatiq merged 7 commits into

NVIDIA:mainfrom

Funatiq:dev/feat/qk_norm

Sep 5, 2025

Commits on Sep 3, 2025

[feat] Introduce QKNormRoPEAttention class for enhanced attention mechanism in Qwen3 model
Funatiq
committed
[feat] Refactor QKNormRoPEAttention initialization in Qwen3 model layers
Funatiq
committed
[feat] Add SPDX license headers to QKNormAttention module
Funatiq
committed
[feat] Restore Qwen3Attention class as a subclass of QKNormRoPEAttention
Funatiq
committed
[feat] Refactor Gemma3Attention to inherit from QKNormRoPEAttention
Funatiq
committed
[feat] Refactor Exaone4Attention to inherit from QKNormRoPEAttention
Funatiq
committed
[fix] Update yarn parameter computation in QK normalization attention
Funatiq
committed