Commit 8188ca1
fix: add Q/K norm handling to BitNetAttention.forward
Address Copilot reviewer comment: BitNetAttention.forward was missing
the optional Q/K normalization path (q_norm/k_norm) that the base
Attention class supports. Added the full q_norm/k_norm handling
(both full and per-head modes) between Q/K/V projections and RoPE,
matching the base Attention.forward logic.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>1 parent be515c4 commit 8188ca1
1 file changed
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
64 | 77 | | |
65 | 78 | | |
66 | 79 | | |
| |||
0 commit comments