Skip to content

fix: disable KV cache reuse if using attention sink - #3021

Merged
Funatiq merged 3 commits into
NVIDIA:mainfrom
Funatiq:disable_sink_reuse
Apr 15, 2025
Merged

fix: disable KV cache reuse if using attention sink#3021
Funatiq merged 3 commits into
NVIDIA:mainfrom
Funatiq:disable_sink_reuse

add comment

f7568c4
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs