Skip to content

Commit c598f42

Browse files
titaiwangmsCopilot
andcommitted
docs(example): clarify nonpad_kv_seqlen is the after-scatter valid count
Update the module-docstring bullet to match the fix's after-scatter semantics: nonpad_kv_seqlen is the number of valid KV entries after the current chunk is scattered (write_indices + cur_seq_len), not a generic running count. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: titaiwang <titaiwang@microsoft.com>
1 parent 2a73352 commit c598f42

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

examples/static_cache_generation.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,8 @@
1313
- No ``attention_mask`` input — causal masking is handled internally.
1414
- 3-D cache shape ``[batch, max_seq_len, kv_hidden]`` (not 4-D).
1515
- ``write_indices`` tracks where to write the next token's KV entry.
16-
- ``nonpad_kv_seqlen`` tracks how many valid KV entries exist.
16+
- ``nonpad_kv_seqlen`` is the number of valid KV entries after the current
17+
chunk is scattered into the cache (``write_indices + cur_seq_len``).
1718
- Outputs are ``updated_key_cache.{i}`` / ``updated_value_cache.{i}``.
1819
1920
Usage::

0 commit comments

Comments
 (0)