[Bugfix][PyTorch][FlashAttention][CP] Honor requested FA padding in CP tests - #3269
Conversation
The CP test runner creates inter-sequence padding when the FlashAttention padding case is requested, but it unconditionally told DPA that FlashAttention THD inputs had no padding. That mismatch left CP backward padding uninitialized.\n\nDerive the explicit padding state from the same condition used to generate the inputs and reuse it for both reference and CP calls, preserving the sync-free CUDA-graph path for non-padding cases. Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
for more information, see https://pre-commit.ci
|
@HaochenYuan, #2898 recently changed some assumptions with FlashAttention which broke CI, this is a fix, could you confirm/review that these changes don't affect your use-case? Thanks! |
Greptile SummaryUpdates the context-parallel attention test runner to derive the explicit THD inter-sequence padding state from the same condition used to generate inputs, then passes that state consistently to both reference and CP attention calls. Confidence Score: 5/5The PR appears safe to merge with no actionable defects identified. The new explicit padding state exactly matches the existing THD input-generation condition, preserves None-based handling for non-THD formats, and is reused consistently by both reference and context-parallel calls. Important Files Changed
Reviews (1): Last reviewed commit: "[pre-commit.ci] auto fixes from pre-comm..." | Re-trigger Greptile |
There was a problem hiding this comment.
Pull request overview
Fixes incorrect pad_between_seqs signaling in PyTorch Context Parallel (CP) attention tests for THD inputs when FlashAttention inter-sequence padding is requested, ensuring DPA/CP see the same explicit padding state used to generate inputs (and avoiding the conservative auto-detect behavior that can alter backend eligibility).
Changes:
- Derive
pad_between_seqsforqkv_format="thd"from the same condition used ingenerate_input_shapes. - Reuse the derived
pad_between_seqsfor both the non-CP reference call and the CP call to avoid mismatched padding handling.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/te-ci pytorch L3 |
|
/te-ci pytorch L3 |
Thanks for the fix, and it won't affect my use case~ |
…P tests (#3269) * [PyTorch] Honor requested FA padding in CP tests The CP test runner creates inter-sequence padding when the FlashAttention padding case is requested, but it unconditionally told DPA that FlashAttention THD inputs had no padding. That mismatch left CP backward padding uninitialized.\n\nDerive the explicit padding state from the same condition used to generate the inputs and reuse it for both reference and CP calls, preserving the sync-free CUDA-graph path for non-padding cases. Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The CP test runner creates inter-sequence padding when the FlashAttention padding case is requested, but it unconditionally told DPA that FlashAttention THD inputs had no padding. That mismatch left CP backward padding uninitialized.
Derive the explicit padding state from the same condition used to generate the inputs and reuse it for both reference and CP calls, preserving the sync-free CUDA-graph path for non-padding cases.
Description
Please include a brief summary of the changes, relevant motivation and context.
Fixes # (issue)
Type of change
Changes
Please list the changes introduced in this PR:
Checklist: