Skip to content

[None] Add the torch source implementation for new params. - #89

Merged
nvchenghaoz merged 3 commits into
feat/ad-2025-07-07from
chenghao/torch-ref-0715
Jul 17, 2025
Merged

[None] Add the torch source implementation for new params.#89
nvchenghaoz merged 3 commits into
feat/ad-2025-07-07from
chenghao/torch-ref-0715

Conversation

@nvchenghaoz

@nvchenghaoz nvchenghaoz commented Jul 16, 2025

Copy link
Copy Markdown

Since we added logit cap / attention sinks / sliding window for the attention backend operators, update the torch source implementation for new these parameters.

@nvchenghaoz
Add the torch ref implementation for new params.
23d03c7 
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
@nvchenghaoz nvchenghaoz self-assigned this Jul 16, 2025
@nvchenghaoz
Remove comment
ded8f9a 
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
@nvchenghaoz
nvchenghaoz enabled auto-merge (squash) July 16, 2025 18:33
lucaslie
lucaslie approved these changes Jul 17, 2025
View reviewed changes

@lucaslie lucaslie left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. One follow-up task that could be interesting is to unify the bsnd and bnsd implementation and just use an argument for that.

Let's wait for #88 to land and then we can discuss further

@nvchenghaoz
nvchenghaoz merged commit 72b9ee9 into feat/ad-2025-07-07 Jul 17, 2025
1 check passed
lucaslie pushed a commit that referenced this pull request Jul 18, 2025
@nvchenghaoz @lucaslie
[None] Add the torch source implementation for new params. (#89)
c245cf3 
* Add the torch ref implementation for new params.

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Remove comment

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

---------

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
lucaslie added a commit that referenced this pull request Jul 18, 2025
@lucaslie
Revert "[None] Add the torch source implementation for new params. (#89…
1f5e88c 
…)"

This reverts commit c245cf3.
lucaslie pushed a commit that referenced this pull request Jul 21, 2025
@nvchenghaoz @lucaslie
[None] Add the torch source implementation for new params. (#89)
5b69d2c 
* Add the torch ref implementation for new params.

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Remove comment

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

---------

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
lucaslie added a commit that referenced this pull request Jul 21, 2025
@lucaslie
Revert "[None] Add the torch source implementation for new params. (#89…
810df7d 
…)"

This reverts commit c245cf3.
@nvchenghaoz nvchenghaoz mentioned this pull request Jul 23, 2025
Merged
nvchenghaoz added a commit that referenced this pull request Jul 26, 2025
@nvchenghaoz @Fridah-nv
[Update #89] Add torch ref attention (#107)
7810dd0 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
MrGeva pushed a commit that referenced this pull request Jul 29, 2025
@nvchenghaoz @Fridah-nv @MrGeva
[Update #89] Add torch ref attention (#107)
067989b 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
h-guo18 pushed a commit that referenced this pull request Jul 30, 2025
@nvchenghaoz @Fridah-nv @h-guo18
[Update #89] Add torch ref attention (#107)
079aaa3 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
lucaslie pushed a commit that referenced this pull request Jul 31, 2025
@nvchenghaoz @Fridah-nv @lucaslie
[Update #89] Add torch ref attention (#107)
6861964 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Fridah-nv added a commit that referenced this pull request Jul 31, 2025
@nvchenghaoz @Fridah-nv
[Update #89] Add torch ref attention (#107)
a6b6113 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Fridah-nv added a commit that referenced this pull request Jul 31, 2025
@nvchenghaoz @Fridah-nv @h-guo18
[Update #89] Add torch ref attention (#107)
3301337 
* attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>

* Update the torch ref op

Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>

* Revert "attention matcher with torch._inductor pattern matcher,matching repeat kv, sdpa and group attention, update unit tests"

This reverts commit 5743fb3.

---------

Signed-off-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Signed-off-by: nvchenghaoz <211069071+nvchenghaoz@users.noreply.github.com>
Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@suyoggupta suyoggupta Awaiting requested review from suyoggupta
@Fridah-nv Fridah-nv Awaiting requested review from Fridah-nv
1 more reviewer
@lucaslie lucaslie lucaslie approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@nvchenghaoz nvchenghaoz

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nvchenghaoz @lucaslie