Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Type
/
to search
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading.
Please reload this page
.
NVIDIA
/
TransformerEngine
Public
Notifications
You must be signed in to change notification settings
Fork
798
Star
3.5k
Code
Issues
159
Pull requests
161
Discussions
Actions
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Security and quality
Insights
[PyTorch] FP8 MHA with RoPE and Miscellaneous Improvements
- #1100
#1100
Merged
yaox12
merged 24 commits into
NVIDIA:main
NVIDIA/TransformerEngine:main
from
yaox12:xiny/fp8_mha_with_rope
yaox12/TransformerEngine:xiny/fp8_mha_with_rope
Copy head branch name to clipboard
Sep 5, 2024
Conversation
Commits
24
(24)
Checks
Files changed
Merged
[PyTorch] FP8 MHA with RoPE and Miscellaneous Improvements
#1100
yaox12
merged 24 commits into
NVIDIA:main
NVIDIA/TransformerEngine:main
from
yaox12:xiny/fp8_mha_with_rope
yaox12/TransformerEngine:xiny/fp8_mha_with_rope
Copy head branch name to clipboard
Commits
Commits on Aug 15, 2024
fp8 mha with rope
Show description for 59b99ca
yaox12
committed
59b99ca
View commit details
Copy full SHA for 59b99ca
Browse repository at this point
avoid index select in cast ops
Show description for c46f82c
yaox12
committed
c46f82c
View commit details
Copy full SHA for c46f82c
Browse repository at this point
avoid index select in fused_attn_fwd
Show description for dafd73f
yaox12
committed
dafd73f
View commit details
Copy full SHA for dafd73f
Browse repository at this point
rename is_first_module_in_mha to fp8_output
Show description for 0d2ff34
yaox12
committed
0d2ff34
View commit details
Copy full SHA for 0d2ff34
Browse repository at this point
resolve comments
Show description for 0e837c3
yaox12
committed
0e837c3
View commit details
Copy full SHA for 0e837c3
Browse repository at this point
resolve comments
Show description for 33c3ed6
yaox12
committed
33c3ed6
View commit details
Copy full SHA for 33c3ed6
Browse repository at this point
[pre-commit.ci] auto fixes from pre-commit.com hooks
Show description for 13feabb
pre-commit-ci[bot]
committed
13feabb
View commit details
Copy full SHA for 13feabb
Browse repository at this point
Commits on Aug 16, 2024
move transpose to backward for fp8 input
Show description for ae856e4
yaox12
committed
ae856e4
View commit details
Copy full SHA for ae856e4
Browse repository at this point
Commits on Aug 19, 2024
fix ut
Show description for 7e26d22
yaox12
committed
7e26d22
View commit details
Copy full SHA for 7e26d22
Browse repository at this point
Commits on Aug 21, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
committed
fae44b6
View commit details
Copy full SHA for fae44b6
Browse repository at this point
resolve comments
Show description for 521c77a
yaox12
committed
521c77a
View commit details
Copy full SHA for 521c77a
Browse repository at this point
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
committed
10c6961
View commit details
Copy full SHA for 10c6961
Browse repository at this point
update argument list for CP
Show description for dd30c2d
yaox12
committed
dd30c2d
View commit details
Copy full SHA for dd30c2d
Browse repository at this point
[pre-commit.ci] auto fixes from pre-commit.com hooks
Show description for a94b3ad
pre-commit-ci[bot]
committed
a94b3ad
View commit details
Copy full SHA for a94b3ad
Browse repository at this point
Commits on Aug 26, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
committed
bf56399
View commit details
Copy full SHA for bf56399
Browse repository at this point
fix for FA3
Show description for 400d526
yaox12
committed
400d526
View commit details
Copy full SHA for 400d526
Browse repository at this point
[pre-commit.ci] auto fixes from pre-commit.com hooks
Show description for b935e13
pre-commit-ci[bot]
committed
b935e13
View commit details
Copy full SHA for b935e13
Browse repository at this point
remove unnecessary copy of scale_inv
Show description for 9eca369
yaox12
committed
9eca369
View commit details
Copy full SHA for 9eca369
Browse repository at this point
Commits on Aug 27, 2024
skip fp8 dpa/mha tests when fa3 is not available
Show description for e3b75db
yaox12
committed
e3b75db
View commit details
Copy full SHA for e3b75db
Browse repository at this point
Commits on Aug 28, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
authored
46d428f
View commit details
Copy full SHA for 46d428f
Browse repository at this point
Commits on Aug 30, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
committed
6b80dd6
View commit details
Copy full SHA for 6b80dd6
Browse repository at this point
Commits on Sep 3, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
authored
df6132f
View commit details
Copy full SHA for df6132f
Browse repository at this point
Commits on Sep 4, 2024
Merge branch 'main' into xiny/fp8_mha_with_rope
yaox12
authored
c017154
View commit details
Copy full SHA for c017154
Browse repository at this point
fix a merge bug
Show description for f9da6d7
yaox12
committed
f9da6d7
View commit details
Copy full SHA for f9da6d7
Browse repository at this point
You can’t perform that action at this time.
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=32){kind=avatar}