Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Type
/
to search
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading.
Please reload this page
.
NVIDIA
/
TransformerEngine
Public
Notifications
You must be signed in to change notification settings
Fork
795
Star
3.5k
Code
Issues
156
Pull requests
157
Discussions
Actions
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Security and quality
Insights
[PyTorch] Adjust checkpointing of FP8 metadata for attention
- #917
#917
Merged
cyanguwa
merged 15 commits into
NVIDIA:main
NVIDIA/TransformerEngine:main
from
cyanguwa:basemodule_dpa
cyanguwa/TransformerEngine:basemodule_dpa
Copy head branch name to clipboard
Jun 15, 2024
Conversation
Commits
15
(15)
Checks
Files changed
Merged
[PyTorch] Adjust checkpointing of FP8 metadata for attention
#917
cyanguwa
merged 15 commits into
NVIDIA:main
NVIDIA/TransformerEngine:main
from
cyanguwa:basemodule_dpa
cyanguwa/TransformerEngine:basemodule_dpa
Copy head branch name to clipboard
Commits
Commits on Jun 11, 2024
subclass DPA with BaseModule and test with test_gpt_checkpointing
Show description for 9f20e86
cyanguwa
committed
9f20e86
View commit details
Copy full SHA for 9f20e86
Browse repository at this point
test DPA only
Show description for 82b2451
cyanguwa
committed
82b2451
View commit details
Copy full SHA for 82b2451
Browse repository at this point
test save and load
Show description for f9ec82e
cyanguwa
committed
f9ec82e
View commit details
Copy full SHA for f9ec82e
Browse repository at this point
Commits on Jun 12, 2024
remove debug info
Show description for 7c7cc74
cyanguwa
committed
7c7cc74
View commit details
Copy full SHA for 7c7cc74
Browse repository at this point
minor tweaks
Show description for c8dbdc9
cyanguwa
committed
c8dbdc9
View commit details
Copy full SHA for c8dbdc9
Browse repository at this point
minor tweak
Show description for 02db239
cyanguwa
committed
02db239
View commit details
Copy full SHA for 02db239
Browse repository at this point
add hook in case core_attention._extra_state is missing
Show description for b2210f8
cyanguwa
committed
b2210f8
View commit details
Copy full SHA for b2210f8
Browse repository at this point
Merge branch 'main' into basemodule_dpa
cyanguwa
authored
dd2e2fa
View commit details
Copy full SHA for dd2e2fa
Browse repository at this point
check named buffers in BaseModule; remove FP8 scratchpad override function; test FP8 for sm90+
Show description for 8d43d71
cyanguwa
committed
8d43d71
View commit details
Copy full SHA for 8d43d71
Browse repository at this point
Merge branch 'main' into basemodule_dpa
cyanguwa
authored
9a3e08f
View commit details
Copy full SHA for 9a3e08f
Browse repository at this point
Commits on Jun 14, 2024
Merge branch 'NVIDIA:main' into basemodule_dpa
cyanguwa
authored
591cb1d
View commit details
Copy full SHA for 591cb1d
Browse repository at this point
[pre-commit.ci] auto fixes from pre-commit.com hooks
Show description for ff3839d
pre-commit-ci[bot]
committed
ff3839d
View commit details
Copy full SHA for ff3839d
Browse repository at this point
minor fixes: test size, interval in recipe, named_buffer loop
Show description for fb88677
cyanguwa
committed
fb88677
View commit details
Copy full SHA for fb88677
Browse repository at this point
Commits on Jun 15, 2024
move BaseModule from FusedAttention to DPA
Show description for dd2cf68
cyanguwa
committed
dd2cf68
View commit details
Copy full SHA for dd2cf68
Browse repository at this point
[pre-commit.ci] auto fixes from pre-commit.com hooks
Show description for 2fa3f36
pre-commit-ci[bot]
committed
2fa3f36
View commit details
Copy full SHA for 2fa3f36
Browse repository at this point
You can’t perform that action at this time.
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=32){kind=avatar}