Skip to content

Commit 564fc93

Browse files
authored
Update README.md
1 parent 90697d1 commit 564fc93

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,16 @@
11
# Ranger-Deep-Learning-Optimizer
22
Ranger - a synergistic optimizer combining RAdam (Rectified Adam) and LookAhead in one codebase.
33

4-
Latest version 9.3.19 - full refactoring for slow weights and one pass handling (vs two before). Refactor should eliminate any random save/load issues regarding memory.
4+
Latest version 9.3.19 - full refactoring for slow weights and one pass handling (vs two before). Refactor should eliminate any random save/load issues regarding memory.
5+
6+
----------------------------------------------
7+
**Beta Version - Ranger913A.py:
8+
9+
For anyone who wants to try this out early, this version changes from RAdam to using calibrated anistropic adaptive learning rate per this paper:
10+
https://arxiv.org/abs/1908.00700v2
11+
"Empirical studies support our observation of the anisotropic A-LR and show that the proposed methods outperform existing AGMs and generalize even better than S-Momentum in multiple deep learning tasks."
12+
Initial testing looks very good for training stabilization. Any feedback in comparsion with current Ranger (9.3.19) is welcome!
13+
----------------------------------------------
514

615
Medium article with more info:
716
https://medium.com/@lessw/new-deep-learning-optimizer-ranger-synergistic-combination-of-radam-lookahead-for-the-best-of-2dc83f79a48d

0 commit comments

Comments
 (0)