Skip to content

Commit 7ceeaec

Browse files
authored
setting both thresholds to N_sma_threshhold
missed 1 of 2 earlier...
1 parent 1e296e8 commit 7ceeaec

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

ranger.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ def step(self, closure=None):
113113
if group['weight_decay'] != 0:
114114
p_data_fp32.add_(-group['weight_decay'] * group['lr'], p_data_fp32)
115115

116-
if N_sma > 4:
116+
if N_sma > N_sma_threshhold:
117117
denom = exp_avg_sq.sqrt().add_(group['eps'])
118118
p_data_fp32.addcdiv_(-step_size, exp_avg, denom)
119119
else:

0 commit comments

Comments
 (0)