Skip to content

Commit

Permalink
Merge pull request #73 from dpaetzel/fix-cosanneal-docstring
Browse files Browse the repository at this point in the history
  • Loading branch information
darsnack authored Nov 11, 2024
2 parents 98d072d + a445060 commit a635001
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/cyclic.jl
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ A cosine annealing schedule
(see ["SGDR: Stochastic Gradient Descent with Warm Restarts"](https://arxiv.org/abs/1608.03983v5))
The output conforms to
```text
t̂ = restart ? (t - 1) : mod(t - 1, period)
t̂ = restart ? mod(t - 1, period) : (t - 1)
abs(l0 - l1) * (1 + cos(π * t̂ / period)) / 2 + min(l0, l1)
```
This schedule is also referred to as "cosine annealing (with warm restarts)"
Expand Down

0 comments on commit a635001

Please sign in to comment.