From a445060a4069edd6c4da19dcd70f0d4087dc5fc1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?David=20P=C3=A4tzel?= Date: Fri, 8 Nov 2024 14:51:37 +0100 Subject: [PATCH] Fix docstring of `CosAnneal` Fixes #67. --- src/cyclic.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cyclic.jl b/src/cyclic.jl index abce164..c76ff31 100644 --- a/src/cyclic.jl +++ b/src/cyclic.jl @@ -186,7 +186,7 @@ A cosine annealing schedule (see ["SGDR: Stochastic Gradient Descent with Warm Restarts"](https://arxiv.org/abs/1608.03983v5)) The output conforms to ```text -t̂ = restart ? (t - 1) : mod(t - 1, period) +t̂ = restart ? mod(t - 1, period) : (t - 1) abs(l0 - l1) * (1 + cos(π * t̂ / period)) / 2 + min(l0, l1) ``` This schedule is also referred to as "cosine annealing (with warm restarts)"