AdamW optimizer details between jittor/optim and axlearn/common/optimizers.

[linjing-lab]

Research on infinity functional conditions, which memoried in mind that transfer of M-dimensional finite compute to N-dimensional functional properties should be achieved through differentiation in Euclidean real spaces. Now think much days about new released AdamW optimizer which may not replace actual converge in ground state materials with optimtool L_BFGS method, and its must shot system from anti missile reality of electromagnetic tracking and thermal sensing, which reflects in complex abnormal curvature.

This issue is inspired from long time reality of my perception, affinity on AdamW optimizer between jittor and axlearn, gives iteration from jittor's AdamW, located at lines within L482-L489 of file optim.py.

p.update(p * (1 - lr * weight_decay))
bias_correction1 = 1 - b0 ** n
bias_correction2 = 1 - b1 ** n
m.update(b0 * m + (1-b0) * g) #exp_avg
v.update(b1 * v + (1-b1) * g * g) #exp_avg_sq
denom = jt.sqrt(v) / jt.sqrt(bias_correction2) + eps
step_size = lr / bias_correction1
p.update(p - step_size * m / denom)

While axlearn/common/optimizers.py implemented AdamW, from one line that affinity might be hard-to-analyze, method framework with superlinear speed in ground state tells in my now reference book, words mapping rules and long-text generations still effectively produced policy-based textbooks of cluster test verification.

tx.append(maybe_instantiate(adam_update_transformation))