Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler with gradient clipping #54

Closed
JinraeKim opened this issue Dec 5, 2023 · 1 comment
Closed

Scheduler with gradient clipping #54

JinraeKim opened this issue Dec 5, 2023 · 1 comment

Comments

@JinraeKim
Copy link

Motivation and description

I think most implementations would require gradient clipping.
For example, Step (exp-decay with step size) with gradient clipping setting a lower bound of the learning rate.

These were common in previous versions of Flux.
So I think it would be very useful if such functionality is provided by default with keyword arguments.

If it is already implemented, I don't think the current documentation describes it well as I couldn't find it in the docs.

Possible Implementation

No response

@ToucheSir
Copy link
Member

https://discourse.julialang.org/t/learning-rate-scheduler-with-the-new-interface-of-flux/107142 is a cross-post of this, so closing the GH issue in favour of the more appropriate platform for questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants