Skip to content

JAX implementation of 'Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training'

Notifications You must be signed in to change notification settings

tensor-fusion/sophia-jax

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

About

JAX implementation of 'Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training'

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages