Implementation of PSGD optimizer in JAX
-
Updated
Dec 31, 2024 - Python
#
psgd
Here are 2 public repositories matching this topic...
Newton-Muon + Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.
amd optimization cuda transformers pytorch parallelism transformer muon gpt adam preconditioning kfac megatron adamw optimizers foof large-language-models psgd newton-muon newton-muo
-
Updated
Jun 17, 2026
Add this topic to your repo
To associate your repository with the psgd topic, visit your repo's landing page and select "manage topics."