SPIN: added rollout_micro_batch_size parameter which allows users to set the batch size for doing generation during SPIN training.
previously the generation batch size was automatically set to the data parallel size (DP) of the model

Add MoE Support for our reward models.
SFT/SteerLM: LoRA can now be enabled on all model layers
DPO: Enable LoRA on all model layers (In this case the actor will be reference model + LoRA weights, we can switch between actor/reference model by enabling/disabling LoRA)
PPO: Enable LoRA on all model layers (In this case the actor will be init policy + LoRA weights, we can switch between actor/init_policy model by enabling/disabling LoRA)

Fixed issue where random sampler keeps state when resetting for validation, leading to a different validation batch each validation step. Fixed by using a deterministic sampler
Fixed crash with float val check interval in DPOTrainer
Fixed crash with float val check interval when checking progress in DPOTrainer
Fixed potential crash in SPIN when prompts are longer than encoder_seq_len - generation.max_length
Fixed crash when calling the generate() method of an SFT model with pipeline parallelism greater than two
Fixed crash when calling the generate() method of an SFT model with compute_logprob=True and string inputs
Fixed crash when model.micro_batch_size > 1 in DPO
Fixed issue when model.encoder_seq_length is mismatched with model.data.train_ds.max_seq_length in SFT and SPIN.
Delete MegatronPretrainingRandomSampler from Aligner since it has been upstreamed into NeMo

Container

docker pull nvcr.io/nvidia/nemo:24.05

To get access:

Sign up to get free and immediate access to NVIDIA NeMo Framework container. If you don’t have an NVIDIA NGC account, you will be prompted to sign up for an account before proceeding.
If you don’t have an NVIDIA NGC API key, sign into NVIDIA NGC, selecting organization/team: ea-bignlp/ga-participants and click Generate API key. Save this key for the next step. Else, skip this step.
On your machine, docker login to nvcr.io using

docker login nvcr.io
Username: $oauthtoken
Password: <Your Saved NGC API Key>

PyPi