Replacing weight with multiplier

[liPatrick]

To make running in epoch mode easier, we redefine weights to effectively be

1. the number of times that dataset will be copied when weight > 1
2. fraction of the dataset to be used if the weight < 1.
3. By default, the weight is 1 for all datasets.

In the interleave class, I redefine the probability of select the next sample using the dataset length and weight to reflect the above (only in epoch mode)

[liPatrick]

TODO: verify num_epochs # of steps matches what we expect.

[liPatrick]

TODO: verify num_epochs # of steps matches what we expect.

set to 1 epoch with 3000 total samples.
batch size of 24, with 8 gpus.
We expect 16 steps because 3000 / (8 * 24))
https://wandb.ai/fixie/ultravox/runs/widpk5aw/logs

[liPatrick]

With this change, we can no longer use non generic datasets (with or without epoch), because the multiplier requires a length associated with the dataset, which only generic datasets allow you to configure. I'll keep this on the sideline until #111 is merged in.

[juberti]

I think life might be easier if we didn't try to read the multiplier value out of each dataset class, and instead read it from a config passed to the interleave class, similar to how this is done with probabilities in HF's interleave_datasets: https://huggingface.co/docs/datasets/en/package_reference/main_classes#datasets.interleave_datasets

[juberti]

Then we wouldn't have to worry about the interaction with len, etc

[liPatrick]

Then we'd need to calculate the probabilities by hand in the config? I think we'd need len either way because that's how hf determines the # of steps to take in epoch mode.

[juberti]

Well, the weights are being set by hand already, right? It just seems a bit strange to have to read the weight/multiplier property from the dataset class, especially when the dataset class doesn't use it internally.