Questions related to PR that adds support to train with durations

[shreyasinghal-17]

1.) Have you seen any tangible benefits of retraining with extracted durations ?

2.) If I understand correctly, the first training pass uses MAS to train a base model, which is used to extract phoneme duration information. Once that information is extracted the model is retrained with this duration information instead of MAS? is this correct?

3.) in a issue I saw you commented, doubling the decoder params could be beneficial, I wanted to ask intuitively which model/training run would be a better choice 2nd one or 1st. Or do you recommend to train both runs with double decoder params?

[shivammehta25]

Before I answer, I must say that this is an experimental feature and we do not compare this in our paper.

1. I have tried to answer this question in the wiki:

When Matcha-TTS is trained enough that its alignment plots are not changing drastically, one can extract and train on these alignments. This improves GPU utilisation and makes training even faster as you can increase the batch size without any overhead due to GPU parallelization over batches.

2. Yes! That is the idea
3. That is a hyperparameter fine-tuning question that depends a lot on your data, hardware etc. But from what I have seen, if you can increase the parameters and batch size almost always do it! I don't think it matters so much for learning alignments but obviously the better the model has learnt the better the alignment, but this is something you will have to experiment it because I haven't done it.