I have a question about the paper. about multilingual_text_to_speech

[Tomiinek]

Hello 🙂

The figure tries to show a training batch with two examples, one is in Latin and the other in Chinese. So the model is given also a vector of two language IDs (like [0,1]) which are converted into an embedding with a higher dimension.

During inference, when a single code-switched input such as "Hello 王" is pushed into the model, it is converted into a batch of the same two textual inputs and with different languages, i.e. [0,1]. The two outputs of the encoder are then mixed together, as we know from the input which characters are from which language and then this mixed vector of encoder outputs are passed into the decoder.

[Tomiinek]

I hope I remember it correctly 😄

There is no separate language identification module, we expect that the information about languages is provided besides the input texts.

[sooftware]

You mean, Language ID 0 generate parameter to Conv group 1, Language ID 1 generate parameter to Conv group 2 right??

[Tomiinek]

Exactly 🙂

And the parameters are not generated for each example in the batch, but just for each unique language ID in the batch.

[sooftware]

Thank You Tomminek!!