Continuing training on your shared model about face-vid2vid

[zhengkw18]

Maybe you didn't get the right cropping so the scene is moving? I'm not sure. Or the dataset transfer requires more training. I didn't try VoxCeleb2, but my continue training on VoxCeleb1 is ok. You can check the reconstruction samples after each epoch to see whether such problem exists, or whether there are ridiculous keypoints residing far from the face.

Actually I didn't use training tricks. Maybe the only thing needed is to increase the learning rate if you have large batch.

[Vijayue]

Thank u a lot! I have solved the problem, it's the problem of the datasets.

And now I have another question about the training process. Now I'm training on vox2 from the scratch, has been two days about 10 epochs, but the vis results demonstrate there is no difference between source and prediction.

Considering the cost of training(time and money), I want to know whether my training params is wrong, or training time is not enough. Could you share with me, you can see some difference between source and predictions after how many epochs?

Thank you.

[zhengkw18]

When the headpose loss (H in log) decreases from 200+ to 50, the prediction pose starts to align with driving. On Vox1 10 epochs is enough (for num_repeates=100), but Vox2 may take more. At first the reconstruction demonstrates much distortion, as the training goes on, the distortion is weaker and the expressions start to align with driving, too.

[Liangtian96]

您好！想问一下设置num_repeates=100和将num_repeates设置成1然后epoch设置*100倍有区别吗？不是很清楚为什么一个epoch下还要设置多个num_repeates

[zhengkw18]

没有区别。因为默认开启id sampling，会对identity均匀采样，vox1共1152个id，也就是一个repeat只有1152组样本，通过增加repeat来减少epoch数。