Comments (6)
Hello!
Well, I've never done fine-tuning on single-speaker datasets before.
Hence I'm not sure actually fine-tuning the VC decoder will work well.
If fine-tuning does not work well, it would be good to use VCTK or LibriTTS and the single-speaker dataset together.
On the other hand, HiFi-GAN worked well even with finetuning on single-speaker datasets in my experience.
Please refer to my answer and try it. thank you.
from assem-vc.
Hi @wookladin ,
- As expected the single speaker fine-tuning for VC decoder resulted in overfitting because of low data. The lowest val loss ~0.6 still seems pretty high. What was the best val loss for your multi-speaker experiments?
- GTA finetuning HI-FI GAN with the above model gave surprising results. The loss did not improve with training time. Is it because the decoder itself was not good enough to create decent gta mels?
P.S: I'm trying multi-speaker training now. I'll keep you posted on the results.
from assem-vc.
Hi.
-
I've uploaded the validation loss graph of the VC decoder at issue #17
Loss converges to around 0.2. It seems like your VC decoder is overfitted.
A multi-speaker setting will help you to avoid overtiffing.
Thank you for sharing the results! -
Unfortunately, I am not sure by looking at the loss graph.
Did you hear the logged audios at the validation step?
The perceptual quality at that time seems to be important in judgment.
Thank you!
from assem-vc.
Hi @wookladin ,
Multi-speaker training solved the overfitting problem as expected. Logged audios in vocoder training also sound good after that. Thanks
from assem-vc.
@vishalbhavani Thanks for the confirmation that it worked in your case. Did you warm start pre-trained model with the new speaker or did you train from scratch?
from assem-vc.
I started with the pre-trained model
from assem-vc.
Related Issues (20)
- Pre-trained model HOT 1
- Reason to use speaker encoder over speaker embeddings? HOT 2
- NaN loss on cotatron_trainer HOT 3
- One-to-Many HOT 1
- Regarding teacher forcing to calculate alignment HOT 2
- Training HIFI-GAN faster HOT 2
- Controllable and Interpretable Singing Voice Decomposition via Assem-VC HOT 9
- Speech+Transcript conditioned phoneme recognition as an alternative to G2P HOT 1
- Trouble importing AttrDict HOT 1
- Changin the sampling rate HOT 3
- 어떻게해야 모델을 한글 음소로 학습시킬 수 있나요? HOT 2
- Extending to n+1 target speakers using pretrained Cotatron
- Build custom non-English dataset with ARPABET HOT 2
- other models
- How to split singing voices
- teacher-forcing HOT 1
- Cross-lingual supported? HOT 1
- Training Cotatron has a problem HOT 1
- Where can I get audio samples? The link in the README is broken. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from assem-vc.