Comments (4)
By conditioning on speaker embedding, it changes the rhythm and timbre at the same time.
from autopst.
Hi @auspicious3000
I am computing the spectrogram like this:
def butter_highpass(cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = signal.butter(order, normal_cutoff, btype='high', analog=False)
return b, a
def pySTFT(x, fft_length=1024, hop_length=256):
x = np.pad(x, int(fft_length//2), mode='reflect')
noverlap = fft_length - hop_length
shape = x.shape[:-1]+((x.shape[-1]-noverlap)//hop_length, fft_length)
strides = x.strides[:-1]+(hop_length*x.strides[-1], x.strides[-1])
result = np.lib.stride_tricks.as_strided(x, shape=shape,
strides=strides)
fft_window = get_window('hann', fft_length, fftbins=True)
result = np.fft.rfft(fft_window * result, n=fft_length).T
return np.abs(result)
mel_basis = mel(16000, 1024, fmin=90, fmax=7600, n_mels=80).T
min_level = np.exp(-100 / 20 * np.log(10))
b, a = butter_highpass(30, 16000, order=5)
AUDIO_DIR = "/content/drive/MyDrive/CODE/VoiceAE/audio/"
filename = os.path.join(AUDIO_DIR, "test.wav")
# Read audio file
#x, fs = sf.read(filename) sr=16000
x, fs = librosa.load(filename, duration=4)
# Remove drifting noise
#y = signal.filtfilt(b, a, x)
y = x
# Ddd a little random noise for model roubstness
#wav = y * 0.96 + (prng.rand(y.shape[0])-0.5)*1e-06
wav = y
# Compute spect
D = pySTFT(wav).T
# Convert to mel and normalize
D_mel = np.dot(D, mel_basis)
D_db = 20 * np.log10(np.maximum(min_level, D_mel)) - 16
S = np.clip((D_db + 100) / 100, 0, 1)
# save spect
np.save(os.path.join(AUDIO_DIR, filename[:-4]), S.astype(np.float32), allow_pickle=False)
Then I do the inference like this:
waveform = wavegen(model, c=S)
But of course something is missing, how can I condition the embedding on a specific speaker? Could you point me to some code please?
from autopst.
with torch.no_grad(): spect_output, len_spect = P.infer_onmt(cep_real_A.transpose(2,1)[:,:14,:], real_mask_A, len_real_A, spk_emb_B)
from autopst.
@shoegazerstella figured it out?
from autopst.
Related Issues (17)
- ModuleNotFoundError: No module named 'onmt' HOT 1
- KeyError when run prepare_train_data.py HOT 2
- How to solve SEA model problem
- the speech content of converted voice with my own trained model changed HOT 2
- SpeechSplit actually better than AutoPST for seen speakers? HOT 1
- Missing basic execution with different set of speakers. HOT 4
- Error while running demo.ipynd
- Issue with stop prediction for longer utterances. HOT 1
- test_vctk.meta HOT 5
- Unable to reproduce results HOT 1
- License of this repository and model HOT 3
- How to test AutoPST in onother languages? HOT 6
- How to train SEA model HOT 14
- How to make 'mfcc_stats.pkl' and 'spk2emb_82.pkl'? HOT 3
- How to find mean and std of MFCC? HOT 8
- 請問我該如何解決 repeats has to be Long tensor 的問題?(How to solve a problem) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autopst.