This repository contains the code to reproduce the core results from the paper "Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data"
License: Apache License 2.0
abd how to generate it for a new dataset?
thank you
Hi, I tried to run this work, but it needs BaseFloatMatrixWriter, BaseFloatVectorWriter and SequentialBaseFloatVectorReader from kaldi_io.
Now I can install kaldi_io package through pip but it just provide the version 0.9.0, 0.9.1, 0.9.3, 0.9.4. I tried to use it, but it reported that cannot import 'BaseFloatMatrixWriter/BaseFloatVectorWriter/SequentialBaseFloatVectorReader' form 'kaldi_io'.
I think this problem is caused by the version of kaldi_io, but I can't find a suitable version.
How could I use it or use another package to replace it?
Hi, thanks for open sourcing the code!
I am trying to reproduce your experiments, but I have no experience with Kaldi and I have a hard time understanding what kind of features are extracted.
Am I correct, that Kaldi is only used to extract 80-dim mel-spectrogram features for the input and 200-dim log-spectrogram features for the output to synthesize voice with griffin-lim?
However, you seem to also use time aligned phoneme labels. What are these used for?
Thanks!
Hello Wei-Ning,
Truly great job on this paper. I enjoyed reading it and listening to samples.
I am trying to run the code on TIMIT example, however some file are hardcoded. I was wondering is there a way to guide how to create those files?
/data/sls/scratch/wnhsu/vae_gan/audio_encoder/egs/timit/data/train/phn.scp
/data/sls/scratch/wnhsu/code/test_slurm.sh #this is probably environment-specific
Thanks!
Hamid
In the paper, it says that the 80-dimensional Mel-scale filter bank features are used or 200 dimensional log-magnitude spectrum is used but only for audio reconstruction. What does this mean?
I cannot find a certain configure item in the code. Would you like to give more detail ?
thank you
I am unable to locate the file being pointed to "data/spec_scp/train/dataset.cfg". Can you please let me know where it is?
Thanks
I am facing an error when running the TIMIT example:
./run_fhvae.sh --TIMIT_RAW_DATA ~/TIMIT/TIMIT --stage -1
It seems that the code expects "[" but the audio file header is "RIFF" type.
ERROR (Read():kaldi-matrix.cc:1432) Failed to read matrix from stream. : Expected "[", got "RIFFXS
Here is the full log:
Loading datasets: train=True, dev=True, test=False
KaldiRADataset constructor
ERROR (Read():kaldi-matrix.cc:1432) Failed to read matrix from stream. : Expected "[", got "RIFFXS
WARNING (Read():util/kaldi-holder-inl.h:82) Exception caught reading Table object
WARNING (LoadCurrent():util/kaldi-table-inl.h:246) TableReader: failed to load object from /home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/data/wav/train/FAEM0_SI1392.wav
ERROR (Value():util/kaldi-table-inl.h:149) TableReader: failed to load object from /home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/data/wav/train/FAEM0_SI1392.wav (to suppress this error, add the permissive (p, ) option to the rspecifier.
Traceback (most recent call last):
File "src/scripts/run_nips17_fhvae_exp.py", line 500, in <module>
main()
File "src/scripts/run_nips17_fhvae_exp.py", line 115, in main
set_name, is_train, True, True, False)
File "src/scripts/run_nips17_fhvae_exp.py", line 342, in _load_datasets_and_model
sets = datasets_loader(dataset_conf, train, dev, test)
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/datasets_loaders.py", line 45, in datasets_loader
return kaldi_ra_datasets_loader(conf, train, dev, test)
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/datasets_loaders.py", line 55, in kaldi_ra_datasets_loader
return _kaldi_ra_datasets_loader(Dataset, conf, train, dev, test)
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/datasets_loaders.py", line 117, in _kaldi_ra_datasets_loader
q_type=q_type)
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/kaldi_ra_dataset.py", line 81, in __init__
super(KaldiRADataset, self).__init__(**kwargs)
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/base_dataset.py", line 14, in __init__
self._load_data()
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/kaldi_ra_dataset.py", line 112, in _load_data
self._load_kaldi_feat_list()
File "/home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/src/datasets/kaldi_ra_dataset.py", line 126, in _load_kaldi_feat_list
utt_id, utt_feats = f.next()
RuntimeError: ERROR (Value():util/kaldi-table-inl.h:149) TableReader: failed to load object from /home/hamid/VAE/FactorizedHierarchicalVAE/egs/timit/data/wav/train/FAEM0_SI1392.wav (to suppress this error, add the permissive (p, ) option to the rspecifier.
[stack trace: ]
kaldi::KaldiGetStackTrace()
kaldi::KaldiErrorMessage::~KaldiErrorMessage()
kaldi::SequentialTableReaderScriptImpl<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >::Value()
kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >::Value()
boost::python::api::object sequential_reader_next<kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > > >(kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&)
_object* boost::python::detail::invoke<boost::python::to_python_value<boost::python::api::object const&>, boost::python::api::object (*)(kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&), boost::python::arg_from_python<kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&> >(boost::python::detail::invoke_tag_<false, false>, boost::python::to_python_value<boost::python::api::object const&> const&, boost::python::api::object (*&)(kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&), boost::python::arg_from_python<kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&>&)
boost::python::detail::caller_arity<1u>::impl<boost::python::api::object (*)(kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::api::object, kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&> >::operator()(_object*, _object*)
boost::python::objects::caller_py_function_impl<boost::python::detail::caller<boost::python::api::object (*)(kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&), boost::python::default_call_policies, boost::mpl::vector2<boost::python::api::object, kaldi::SequentialTableReader<PythonToKaldiHolder<MatrixToNdArrayConverter<float> > >&> > >::operator()(_object*, _object*)
boost::python::objects::function::call(_object*, _object*) const
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0(+0x279b8) [0x7f36f68629b8]
.
.
.
python(PyEval_EvalFrameEx+0x7e8) [0x524338]
python(PyEval_EvalFrameEx+0xc9a) [0x5247ea]
python(PyEval_EvalCodeEx+0x2b1) [0x555551]
python(PyEval_EvalFrameEx+0x1a10) [0x525560]
python() [0x567d14]
python(PyRun_FileExFlags+0x92) [0x465bf4]
python(PyRun_SimpleFileExFlags+0x2ee) [0x46612d]
python(Py_Main+0xb5e) [0x466d92]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f373d1daf45]
python() [0x577c2e]
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.