Tutorial for application on custom dataset about s3prl

[andi611]

I've written a preprocessing script for this: 27ff0d5

For any arbitrary dataset that looks like this:

- Custom_dataset/
    - Custom_train/
       - *.wav / flac / mp3 ...
    - Custom_dev/
       - *.wav / flac / mp3 ...
    - Custom_test/
       - *.wav / flac / mp3 ...

The script will process the "train", "dev", "test" set one by one,
and users only need to specify the path of the directory of each set.
So for the example above,
the path to the "train" set should be: Custom_dataset/Custom_train/
the path to the "dev" set should be: Custom_dataset/Custom_dev/
the path to the "test" set should be: Custom_dataset/Custom_test/
The generated files will be compatible to our dataloader.

Also, in your config file, these should be changed:

  data_path: 'data/NewData_fbank80' 
  train_set: ['train']
  dev_set: ['dev'] 
  test_set: ['test']

If it is convenient, can you please test this script on your own dataset to see if it works.
(I currently don't have any other dataset to process)
Let me know if there is any problem.

[shivam-chandhok]

Thank You very much for your help.I will look into it and get back.
Kindly have a look @Dhumketu

[juanting]

Hello, thank you very much for sharing such an excellent project. As a newcomer in this field, I would like to ask whether this project can be used to generate speaker embedding for my future work. If so, could you please introduce the general process? Looking forward to your reply.

[andi611]

I would like to ask whether this project can be used to generate speaker embedding for my future work. If so, could you please introduce the general process?

Yes, of course. The general process is as follow:

1. Pre-train an upstream model in a self-supervised manner: Mockingjay, TERA, Audio ALBERT, APC, CPC, etc.
2. Extract representations from the pre-trained upstream model, these representations are the speaker embedding you are looking for.
3. Apply the extracted representations to your downstream task.

However, whether the learned representations are good speaker embedding largely depends on your downstream task, we've only verified them with speaker classification tasks using the LibriSpeech corpus. Various speaker classification experiment results are presented in the Mockingjay, TERA paper.

[juanting]

Thank you very much for your reply. I will study the process

[aviasd]

I've written a preprocessing script for this: 27ff0d5

For any arbitrary dataset that looks like this:

- Custom_dataset/
    - Custom_train/
       - *.wav / flac / mp3 ...
    - Custom_dev/
       - *.wav / flac / mp3 ...
    - Custom_test/
       - *.wav / flac / mp3 ...

The script will process the "train", "dev", "test" set one by one,
and users only need to specify the path of the directory of each set.
So for the example above,
the path to the "train" set should be: Custom_dataset/Custom_train/
the path to the "dev" set should be: Custom_dataset/Custom_dev/
the path to the "test" set should be: Custom_dataset/Custom_test/
The generated files will be compatible to our dataloader.

Also, in your config file, these should be changed:

  data_path: 'data/NewData_fbank80' 
  train_set: ['train']
  dev_set: ['dev'] 
  test_set: ['test']

If it is convenient, can you please test this script on your own dataset to see if it works.
(I currently don't have any other dataset to process)
Let me know if there is any problem.

Is this working for pretrained TERA too?
Or is it just suitable for training our own original model from scratch without using the pretrained TERA?

[andi611]

Is this working for pretrained TERA too?
Or is it just suitable for training our own original model from scratch without using the pretrained TERA?

No, this will not work for the pre-trained TERA. As pre-trained TERA requires fmllr data, which can be download from the provided Google drive link. (Pre-trained TERA needs the original fmllr data, not new extracted ones.)