tyiannak / readys

A Speech Analytics Python Tool for Speaking Assessment

readys

A Speech Analytics Python Tool for Speech Quality Assessment

Recording-level feature extraction

The goal of these modules is to extract features that provide an intermediate representation to speech recordings towards the assessment of speech quality.

Text: text_analysis.py

In order to get text features from an audio file run the below command in your terminal

python3 text_analysis.py -i wav_file -g google_credentials -c classifiers_path -r reference_text -s segmentation_threshold -m segmentation_method

Where:

• wav_file : the path of audio file where the recording is stored

• google_credentials : a json file which contains the google credentials for speech to text functionality

• classifiers_path: the directory which contains all text trained classifiers

• reference_text(optional): path of .txt file of reference text

• segmentation_threshold(optional): if you want to segment text by punctuation, don't use this argument (or use None as value), otherwise it is the number of words or seconds of every text segment

• segmentation_method(optional): if the method of segmentation is punctuation (by sentences) then don't use this argument (or use None as value), otherwise use "fixed_size_text" for segmentation with fixedwords per segment or "fixed_window" for segmentation with fixed time window.

The feature_names , features and metadata will be printed.

Audio: audio_analysis.py

In order to get audio features from audio file (silence features + classification features) run the below command in your terminal

• Case 1: Not using pyaudio recording level features:

python3 audio_analysis.py -i wav_file -c classifiers_path

• Case 2: Adding pyaudio recording level features:

python3 audio_analysis.py -i wav_file -c classifiers_path -f

Where:

• wav_file : the path of audio file

• classifiers_path : the directory which contains all audio trained classifiers

The feature_names , features and metadata will be printed

Note: See models/readme for instructions how to train audio and text models

readys's Issues

Plot recording-level statistics

Study SER and train 2 models (V/A)

Create a dummy dash UI

Enable dynamic recording

Dynamically add segment-level audio and text classifiers in audio_analysis.py and text_analysis.py

Online alignment and reference text highlighting

Add text-related features in the recording-level features (optional argument)

Train and deploy recording-level classifiers

Baseline recording level classifier

Add play button on recorded signal

Refactor the recording-level feature vector calculation

Remove recording-level classification decision from UI (and recording-level-features)

Text-only score and offline text alignment

Code refactoring

Speaker independent cross-val

Train a segment classifier (low, neutral, high)

Make pipeline compatible with features input

Mouseover visualizations

Models

Text train and test functions --> fast text pretrained model + svm classifier

Silence removal

Classifier-based aggregates

Read texts from a pool of texts

Calculate score 1

Extend recording-level text and audio features

Improve training of audio and test segment classifiers (train_text and train_audio)

Hand Crafted recording-level text features

Temporal Score Extraction

Use the recording-level-feature-extraction function to extract features for a simple binary task

Train a speech,music,others,silence segment classifier and extract all speech-related features only for the speech segments