edabati / whisper-live

A CLI tool to transcribe audio in real-time using Whisper-based models.

Whisper Live

---

Table of Contents

• Description
• Contributing
• License
• Inspirations and Related Work

Description

Warning

This project is still a work in progress! See Issues for a list of known bugs and missing features.

whisper-live is a CLI tool for real-time audio transcription using Whisper-based models.

It currently loads models using the HuggingFace transformers library.

Implementation logic

The implementation is highly based on https://github.com/davabase/whisper_real_time. More advanced logic could be considered for future releases.

The process is as follows:

• The audio is recorded via the microphone using a background thread
• Every N seconds, an audio chunk is created. N is defined by the --recording-duration argument.
• The audio chunk is sent to the model for transcription.
• The transcription is printed to the console.

Note: audio chunks (and transcriptions) are not aware of previous chunks. Once a chunk is processed, its audio is discarded. Once a transcription is printed, its text is not considered anymore.

Contributing

1. Clone this repo
2. Make sure you have hatch installed. If not, follow the guide here.
3. Create a virtual environment hatch env create and activate it hatch shell
4. Install pre-commit hooks for linting: pre-commit install

Warning

No unit tests are currently implemented!

License

whisper-live is distributed under the terms of the MIT license.

Inspirations and Related Work

This project was inspired by:

• https://github.com/davabase/whisper_real_time
• https://github.com/ufal/whisper_streaming
• https://github.com/oliverguhr/wav2vec2-live

whisper-live's Issues

Add unit tests

Add `faster-whisper` support

We need to add the support to faster-whisper. It may be faster during inference compared to HF transformers.
We should also test the performance increase/decrease.

faster-whisper: https://github.com/SYSTRAN/faster-whisper

Test other model configs

We need to test different model configs and defaults that may speed up inference time.

E.g. using whisper as in https://github.com/Vaibhavs10/insanely-fast-whisper

Add logs for inference time

We need to test different Whisper libraries, configs and models.
It will be useful to have an indication of the model inference time in the logs