Whisper-live taking same time on CPU and GPU to transcribe an audio about faster-whisper

[trungkienbkhn]

@prem1303 , hello. FW 0.10.0 is broken tag, could you update FW and Ctranslate2 to latest version (FW 1.0.1 + Ctranslate2 4.2.0 + CUDA 12) ?

[prem1303]

Thanks @trungkienbkhn ,
I tried but still getting same time on GPU, any other suggestions????

[trungkienbkhn]

@prem1303 Could you show your code and attach example audio ? I will try testing them to evaluate throughput. I think you could try using smaller models or distil models to reduce time execution.

[prem1303]

I am using (https://github.com/collabora/WhisperLive) I have set up a client-side application locally and deployed a server.py (server side) on a GPU remote host. connecting the client and server using WebSockets and passing audio files using a Flask API.

[prem1303]

I solved the issues.
Thank you very much for your support and time.

[ruifengma]

@prem1303 do you use docker to deploy the model?