batching multi-client server about whisper_streaming

[Gldkslfmsd]

I also found this fast batching whisper backend: https://github.com/Vaibhavs10/insanely-fast-whisper

[Gldkslfmsd]

you can point me to the parts where changes are needed to implement this.

First, you need a multi-client server. It handles each client the same way as single client, but it needs a new subclass of ASRBase that would connect through API to a batching backend. Maybe the API could be shared with #34 ?

And then you need the Whisper batching backend and API -- I don't know which way is optimal, a subprocess, network API, etc.

From code policy point of view, make a new entry point for the multi-client server. I suggest a separate project which would use Whisper-Streaming as a module. I could not be available to maintain it in this repo.

[Gldkslfmsd]

but more projects could use this feature, like https://github.com/ufal/correctable-lecture-translator . Open-sourcing and collaboration is welcome!