klxu03 / godcaster Goto Github PK

Because we are processing alot of videos, ideally we want it to run fast

We should look into extracting as much as possible from WhisperX

Tasks

1. Make runner for WhisperX ideally work with a simple map reduce system i.e make it work by feeding a list of files so that we can replicate it across multiple nodes
2. Look into whether we can use a smaller model
3. If not look whether we can quantize WhisperX
4. If not look into whether we can compiler WhisperX

Very exploratory but the main thing is to get the runner work so we can replicate it

1. Find a way within whisper i.e through a prompt possibly for it to either weight the names of the operators/special words more or to replace them with tags
2. Manually comb through some of the outputs and then make a simple look up and iterate on that until its mostly clean
3. Use fastpunct maybe? https://pypi.org/project/fastpunct/ it might have rules
4. Handcurate list of important terms and teams and stuff i.e https://github.com/Tsunehiko/create-lol-v2t/blob/master/dataset/

Captions is not up to code with python standards

We need init.py, folders, division of classes among individual files, removal of useless code

We need classes to be separate from scripts

Tasks

1. Split captions/module.py into 3 files containing Whisper class, WhisperX class, and runner for WhisperX
2. add init.py to make module
3. Remove youtube downloading stuff as we are ingesting the files from a downloaded source