chanjianhao / subme

Cloud-based solution to automatically generate and translate subtitles for video files

Deprecated

Hello everyone, sorry I haven't had time to maintain this project. I have moved on to working on Faster-Whisper instead, which also does translation and transcribing of video files, please do check it out if is what you are looking for. It should be quite similar to SubMe feature wise.

SubMe

A cloud deployment of Autosub on Google Colab. It leverages Google's cloud computing clusters and GPU to automatically generate subtitles for uploaded video files in various languages. Audio is first pre-processed using ffmpeg and ffmpeg-normalize. Speech recognition is then conducted on the pre-processed audio using Auditok, after which ffmpeg will be used to splice the audio files into individual segments. The segments will be transcribed using Google's Cloud Speech-to-Text API, then translated into your desired language using py-googletrans.

Why use our tool?

Firstly, its free! Other similar products, such as Veed, are paid services with restrictions on video length. We offer a completely free service with no restrictions on video length.

Secondly, our tool significantly speeds up the runtime of Autosub. A study has shown that despite being a free service, the GPU-enabled Google Colab environment (which is what we are using now) is significantly faster than a GPU-enabled MacBook Pro, Lenovo Legion and Lenovo Thinkpad, contributing to greater time savings.

Thirdly, without the need to provision and maintain expensive hardware, users will also enjoy greater cost savings. Moreover, you will no longer have to worry about overheated machines or exorbitant electricity bills.

Fourthly, we have an easy to use interface. No need to concern yourself with the complex technology of voice extraction and language translation. Just click and few buttons and you’re set.

Lastly, our tool will also help the less fortunate, such as those with hearing disabilities, to decipher what is being spoken in videos and movies.

Getting Started

• Click on to open the notebook in Google Colab.
  
• Follow the instructions on the notebook

Uploading from Google Drive

When you encounter the above screen, click on the link and a new tab will be opened.

Click allow. A code will be generated. Copy the code and paste it into the empty field in the notebook. Press Enter.

After successfully mounting Google Drive, you will be able to access your directory by clicking on File on the left.

Output

When uploading, you will have to specify the extension of the output files (srt, ass, sub, json or txt). The output files can be found in the same directory as the uploaded video file. For example, if you uploaded a video file from /content/drive/My Drive, the subtitle file will also be found here.

When downloading, the output files can be found in /content/drive/My Drive/Torrent.

Updates

10 June 2021 - Subtitling/captioning of video is now possible. Just put the same language code for source and destination language and autosubs will generate subtitles for your video in its original language.

Authors

We are two Singaporean university undergraduates with curiosity and passion for social causes. In particular, we are interested in using technology to benefit the less fortunate people in our society.

• Chan Jian Hao
• Ng Wei Jie

Acknowledgments

• agermanidis
• BingLingGroup

subme's Issues

error runnig

how to resolve ?? i gave an one min .mp4 japense(ja) file

Error command not found

Error command not found

how to same language on video process?

if src_language and dst_language are same its not working
example:
src_language: en
dst_language: en

Slow

Hello!

Thank you for your awesome work on this tool! I appreciate your social justice work.

I ran the Colab notebook, and for some reason it’s slow. I have GPU chosen in runtime settings, so there shouldn’t be a problem with performance. But it’s still comparable to performance I have on my Core i3 CPU.

Is there a way to speed up the process? Thanks a lot!

Colab limitations

Hi,

when i access SubMe notebook in Colab, i cannot start other colab notebooks simultaneously.

On the other hand, if i'm already using Colab on a different notebook, when i try to connect to Subme notebook i always receive the message:

"Too many active sessions. Close an existing session to continue". This is unlike other notebooks on the web, with or without the use of online GPU.

Is it intentional or it's a problem linked to my account (i have Colab Pro)?

Thank you

MessageError: RangeError: Maximum call stack size exceeded.

I am trying to 'Upload from Local' and it gives this error.

TypeError: the JSON object must be str, bytes or bytearray, not NoneType

Conversion completed.
Use Auditok to detect speech regions.
Auditok detection completed.
"/tmp/tmppaa509k1.wav" has been deleted.

Converting speech regions to short-term fragments.
Converting: 100% |##############################################| Time:  0:00:51

Sending short-term fragments to Google Speech V2 API and getting result.
Speech-to-Text: 100% |##########################################| Time:  0:06:25
Traceback (most recent call last):
  File "/usr/local/bin/autosub", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/autosub/__init__.py", line 169, in main
    styles_list=styles_list)
  File "/usr/local/lib/python3.7/dist-packages/autosub/cmdline_utils.py", line 1619, in audio_or_video_prcs
    delete_chars=args.trans_delete_chars)
  File "/usr/local/lib/python3.7/dist-packages/autosub/core.py", line 772, in list_to_googletrans
    result_src = translator.detect(content_to_trans).lang
  File "/usr/local/lib/python3.7/dist-packages/googletrans/client.py", line 369, in detect
    translated = self.translate(text, src='auto', dest='en')
  File "/usr/local/lib/python3.7/dist-packages/googletrans/client.py", line 219, in translate
    parsed = json.loads(data[0][2])
  File "/usr/lib/python3.7/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

Is this due to python being 3.7 instead of 3.6?