sanduhrs / vp9-encode-distributed

Easily re-encode HDTV (MPEG2/AC3) to VP9/Opus using multiple Linux machines (distributed).

This script allows you to use several PCs or VMs or Containers to distribute the load of transcoding a video to VP9/Opus format.
Input: Varies
Output: VP9 + Opus
It is intended to copy 5.1 channel audio and up to 1080i video. It will automatically deinterlace the video if needed.

REQUIREMENTS

Either shared storage (preferred) for the video processing, or a copy of the video in the same location on each system

Linux with ffmpeg installed on each system

libvpx installed

INSTRUCTIONS

Run ssh-keygen on all of the nodes

Copy vp9-encode-distributed to one of the servers in the cluster. That will be your master node as well as a worker node.

Make the script executable by running: chmod a+x vp9-encode-distributed.

Run ssh-copy-id from the master node (the one with this script) to all other nodes

Mount a share in the same mount point on each worker node. You can use NFS, or if you do not have common storage, just use sshfs to mount.

Example:
Login to server1
wget https://raw.githubusercontent.com/Petelombardo/vp9-encode-distributed/master/vp9-encode-distributed
chmod a+x vp9-encode-distributed
Edit the script and set the variable SERVERS="server1 server2".
ssh-keygen
ssh-copy-id server2
ssh server2 "sshfs root@server1:/mnt/shared /mnt/shared"; # Mount server1's /mnt/shared folder on server2
./vp9-encode-distributed /mnt/shared/movie.ts benchmark; # Run this only the first time, or if your server list changes
./vp9-encode-distributed /mnt/shared/movie.ts; # This kicks off the actual vp9 encoding
./vp9-encode-distributed /mnt/shared/movie.ts status; # Tells you the current status of all of the systems.
watch bash ./vp9-encode-distributed /mnt/shared/movie.ts status; # Keeps the status of all of the systems up on your screen perpetually. ctl-c to stop it.
./vp9-encode-distributed /mnt/shared/movie.ts join; # Run this after encoding completes on all nodes. Give the complete path+filename when prompted. example: /mnt/movies/movie.mkv