sistason / rtmp_stream_delayer

Uses ffmpeg and nginx to delay (buffer) an incoming rtmp-stream for an amount of time.

rtmp_stream_delayer

Uses ffmpeg and nginx to delay (buffer) an incoming rtmp-stream for an amount of time.

In the (rare) case that you want to delay a rtmp-stream instead of minimise the delay, this python-script plus the nginx-config will rudimentarily do the job.

The most common scenario requiring a delayed stream is a game stream. Since you do not want your opponents to know what you see, to prevent them from cheating, you stream with a delay, usually between 30sec and 5mins. Most streaming software has already implemented such a feature, for example OBS and xsplit, BUT the stream is buffered in the client, which is very near the point of a failure, for example PC crashes, network outages or software bugs. In that case, the buffered content can be lost, which leads to the stream being down for $buffered_time, $time_to_fix and $delay_until_start_streaming. When delaying the stream at a server (with hopefully better stability), the downtime can be reduced to $time_to_fix, while also displaying a backup-stream while the main-stream is down.

The setup is based on nginx, although other rtmp-streaming(/multiplexing)-software could also work, if you configure it right.

Alternative

@invisan also coded something like this, but more recently. Check out their work if you have problems with this one or need the features they provide: https://github.com/invisan/rtmpdelayer

Installation

• Python
• nginx + the nginx-rtmp-module
• Merge the nginx.conf here with your local nginx.conf

Usage

• nginx.conf:
  • Setup your push-targets and specify the location of the recording-location
• delayer_settings.py
  • Specify the required delay
  • Specify the stream destination (the server location in your nginx conf, or the destination directly)
  • Optional:
    • Specify the backup streams OR
    • Set SINGLE=True to exit after delaying one stream
    • set FFMPEG_EXECUTABLE to avconv or a custom path

1. Start the python-script somewhere with the stream_directory as argument.
2. Nginx gets the stream and records it to the stream_directory.
3. The python-script monitors the stream_directory, delays incoming streams and publishes them again.

You can run one stream-delayer per stream_directory.

Caveats

• Backup-Streams work, but the distinction between the stream legitimately ending and it crashing is not known. We assume: no new file after the stream ended => end. Because of that, the downtime of the client cannot be longer than $DELAY
• There will be artefacts/gaps even when using backup-streams, as downtimes/buffers are at various places in nearly all video processing pipelines (at twitch for example)

rtmp_stream_delayer's Issues

Multiple Stream delay jobs support

Flat stream delay option

Hi!

I love the idea of this tool and would love to use it on my setup but I have no need for a backup stream and all that. I'd simply like to be able to add a stream delay at the nginx/rtmp level. Is there any optimization I can do to the Python scripts to just utilize that feature. Any help would be huge! Thanks!

Only showing backup stream

I was wondering why the script would never publish my main stream from the file but always the backup stream. I don't know Python but is it correct that the publish_stream function will never set "ret = True" anywhere?

Just for testing I changed line 43 to "ret = True" and and finally my main stream was published. However, now it won't fall back to the backup stream if I end the main stream. The script will just end and delete the .flv file.

Having trouble getting this to work

Hi. Been unable to get this to work. Cant understand what the issue seems to be. I merged the nginx.conf file and it runs without an error. Tried streaming to ip:port/delay and the stream goes successfully via OBS without an error. I can see a file being created inside /tmp/nginx. Tried switching ports, endpoints of the outgoing servers, etc but no stream comes after the delay scheduled. Nothing goes out to Twitch, YouTube for example. Any help is appreciated.

Work with record_append

Use record_append, so the destination-stream does not come down.
If no write is detected on the local file any more, check if the end shows a crash (TODO) and if it does, append the backup-stream until the file gets locked again by nginx, eg. the stream is up again.

Again many troubles - sorry for bothering

Hello Sir,

we again expierencing a lot of troubles. Following is happening and it sounds like an network error but we only have this issues when we use the delay script. Maybe you can help us:

We are streaming to our Nginx-Server (a big 64GB RAM, i9 9900k machine on Ubuntu).
Delay is working fine
We also stream with 2 additional observer to the server without stream on different applications (but the error occurs also without them).
After more then 1 hour suddenly the upstream from OBS is not able to stream anymore to the server, the bitrate goes down to 0

On the server I see in the folder with the pid-file that all x-seconds a new 0Byte Flv file appears.
The delayer run into problem (Input wrong): Invalid data found when processing input

The sender is not able to restart OBS because the NGINX-RTMP Key is blocked.
I have to stop the delayer, restart nginx and then all things will work on but it happens also again after a couple of time.

I really dont understand what is going on, also asked the Server provider to send me network statistics.

Best

FFMPEG will stop or crash then a restart will not work

Hello another big topic which happend a few times.
.
Sometimes the ffmpeg encoder stops to work with 2 errors:

• av_interleaved_write_frame(): Broke pipe
• [flv @ ************* ] Failed to update header with correct duration
• [flv @ ************* ] Failed to update header with correct filesize
• Error writing trailer of rtmp://.......: Broke pipe

I am not that programmer and I really dont understand a word in python but I found those two things:
https://stackoverflow.com/questions/38156265/getting-av-interleaved-write-frame-broken-pipe-error
https://stackoverflow.com/questions/50300013/ffmpeg-streaming-stops-after-few-seconds

I dont know if this is already in your code and I also dont know if this will solve my problems.

The biggest issue is, that we cant restart the stream after those errors without restarting the whole nginx service.
Normally the python should work on again after restarting the stream automatically, but it dont do that.

Quick note: we have 2-4 scripts running at the same time in several "screens".

Issue with getting Backup streams to work

Really great job with this @sistason and really appreciate the support.

Been trying to get the BACKUPSTREAM_SHORT and/or BACKUPSTREAM_LONG to work. Nothing seems to run though. Appreciate any guidance.

Script wont start with error

I guess I fixed this past in the time but I don`t know how.

Traceback (most recent call last): File "rtmp_stream_delayer.py", line 198, in <module> from delayer_settings import BACKUPSTREAM_SHORT, BACKUPSTREAM_LONG, \ ImportError: cannot import name 'FFMPEG_EXTRA_OPTS' from 'delayer_settings' (/etc/delay/regie-1-900s/delayer_settings.py)

STREAM_DESTINATION = 'rtmp://127.0.0.1/becleanfeed1/live' DELAY = 900 # Seconds SINGLE = False # Set to true to just delay a single stream and exit, no reconnecting/backup-streams BACKUPSTREAM_SHORT = '' # Show while intermission BACKUPSTREAM_LONG = '' # Show while longer downtime FFMPEG_EXECUTABLE = "ffmpeg" # Use avconv if you need

new server old issues I gues.

iostname

I doubt this line should exist:

rtmp_stream_delayer/rtmp_stream_delayer.py

Line 2 in d707006

iostname 195.192.129.164

Can we use this for creating delay while stream on twitch is already live?

I start the stream in "realtime" and after i found a game i would like to prevent getting streamsniped.
Am i able to put a delay into already running twitch stream? Some sort of RTMP switcher (switch from the main live rtmp to the delayed one) without canceling the stream on twitch itself? (to prevent more VODs creation and viewers being dropped out). Is that the main use of this script of Yours?

Cant run your script

Hey sorry man, I am not the big brain in linux and python.
I tried to add your script to my rtmp server but if I start the script I will get this error messages:

Tried to start with: sudo python rtmp_stream_delayer.py /home/trauni/record/cleanfeed/

Traceback (most recent call last):
File "rtmp_stream_delayer.py", line 186, in
if check_running(PID_FILE):
File "rtmp_stream_delayer.py", line 166, in check_running
except FileNotFoundError:
NameError: global name 'FileNotFoundError' is not defined

My config:
delayer_settings:

STREAM_DESTINATION = 'rtmp://148.251.247.165/cf_delay'
DELAY = 120 # Seconds
SINGLE = True # Set to true to just delay a single stream and exit, no reconnecting/backup-streams
BACKUPSTREAM_SHORT = '' # Show while intermission
BACKUPSTREAM_LONG = '' # Show while longer downtime
FFMPEG_EXECUTABLE = "ffmpeg" # Use avconv if you need

nginx.conf

rtmp {
server {
listen 1935;
chunk_size 4096;
allow play all;
max_message 5M;

#Cleanfeed Application
application cleanfeed {
live on;
record all;
record_path /home/trauni/record/cleanfeed;
record_unique on;
record_suffix .flv;
record_append on;
}

#Delayer Script Application
application cf_delay{
live on;
record off;
}

            }

}

Stream is running, flv file is generated and growing.
I can also open the stream in VLC and see its running into the correct application.

Thanks for help

Backup stream not working with rtmp link to same server (other application)

Hello sir,
I`m trying to use another rtmp application of nginx as backup if the first source has an OBS crash or outage.

STREAM_DESTINATION = 'rtmp://148.251.247.165/delay/cf' DELAY = 17 # Seconds SINGLE = False # Set to true to just delay a single stream and exit, no reconnecting/backup-streams BACKUPSTREAM_SHORT = 'rtmp://148.251.247.165/backup/backup' # Show while intermission BACKUPSTREAM_LONG = 'rtmp://148.251.247.165/backup/backup' # Show while longer downtime FFMPEG_EXECUTABLE = "ffmpeg" # Use avconv if you need
But it ends up in "Error header information"

Best