jprjr / docker-multistreamer

Dockerized multistreamer

Archived

Hi all. I don't enjoy working on this anymore. I'm archiving this repo, you're free to fork it.

Stop emailing me directly.

docker-multistreamer

A Docker image for multistreamer

Introduction

Multistreamer is a webapp for publishing to multiple video-streaming services at the same time.

This is an image for running multistreamer in a container.

Now with a handy video! https://youtu.be/HdDDtBOLme4

Quick Start

You can get this up and running with docker-compose. Copy docker-compose.override.yml.example to docker-compose.override.yml and edit as needed, then then run docker-compose up. You'll have multistreamer's web interface running on port 8081, RTMP ingest on 1935, and IRC on 6667.

With the default settings, there's no real authentication - users get automatically created at login and saved to redis. Additionally, the chat web interface is most likely broken, you need to change a few settings to make that work.

Not-so-quick start

If you mount an htpasswd volume into the container at /etc/htpasswd-auth-server/htpasswd, the container will launch an internal htpasswd-auth-server instead of the built-in redis auth. Or if you're using docker-compose, drop an htpasswd file into the htpasswd directory at the root of this repo.

If you want to provide actual authentication, set the MULTISTREAMER_AUTH_ENDPOINT environment variable.

Multistreamer makes HTTP requests against said endpoint, using the status code to determine if the password was correct. Here's a few usable endpoints:

• redis-auth-server - auto-create and store users in redis (included)
• htpasswd-auth-server - auth against an htpasswd file (included)
• ldap-auth-server - auth against LDAP.

Things to know:

When you use container linking, Docker updates the /etc/hosts file with hostnames, but nginx ignores that file, and strictly looks up hosts via DNS.

Required environment variables

Here's the list of environment variables you absolutely need to set to ensure this works correctly:

• MULTISTREAMER_SESSION_SECRET - this is used to encrypt client-side session data.

Multistreamer doesn't have any good way to figure out its public hostname, what proxies it might be behind, whether or not you have an SSL terminator, and so on - you're expected to provide that by setting these environment variables. Note: these HTTP/RTMP URLs should be the root of your web host, regardless of running multistreamer under some path. There's a dedicated MULTISTREAMER_HTTP_PREFIX for changing your prefix.

• MULTISTREAMER_PUBLIC_HTTP_URL - This should be the root of the HTTP/HTTPS URL you want your clients to use. If this is incorrect, things like websockets won't work correctly.
• MULTISTREAMER_PUBLIC_RTMP_URL - This should be the root of the RTMP/RTMPS URL you want your clients to use.
• MULTISTREAMER_PUBLIC_IRC_HOSTNAME - Set this to some hostname for IRC users
• MULTISTREAMER_PUBLIC_IRC_PORT - This should be the public-facing port you want your clients to use.

Running Multistreamer under some prefix

• MULTISTREAMER_HTTP_PREFIX - if you're running Multistreamer at something like http://example.com/multistreamer, set this environment variable to /multistreamer (or whatever your prefix is).

Specifying a database

You can use a linked Postgres container or manually specify a host/user/pass/dbname.

If you link the official postgres container as postgresql, all the database settings will be figured out. Otherwise, set these environment variables:

• DB_HOST
• DB_PORT - defaults to 5432
• DB_USER
• DB_PASS
• DB_NAME

Specifying a redis instance

You can use a linked Redis container or manually specify a redis host/port.

If you link the official redis container as redisio, all the redis settings will be figured out. Otherwise, set these environment variables:

• REDIS_HOST
• REDIS_PORT

Enabling streaming services

The multistreamer wiki has details on registering apps for Facebook, Twitch, and YouTube.

Twitch

Specify your client id, secret, and which ingest server to use:

• MULTISTREAMER_TWITCH_CLIENT_ID
• MULTISTREAMER_TWITCH_CLIENT_SECRET
• MULTISTREAMER_TWITCH_INGEST_SERVER

YouTube

Specify your client id, secret, and country code:

• MULTISTREAMER_YOUTUBE_CLIENT_ID
• MULTISTREAMER_YOUTUBE_CLIENT_SECRET
• MULTISTREAMER_YOUTUBE_COUNTRY_CODE

Facebook

Specify your app ip and secret:

• MULTISTREAMER_FACEBOOK_APP_ID
• MULTISTREAMER_FACEBOOK_APP_SECRET

Mixer

Specify your client id, secret, and which ingest server to use:

• MULTISTREAMER_MIXER_CLIENT_ID
• MULTISTREAMER_MIXER_CLIENT_SECRET
• MULTISTREAMER_MIXER_INGEST_SERVER

Other environment variables

• MULTISTREAMER_RTMP_PREFIX - defaults to 'multistreamer' if unset. This is independent from the HTTP prefix, and should be a single text value without slashes.
• MULTISTREAMER_LOG_LEVEL - defaults to 'error'
• MULTISTREAMER_PUBLIC_IRC_SSL - defaults to 'false', set to 'true' if you have an SSL terminator for multistreamer's IRC port.
• MULTISTREAMER_SESSION_NAME - defaults to 'multistreamer'
• MULTISTREAMER_LOG_QUERIES - true/false, whether to log every SQL query
• MULTISTREAMER_LOG_REQUESTS - true/false, whether to log every HTTP request
• MULTISTREAMER_IRC_FORCE_JOIN - defaults to 'false', set to 'true' to have IRC users force-joined into rooms when they go live.
• MULTISTREAMER_WORKER_PROCESSES - defaults to 1, you can change the number of workers
• MULTISTREAMER_DNS_RESOLVER - defaults to '127.0.0.1:5353 ipv6=off', see http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
• MULTISTREAMER_DICT_STREAMS_SIZE - defaults to 10m, this lets you change the shared dictionary size for keeping active stream information.
• MULTISTREAMER_DICT_WRITERS_SIZE - defaults to 10m, this lets you change the shared dictionary size for keeping active chat reader/writer information.

docker-multistreamer's Issues

ERROR: yaml.scanner.ScannerError: while scanning a block scalar

An error is getting thrown here:

   MULTISTREAMER_IRC_MOTD: |=
     Welcome to the Multistreamer IRC server!
     I hope you have fun.

With the message:

ERROR: yaml.scanner.ScannerError: while scanning a block scalar
  in "./docker-compose.override.yml", line 49, column 28
expected chomping or indentation indicators, but found '='
  in "./docker-compose.override.yml", line 49, column 29

My docker-compose version is:

docker-compose version 1.17.1, build 6d101fb

Workaround I did was to convert the MOTD to one line.

Issue on Deployment

Ok, take a seat, get some coffee cause this going to be a long one.

I had problems with deploying Multistreamer via a VM in my ESXi machine so I decided to something more robust like deploying it on the pretty amazing Mesosphere's DC/OS ;). At first I had to mangle through creating an app definition to launch the docker-multistreamer image, then work my way into setting/passing the environment variables, then upon checking the stderr and stdout of the launched docker-multistreamer I noticed that I needed a Redis and a Postgres deployment, two clicks and I have the two running on DC/OS without any problems. I made sure that the hostnames are reachable from where im launching Multistreamer and boom, got the UI up and running.

Until that little Unrecoverable error! Please check logs and restart Multistreamer error popped out. I checked my DC/OS multistreamer stderr and stdout, its clean. I had to jump into the node running the Multistreamer container and do a docker exec just to get a shell into the running container. I checked /var/log/multistreamer on what ails this deployment, lo and behold:

2018-02-23 10:27:57.652930500  	[C]: in function 'wait'
2018-02-23 10:27:57.652930500  	init_worker_by_lua:12: in function <init_worker_by_lua:7>, context: ngx.timer
2018-02-23 10:27:57.653227500  2018/02/23 10:27:57 [error] 359#359: *12 stream lua user thread aborted: runtime error: ...streamer/lua_modules/share/lua/5.1/lapis/db/postgres.lua:58: postgresql.marathon.l4lb.thisdcos.directory could not be resolved (3: Host not found)
2018-02-23 10:27:57.653230500  stack traceback:
2018-02-23 10:27:57.653231500  coroutine 0:

Which is a bit weird. I know DC/OS takes care of the load balancing to its deployed services and I even checked if I could reach it while inside the container:

bash-4.3# ping postgresql.marathon.l4lb.thisdcos.directory
PING postgresql.marathon.l4lb.thisdcos.directory (11.255.191.179): 56 data bytes
64 bytes from 11.255.191.179: seq=0 ttl=64 time=0.035 ms
64 bytes from 11.255.191.179: seq=1 ttl=64 time=0.057 ms

I might be hitting what you said about /etc/hosts file but I cant be too sure

When you use container linking, Docker updates the /etc/hosts file with hostnames, but nginx ignores that file, and strictly looks up hosts via DNS.

Doing some Googling and I found this: openresty/openresty#298

So if its a lookup error, is there any workaround for this?

When run with docker-compose quickstart, create custom rtmp url(youtube), stream video is not shown on youtube

Can see steam on web UI, but not on Youtube page
In config.lua I can see rtmp = true
Is there a way to debug it?

Update with main repo to fix twitch

It would be nice to get this updated for this fix to work jprjr/multistreamer@933ef7d

As of now this image has twitch auth broken.

Config Persistence after restart

Hi,
Thank you very much for the amazing work. Multistreamer has been very helpful to me and my team.
I'm currently running Multistreamer using docker compose with minimal configuration. I've created a systemd service file for it, and it runs at startup and works beautifully; users being created on the fly at the first login attempt is just fine for my usecase. However, every time the service is restarted users lose all the config they had created in their accounts.

• What would be the quickest way for config to persist across restarts?
• I'm thinking I'll keep a proper postgresql running and link it instead of starting the default postgre container, would that work?

Thanks a lot.

Stop pushing stream not working properly

Hi @jprjr

Thank you for contributing for stream world.

It is working good right now except stopping streaming button.
For example, First we are sending stream with OBS to docker MULTISTREAMER. After this i created CUSTOM RTMP and pushing stream to facebook. No problem. When i clicked to STOP button, MULTISTREAMER shows has been stopped.

But on Facebook stream still standing. I should stop or restart container for stopping it.
What should i do for this ?

Thanks

Port Binding Issue

Hi, I'm able to get this working locally, but want to test it out on an external web server. When I run the docker container itself, it's binding port 8080 to localhost:

tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN

Is there a way to force the app to bind to 0.0.0.0:8080 instead? Maybe there's an environment var that could be used?

Request: Viewer count

it would be great if there was a feature to add a live viewer count to the from the various services

Chat suddenly stops showing messages

Easy as that. Stopped showing Twitch messages but YT messages were shown. Kinda weird.

I'm using docker so don't know what to do to get logs and such.

is this production ready?

i've included fb app_id and yt client_id (and respective secrets).
i can see my fb pages and yt channel details in settings (categories, privacy stuff and so on).
going live generates links to fb and yt (on chat screen), however when clicking on them:

• on fb i'm getting "content not available" screen
• on yt i'm getting "waiting for: <username>" screen

the only thing i'm able to get is yt chat stream.
oh, and stream preview.
i've set logging to debug, but i can't find anything interesting in logs, just regular api chatter.

in netstat i can't really see any outgoing rtsp connections (port 1935).
anything else i should try?

Embedding stream on website

I want to embed live stream from multistreamer to my website without using YouTube, Twitch etc... What's the best way to do that...?

Error 400 'the scope you are requesting is currently unavailable' when adding a new twitch account

Hello,
I would like to report the following issue I am encoutering:
Adding a Twitch account from multistreamer UI leads to the error mentioned in the title.
I have done the following as configuration:

• I've created an app on dev.twitch.tv
• I've added the client ID, Secret key, and rtmp to the yml config file of docker compose.
• I've authorized the redirect URL 'https://<mywebsite.com>/auth/twitch' in the twitch console.
• I've checked that the URL is publicly exposed (in fact, I could add a facebook and a youtube account using the same URL pattern).

I think the issue is caused by the use of a deprecated scope 'chat_login' which is deprecated in the V5 of Twitch API according to official doc.
This probably still works for old client IDs, but not for new ones...

Anyone experiencing some similar issue?

Error message:

{"status":400,"message":"Sorry, the scope you are requesting is currently unavailable. Please request new scopes documented at https://dev.twitch.tv/docs/authentication/#chat-and-pubsub."}

When add youtube variables to docker-compose cannot see additional option on web UI

Is there anything else I need to do? In config.lua I cannot see additional configuration

When quickstart and try to login getting ERROR: relation "users" does not exist

Nginx: Settings for testing in local network without domain

Hi @jprjr! First of all thank you for this nice package. I was searching exactly for this for some months, but finally found it. It's exactly the use case I want to use.
But I didn't get far. I watched the YouTube videos, I read and changed all config files and I can't get the multistreamer to work in my local network with docker for testing purposes.
May it's this sentence, that causes my problems (but I am not so good at docker, that I understood it directly):

Things to know:
When you use container linking, Docker updates the /etc/hosts file with hostnames, but nginx ignores that file, and strictly looks up hosts via DNS.
So is it possible, that local testing is not possible? Or which config would be usefull then?

This is the error message i get:

user@server:~# curl -v http://localhost:8081
* Rebuilt URL to: http://localhost:8081/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8081 (#0)
> GET / HTTP/1.1
> Host: localhost:8081
> User-Agent: curl/7.52.1
> Accept: */*
> 
* Recv failure: Die Verbindung wurde vom Kommunikationspartner zurückgesetzt
* Curl_http_done: called premature == 1
* stopped the pause stream!
* Closing connection 0
curl: (56) Recv failure: Die Verbindung wurde vom Kommunikationspartner zurückgesetzt

So I guess something is listening to this port, but is not ready to answer me. :)
The browser can't get a connection neither.

And here is my last config-file:

redisio:
  volumes:
   - /home/multistreamer/redis:/var/lib/redis:Z
postgresql:
  volumes:
   - /home/multistreamer/postgresql:/var/lib/postgresql:Z
  environment:
   - POSTGRES_USER=multistreamer
   - POSTGRES_PASS=password
multistreamer:
  ports:
   - "localhost:8081:8081"
   - "localhost:6667:6667"
   - "localhost:1935:1935"
  volumes:
   - /home/multistreamer/multistreamer:/etc/multistreamer:Z
   - ./htpasswd:/etc/htpasswd-auth-server:Z
  environment:
   MULTISTREAMER_APP_NAME: multistreamer
   MULTISTREAMER_LANG_ID: en_us
   MULTISTREAMER_AUTH_ENDPOINT: 
   MULTISTREAMER_SESSION_SECRET: 123456
   MULTISTREAMER_PUBLIC_HTTP_URL: http://localhost:8081
   MULTISTREAMER_PUBLIC_RTMP_URL: rtmp://localhost:1935
   MULTISTREAMER_RTMP_PREFIX: multistreamer
   MULTISTREAMER_HTTP_PREFIX: 
   MULTISTREAMER_PUBLIC_IRC_HOSTNAME: localhost
   MULTISTREAMER_PUBLIC_IRC_PORT: 6667
   MULTISTREAMER_TWITCH_CLIENT_ID: 
   MULTISTREAMER_TWITCH_CLIENT_SECRET: 
   MULTISTREAMER_TWITCH_INGEST_SERVER: 
   MULTISTREAMER_YOUTUBE_CLIENT_ID: 
   MULTISTREAMER_YOUTUBE_CLIENT_SECRET: 
   MULTISTREAMER_YOUTUBE_COUNTRY_CODE: 
   MULTISTREAMER_FACEBOOK_APP_ID: 
   MULTISTREAMER_FACEBOOK_APP_SECRET: 
   MULTISTREAMER_MIXER_CLIENT_ID: 
   MULTISTREAMER_MIXER_CLIENT_SECRET: 
   MULTISTREAMER_MIXER_INGEST_SERVER: 
   MULTISTREAMER_LOG_LEVEL: debug
   MULTISTREAMER_PUBLIC_IRC_SSL: 'false'
   MULTISTREAMER_SESSION_NAME: multistreamer
   MULTISTREAMER_LOQ_QUERIES: 'true'
   MULTISTREAMER_LOQ_REQUESTS: 'true'
   MULTISTREAMER_IRC_FORCE_JOIN: 'false'
   MULTISTREAMER_WORKER_PROCESSES: 1
   MULTISTREAMER_DNS_RESOLVER: 1.1.1.1:53 ipv6=off
   MULTISTREAMER_DICT_STREAMS_SIZE: 10m
   MULTISTREAMER_DICT_WRITERS_SIZE: 10m
   MULTISTREAMER_FFMPEG_MAX_ATTEMPTS: 3
   MULTISTREAMER_PING: 1m
   MULTISTREAMER_PING_TIMEOUT: 30s
   MULTISTREAMER_SHOW_VERSION: 'true'
   MULTISTREAMER_IRC_MOTD: |-
     Welcome to the Multistreamer IRC server!
     I hope you have fun.

Youtube Account

Hi,
Youtube account error with categories...
When I´m try to stream to Youtube cant select categories, and multistreamer send error..
Could you help me?
THX

Installation on an already running nginx vps ?

Please check this issue : jprjr/multistreamer-installer#11

not pushing to some Cutom RTMP like Mips.tv

i tested multistreamer and work fine with youtube, vk ,fb ... but not pushing to some Cutom RTMP like Mips.tv

ARM Support?

I tried running this by following the long 45 minute guide, but it seems like im encountering multistreamer_1 | standard_init_linux.go:178: exec user process caused "exec format error" which I assume is due to libraries or components built for x86.

Can you build a version thats ARM friendly?

What steps are follow on Ubuntu fresh

hi
What steps to follow on Ubuntu fresh installation
I am a beginner, Do you have a text file with all commands used in installation
thank you so much

Docker env vars Update

Seems like these var's have to be updated into:

environment:

• POSTGRES_USER='multistreamer'
• POSTGRES_PASSWORD=password

Not authenticate against Facebook - Facebook not working

HI,

i tried to install docker based installation and i am getting the page and application,

When i click the Facebook, it is redirecting to facebook but it's not working peoperly,
my facebook like look like below:

"https://www.facebook.com/v3.0/dialog/oauth?redirect_uri=https%3A%2F%2Fdiykuwait.com%3A8081%2Fauth%2Ffacebook&client_id=250567575972981&state=ZXlKcFpDSTZNWDA9LkJ6R01ZV2dBUHQrY3VVcmcxdUtpU3RmdjI2UT0%3D&scope=publish_video%2Cuser_events%2Cmanage_pages%2Cpublish_pages"

and the redirect url has 8081 port ,

Nginx conf file:
https://pastebin.com/KrB956vW

and my docker file:

https://pastebin.com/47KYXtQH

Getting 502 Error

Followed your instruction Video and i get an 502 Bad Gateway error. When i try to go directly to it (without nginx via ip and port) i get an empty Response message.