Trouble Installing On Ubuntu 16.04 about multistreamer

[jprjr]

Totally! The htpasswd-auth-server is meant to be more of a demo authentication server, just to help get up and running, the idea is you can swap in nearly anything for authentication.

Now, that covers the "is it possible" part, but the "how" is really up in the air / up to you. I have a different auth server that uses LDAP, for example - so you could setup LDAP and a web interface for it (like PHPLdapAdmin), and manage users that way. That's a bit of a steep learning curve but really nice once you get it up and running.

Or if you know any web programming - PHP, Python, whatever, it's not too hard to write your own. The overall idea is like, most people likely already have something that stores users, whether it's a WordPress install, or a mail server, or whatever. So Multistreamer doesn't store any passwords, it just checks against some other service.

But yeah, the only 3 that I've written are the htpasswd one, redis, and the LDAP one, none have any kind of management interface or anything like that.

[jprjr]

Looks like the readme for htpasswd-auth-server was out of date - it also requires the lecho lua module. I just updated the README for it, let me know if that works!

[jprjr]

You shouldn't have to do that, but I'm a bit out of time to continue helping. Maybe reinstall Apache and look up how to use it as a reverse prize, or just stick with running on a public port.

…

On August 7, 2017 6:43:38 PM CDT, Liam Green ***@***.***> wrote: Think i'm going to have to rebuild the VPS. and then untill apache and reboot the server the do the install for Multistreamer -- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: #19 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[liamgreen]

Thank you very so much. Works now. able to add users :)

[liamgreen]

Also for some reason when i'm trying to install the lua moduels in the miltistreamer part of the video. This is happening. I honestly don't know what i'm doing wrong. Feel like i'm going in circles ahaha. Thank you for all the help.

[liamgreen]

Also having this problem too. Do you do a installing service ahaha

[jprjr]

For your first screenshot - it looks like luarocks isn't in your PATH, try calling it as /opt/openresty-rtmp/bin/luarocks

[jprjr]

For your second screenshot, what does ls -lh /root/htpasswd-auth-server/etc show? Did the htpasswd file get created with some weird permissions? Also do cat /root/htpasswd-auth-server/etc/htpasswd

It looks like you're running as root so it should be able to read it. I usually don't run as root, though. It wouldn't surprise if nginx is dropping privileges to some fallback user, I'd really recommend switching to some other account for all of this.

Docker's my recommended way to install this nowadays - you can run the Docker image directly, or use docker-compose to spin up Multistreamer, Postgres, and Redis all in one easy-to-use command https://github.com/jprjr/docker-multistreamer

[liamgreen]

For your first reply. About calling 'luarocks' i don't understand what you mean by that.

Here is the htpasswd stuff.

Also yeah i understand that the docker way of doing all this would be way faster. But i don't have a clue about docker at all. and i want this to be installed on a VPS i have hence why i'm doing it this way. This side of things i'm very new too. I've managed to install nginx and the rtmp moduel and then edit the nginx.conf to push to twitch. But that's as far as my interlect will take me.

[liamgreen]

I think i've found the problem. The VPS i have is made using somthing called OpenVZ and the kernal is below 3.10. So one of the libre things doesn't install correctly? Looks like i need to find another VPS provider. Do you suggest any?

[jprjr]

Unfortunately, a lot of VPS providers use OpenVZ which won't work with Docker - you typically need a KVM.

Let me see if I can whip up a Ubuntu 16.04 install script, it shouldn't be too hard to do.

[liamgreen]

Yeah that's what i've seemt o of found out. Arhh right i see.

Oh thank you very much :)

EDIT: I may be able to use docker + digitalocean? https://docs.docker.com/machine/examples/ocean/#step-4-run-docker-commands-on-the-droplet

[jprjr]

Try this out https://github.com/jprjr/multistreamer-installer

Make sure you wipe out things you've already put into /opt before running

[liamgreen]

Thank you so much :). Trying it out now

[liamgreen]

Everything seems to be working fine. added user etc. Now when i run the multistreamer -e prod initdb command this happens.

[jprjr]

What's in your config.lua file, at /opt/multistreamer/config.lua ?

…

On August 7, 2017 11:18:33 AM CDT, Liam Green ***@***.***> wrote: Everything seems to be working fine. added user etc. Now when i run the `multistreamer -e prod initdb ` command this happens.  -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #19 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[liamgreen]

`local config = require('lapis.config').config

config({'development','production'}, {
-- name of the cookie used to store session data
session_name = 'multistreamer',

-- key for encrypting session data
secret = 'CHANGEME',

-- whether to log queries and requests
logging = {
queries = true,
requests = true,
},

-- if deploying somewhere other than the root of a domain,
-- set this to your prefix (ie, '/multistreamer')
http_prefix = '',

-- set an rtmp prefix
-- note: this can only be a single string,
-- no slashes etc
-- defaults to 'multistreamer' if unset
rtmp_prefix = 'multistreamer',

-- path to your nginx+lua+rtmp binary
nginx = '/opt/openresty-1.11.2.4-rtmp/nginx',

-- path to psql
psql = '/usr/bin/psql',

-- path to ffmpeg
ffmpeg = '/usr/bin/ffmpeg',

-- set your logging level
log_level = 'debug',

-- setup your external urls (without prefixes)
public_http_url = 'http://development.obinx.com',
public_rtmp_url = 'rtmp://development.obinx.com:1935',

-- setup your private (loopback) urls (without prefixes)
private_http_url = 'http://127.0.0.1:8081',
private_rtmp_url = 'rtmp://127.0.0.1:1935',

-- setup your public IRC hostname, for the web
-- interface
public_irc_hostname = 'example.com',
-- setup your public IRC port, to report in the
-- web interface
public_irc_port = '7000',
-- set to true if you've setup an SSL terminator in front
-- of multistreamer
public_irc_ssl = true,

-- configure streaming networks/services
-- you'll need to register a new app with each
-- service and insert keys/ids in here

-- 'rtmp' just stores RTMP urls and has no config,
networks = {
-- mixer = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- ingest_server = 'rtmp://somewhere',
-- },
-- twitch = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- ingest_server = 'rtmp://somewhere', -- see https://bashtech.net/twitch/ingest.php
-- for a list of endpoints
-- },
-- facebook = {
-- app_id = 'app_id',
-- app_secret = 'app_secret',
-- },
-- youtube = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- country = 'us', -- 2-character country code, used for listing available categories
-- },
rtmp = true,
},

-- postgres connection settings
postgres = {
host = '127.0.0.1',
user = 'multistreamer',
password = 'multistreamer',
database = 'multistreamer'
},

-- nginx http "listen" directive, see
-- http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
http_listen = '127.0.0.1:8081',

-- nginx rtmp "listen" directive, see
-- https://github.com/arut/nginx-rtmp-module/wiki/Directives#listen
-- default: listen on all ipv6 addresses (which includes all ipv4 addresses, too)
rtmp_listen = '[::]:1935'',

-- nginx irc "listen" directive, see
-- https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen
-- default: listen on all ipv6 addresses (which includes all ipv4 addresses, too)
irc_listen = '[::]:6667',

-- set the IRC hostname reported by the server
irc_hostname = 'localhost',

-- should users be automatically brought into chat rooms when
-- their streams go live? (default false)
-- this is handy for clients like Adium, Pidgin, etc that don't
-- have a great IRC interface
irc_force_join = false,

-- number of worker processes
worker_processes = 1,

-- http auth endpoint
-- multistreamer will make an HTTP request with the 'Authorization'
-- header to this URL when a user logs in
-- see http://nginx.org/en/docs/http/ngx_http_auth_request_module.html
-- see https://github.com/jprjr/ldap-auth-server for an LDAP implementation
auth_endpoint = 'http://127.0.0.1:8080/',

-- redis host
redis_host = '127.0.0.1:6379',

-- prefix for redis keys
redis_prefix = 'multistreamer/',

-- path to trusted ssl certificate store
ssl_trusted_certificate = '/etc/ssl/certs/ca-certificates.crt',

-- dns resolver
dns_resolver = '8.8.8.8',

-- maximum ssl verify depth
ssl_verify_depth = 5,

-- sizes for shared dictionaries (see https://github.com/openresty/lua-nginx-module#lua_shared_dict)
lua_shared_dict_streams_size = '10m',
lua_shared_dict_writers_size = '10m',

-- specify the run directory to hold temp files etc,
-- defaults to $HOME/.multistreamer if not set
-- work_dir = '/path/to/some/folder',

-- set the path to sockexec's socket
-- see https://github.com/jprjr/sockexec for installation details
-- sockexec_path = '/tmp/exec.sock',

-- allow/disallow transcoding (default: true)
-- allow_transcoding = true,

-- allow/disallow creating pullers (default: true)
-- allow_custom_puller = true,

})`

That's what it is. and now i get this

[jprjr]

It looks like I've had a bad line in my example config file!!

rtmp_listen = '[::]:1935'',

That should be:

rtmp_listen = '[::]:1935',

(the example had an extra apostrophe).

Also, the path to nginx needs to be the full path to the program, and I'd recommend using /opt/openresty-rtmp, that's a symlink that (if you use the installer script) will always point to the latest version

nginx = '/opt/openresty-rtmp/nginx/sbin/nginx',

[liamgreen]

Thanks you for all the help. I really do appreatie all of this.

Done the fixes you've said. Now i get this.

[jprjr]

I just now noticed you're typing "prod" instead of "production" - try it with -e production

I'll have to find places where I used -e prod and replace it with -e production. I know a while ago I had spots where I used production and spots where I used prod, I'm guessing my installation video used prod still? Sorry about that, that's my fault for not being consistent

Oh, and the setting for sockexec path was commented out, make sure to change that to:

sockexec_path = '/tmp/exec.sock',

(in Lua, the -- is a comment marker)

[liamgreen]

Right done all that :)

Now to set up the psql /usr/bin/psql: No such file or directory

[jprjr]

Yeah, you'll have to install the redis-server and postgresql packages, then create the database and user

So run apt-get install redis-server postgresql, then check out https://youtu.be/Wr4CD6RU_CU?t=30m44s - that's where I create the postgres user and password

[liamgreen]

Yeah i'm at that point in the video now :)

I don't think this wants to work at all for me.

[jprjr]

Or (so you don't have to watch that chunk of the video)

sudo su - postgres
psql
create user multistreamer with password 'multistreamer';
create database multistreamer with owner multistreamer;
\q
exit

[liamgreen]

Thank you :) Getting there now :)

[jprjr]

Whoops, didn't mean to close it, haha

[liamgreen]

Right got all that done. I've done the run command.

[jprjr]

You should be up and running, then!

[liamgreen]

What's happening when you go to http://development.obinx.com:8080

[jprjr]

I get a user/password prompt - I'm guessing that's going to the htpasswd-auth-server

By default, multistreamer only listens on the loopback device on port 8081 - you're supposed to put a reverse proxy in front of it, it's not meant to listen on the internet directly.

If you want to make port 8081 available you can do so - change

http_listen = '127.0.0.1:8081',

to

http_listen = '[::]:8081',

But I really, really stress that should only be temporary, and you should have apache or nginx ahead of it providing SSL

[liamgreen]

As you may tell i'm new to this hole. I'll have todo some research on how you do the reverse proxy.

[liamgreen]

I just seem to be going around and around in circles with no sucsess.

[jprjr]

It looks like you have multiple things using port 8080. You should only have one thing per-port, like htpasswd-auth-server should use port 8080, multistreamer should use port 8081

Now that we've gotten all the lua/multistreamer-specific stuff installed and running I don't know how much more I can help, setting up reverse proxies, ssl etc is all outside the scope of this

[liamgreen]

When i go to here (http://development.obinx.com:8080/) i get a

[jprjr]

I think your port 8080 is the htpasswd-auth-server, which you don't want/need exposed to the public internet anyway. Make sure it's running, but there's no reason to have it listening on an external interface, it should only use the loopback

port 8081 is what you have multistreamer listening on. I think you changed the http_listen line for the wrong program, double check that you have htpasswd-auth-server listening to 127.0.0.1:8080 and multistreamer listening to [::]:8081

[liamgreen]

Right.. So i've done a clean install and even done the http_listen = '[::]:8081', part for now. But still nothing is loading. Don't understand.

[jprjr]

What does ss -tulpn show?

[liamgreen]

[jprjr]

Looks like Multistreamer is still listening on 127.0.0.1:8081

check /opt/multistreamer/config.lua , and try restarting multistreamer. What's in your /opt/multistreamer/config.lua ?

[liamgreen]

How do i restart multistreamer?

`local config = require('lapis.config').config

config({'development','production'}, {
-- name of the cookie used to store session data
session_name = 'multistreamer',

-- key for encrypting session data
secret = 'obinx',

-- whether to log queries and requests
logging = {
queries = true,
requests = true,
},

-- if deploying somewhere other than the root of a domain,
-- set this to your prefix (ie, '/multistreamer')
http_prefix = '',

-- set an rtmp prefix
-- note: this can only be a single string,
-- no slashes etc
-- defaults to 'multistreamer' if unset
rtmp_prefix = 'multistreamer',

-- path to your nginx+lua+rtmp binary
nginx = '/opt/openresty-rtmp/nginx/sbin/nginx',

-- path to psql
psql = '/usr/bin/psql',

-- path to ffmpeg
ffmpeg = '/usr/bin/ffmpeg',

-- set your logging level
log_level = 'debug',

-- setup your external urls (without prefixes)
public_http_url = 'http://development.obinx.com',
public_rtmp_url = 'rtmp://development.obinx.com:1935',

-- setup your private (loopback) urls (without prefixes)
private_http_url = 'http://127.0.0.1:8081',
private_rtmp_url = 'rtmp://127.0.0.1:1935',

-- setup your public IRC hostname, for the web
-- interface
public_irc_hostname = 'example.com',
-- setup your public IRC port, to report in the
-- web interface
public_irc_port = '7000',
-- set to true if you've setup an SSL terminator in front
-- of multistreamer
public_irc_ssl = true,

-- configure streaming networks/services
-- you'll need to register a new app with each
-- service and insert keys/ids in here

-- 'rtmp' just stores RTMP urls and has no config,
networks = {
-- mixer = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- ingest_server = 'rtmp://somewhere',
-- },
-- twitch = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- ingest_server = 'rtmp://somewhere', -- see https://bashtech.net/twitch/ingest.php
-- for a list of endpoints
-- },
-- facebook = {
-- app_id = 'app_id',
-- app_secret = 'app_secret',
-- },
-- youtube = {
-- client_id = 'client_id',
-- client_secret = 'client_secret',
-- country = 'us', -- 2-character country code, used for listing available categories
-- },
rtmp = true,
},

-- postgres connection settings
postgres = {
host = '127.0.0.1',
user = 'multistreamer',
password = 'multistreamer',
database = 'multistreamer'
},

-- nginx http "listen" directive, see
-- http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
http_listen = '[::]:8081',

-- nginx rtmp "listen" directive, see
-- https://github.com/arut/nginx-rtmp-module/wiki/Directives#listen
-- default: listen on all ipv6 addresses (which includes all ipv4 addresses, too)
rtmp_listen = '[::]:1935',

-- nginx irc "listen" directive, see
-- https://nginx.org/en/docs/stream/ngx_stream_core_module.html#listen
-- default: listen on all ipv6 addresses (which includes all ipv4 addresses, too)
irc_listen = '[::]:6667',

-- set the IRC hostname reported by the server
irc_hostname = 'localhost',

-- should users be automatically brought into chat rooms when
-- their streams go live? (default false)
-- this is handy for clients like Adium, Pidgin, etc that don't
-- have a great IRC interface
irc_force_join = false,

-- number of worker processes
worker_processes = 1,

-- http auth endpoint
-- multistreamer will make an HTTP request with the 'Authorization'
-- header to this URL when a user logs in
-- see http://nginx.org/en/docs/http/ngx_http_auth_request_module.html
-- see https://github.com/jprjr/ldap-auth-server for an LDAP implementation
auth_endpoint = 'http://127.0.0.1:8080/',

-- redis host
redis_host = '127.0.0.1:6379',

-- prefix for redis keys
redis_prefix = 'multistreamer/',

-- path to trusted ssl certificate store
ssl_trusted_certificate = '/etc/ssl/certs/ca-certificates.crt',

-- dns resolver
dns_resolver = '8.8.8.8',

-- maximum ssl verify depth
ssl_verify_depth = 5,

-- sizes for shared dictionaries (see https://github.com/openresty/lua-nginx-module#lua_shared_dict)
lua_shared_dict_streams_size = '10m',
lua_shared_dict_writers_size = '10m',

-- specify the run directory to hold temp files etc,
-- defaults to $HOME/.multistreamer if not set
-- work_dir = '/path/to/some/folder',

-- set the path to sockexec's socket
-- see https://github.com/jprjr/sockexec for installation details
sockexec_path = '/tmp/exec.sock',

-- allow/disallow transcoding (default: true)
-- allow_transcoding = true,

-- allow/disallow creating pullers (default: true)
-- allow_custom_puller = true,

})`

[jprjr]

I'm legitimately stumped. With that http_listen set it should be listening on all IP addresses.

At startup an nginx config file is generated, try rm /root/.multistreamer/nginx.production.conf and try restarting, maybe for some reason it didn't get rewritten out?

[jprjr]

Oh I missed where you asked how to restart it.

When you're running it interactively, just hit Ctrl-C

When you're running it as a service - service multistreamer restart

[liamgreen]

Yeah i'm complety stumped too. still nothing after running what you;ve posted. I did have it working. And for some reason now it won't.

[jprjr]

Try rebooting your VPS

[liamgreen]

Done that. What would i do now?

[jprjr]

If you want to email me a username/password I'll take a quick look

[liamgreen]

Okay yeah can do :)

[liamgreen]

Sent a email :)

[jprjr]

Figured it out, making a note here for reference:

In the nginx RTMP module, listening on all ipv6 addresses includes all ipv4 addresses as well, but the HTTP and TCP modules do not do this, you have to pass a the ipv6only=off flag.

On my setups, I've always had the HTTP and TCP modules bound to 127.0.0.1, and have a reverse proxy in front of it (and my reverse proxies don't listen to all interfaces, they listen to specific, listed interfaces). I figured the RTMP module's listen behavior would be the same as the HTTP and TCP modules.

I'm going to update the example config to add the ipv6only=off flag to the RTMP/IRC listeners

[liamgreen]

Thank you ever so much for all the help you have provided me. Just setting up the twitch etc.

One more question. If you don't mind that is. Is this.

[jprjr]

change your public_http_url to http://development.obinx.com:8081 and restart multistreamer service multistreamer restart

You'll also need to login to twitch, pull up your app's settings, and change the redirect uri to http://development.obinx.com:8081/auth/twitch

[liamgreen]

Thank you so much :)

Do you have a video that covers the setting up side of multistreamer once it's installed? Looked on your YouTube. May of missed it.

[jprjr]

Sure do - https://youtu.be/Uz2vXppsMIw

[liamgreen]

Right got it all set up and trying it with OBS now. When i click the start stream it shows this.

Sorry for pestering you like i am.

EDIT: My fault. Didn't set up the information correct. Works now. Ahah :)

[jprjr]

There's a slight bug I'm trying to figure out, it has to do with the timing of launching the server-side ffmpeg.

The workaround: in Multistreamer, go to the stream General Settings (from the main page: click Dashboard, then General) and change "Require Preview" to "yes"

Now when you start streaming from OBS, it should be OK and not show any errors.

Then, back in Multistreamer, pull up your Dashboard, you should see a button that says "go live" (if it's disabled, try refreshing). Click that, and everything should work

[jprjr]

I've been thinking about making that preview option the default, since it's generally a good idea anyway - you can see what Multistreamer is receiving and make sure everything looks alright before actually going live on Twitch etc. But anyways, setting "require preview before going live" also makes troubleshooting a lot easier.

As you saw, OBS can only say "oh, it broke." If you require the preview, then when you click the "Go Live" button, Multistreamer can tell you what the actual, specific problem is.

[liamgreen]

Yeah having that as default would be a good idea.

[liamgreen]

Also about the reverse proxy. You set on up in this video https://www.youtube.com/watch?v=Wr4CD6RU_CU

using nginx-light

is that correct?

[liamgreen]

I've done that but the darn "Apache2 Ubuntu Default Page" is showing instead of multistreamer

[liamgreen]

Right so the reverse proxy system looks like it works. Because when i change the config.lua to the way the reverse proxy is. multistreamer doesn't show up on the port. But the darn "Apache2 Ubuntu Default Page" is showing instead.

[liamgreen]

Just looking now to see what code i need to remove so that the shared streams don't display.

[jprjr]

I think you just need to uninstall Apache and restart nginx

…

On August 7, 2017 5:39:36 PM CDT, Liam Green ***@***.***> wrote: Just looking now to see what code i need to remove so that the shared streams don't display. -- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: #19 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[liamgreen]

Thank you. Just having massive trouble with the hole YouTube api set up now. So confised ahaha.

[liamgreen]

Well i've uninstalled apache. Tried to restart nginx using the nginx restart command. and just nothing.

[liamgreen]

Think i'm going to have to rebuild the VPS. and then untill apache and reboot the server the do the install for Multistreamer

[liamgreen]

Arhh right i see. Well i'll do that and look into it. And yeah i understand about the helping side. You have spent a lot of your free time helping me. Witch i do appreatie very much 👍 :)

[liamgreen]

Can i ask a question. Instead of using the htpasswd-auth-server add command. Is it possible to make a website side file that i can go to to add users instead of having to use the command line?

[liamgreen]

Arhh right i see. I'm glad it's possiable. And right will get looking into all this. All i want is to be able to add users to the file via a simple form. Don't want to be able to manage users etc. can do that via the WinSCP remove them that way. All i want is like i say a simple from to just add the username and password to the htpasswd file then go to example.com:8081 and login

[danraf77]

Hello jprjr:
Grear Job.
A have a problem.
./bin/multistreamer -l /opt/openresty-rtmp/luajit/bin/luajit -e production run
then
[error] 9379#9379: init_worker_by_lua error: ./multistreamer/config.lua:1: module 'lapis.config' not found:
Why?
thank you