tomeshnet / ipfs-live-streaming Goto Github PK

Since Our Networks is happening shortly, we are finalizing features so we can tag this and run a persistent cluster, then let the designers polish player UI and @ASoTNetworks stress test with his OBS on-premise set up. Here are remaining tasks before tagging:

• Disable IPNS due to #3
• Publish one-time a message on IPNS message when ipfs-server starts up, stating IPNS currently disabled
• By default, use /var/www/html/live.m3u8 to stream, both on ipfs-server and ipfs-mirror, the player URL should not need the ?url=live.m3u8 param
• Use plain old wget to distribute live.m3u8 to mirrors (since IPNS is disabled and ipfs pubsub pub <topic> <data> is too late to make the feature cut)
• Work across IPFS restart #27
• Work across ffmpeg restart #28

Let's link the presentation video to our repo, it's a good explanation to how this works.

https://www.youtube.com/watch?v=VFsKl6UMwqM

Thoughts about the following?

1. Perhaps download from YouTube and link a IPFS pinned version of the raw file?
2. Pin the embedded player with it?
3. Crop off the rest of the call so it begins at the Live Streaming presentation?

IPNS publishing takes more than a minute to propagate which is currently a blocker.

Examples of the different ways to access ipfs resources would be helpful.

i.e. the README has the params needed for different settings, which is good:

Parameter | Description
-- | --
gw | Set IPFS gateway URL to override playback gateway
live | Set m3u8 file URL to override IPFS live stream
vod | Set IPFS content hash of mp4 file to play IPFS on-demand video stream

An improvement would be to show the usage also i.e.

https://mysite.com/?vod=QmPk33q8z8GHhJoDKXGDGn5CWFu61TzTprgupfGe4VNwho

also, a clarification of when to use a gateway would be helpful, because for live streams it appears that a gateway is required but for vod it's not required:

https://mysite.com/?live=https://ipfs.io/ipns/QmZQeeZUCX1boerKc8VH42JRoXDNys1XCgVYnPd18EQtDT

Add support for VOD streams: ournetworks/ournetworks.ca#79
Maybe some minor changes from: ournetworks/ournetworks.ca#78

Next talk is...

See: ournetworks/2018#26 (comment)

Currently the player just defaults to the last known good block of video. We need some way of the stream actually not being live for when we are not broadcasting.

ipfs pin add takes about a minute, too slow for pinning 15 second ts chunks:
https://github.com/tomeshnet/ipfs-live-streaming/blob/master/terraform/ipfs-mirror/ipfs-pin.sh#L6

@darkdrgn2k mentioned adding poor man's DNS load balancing to the provisioning. If we deploy multiple mirrors it's probably a good idea to also front that infrastructure with a load balancer. Digital Ocean has this option for us https://www.terraform.io/docs/providers/do/r/loadbalancer.html should we need it.

Two sets of videos will be archived:

1. All live streamed 720p HLS chunks published as a single DVR m3u8 to ipfs end of each day (how to do this should be documented)
2. OBS will record 1080p .mp4 of each talk individually, stored on a hard disk on-site (how these get published is out of scope for this repo)

#44 (comment)

You can unpin all the pinned segments by running

cat /var/www/html/live.log | awk '{print $2}' | xargs -n 1 ipfs pin rm
ipfs repo gc
rm -rf  /var/www/html/live.log

The M3U8 file may have a race condition.

This can be seen when the first chunk is processed. The m3u8 file is missing after the first chunk is created.

Most of the time, people use their phone to watch the video or upload a video.
Is this possible to use in Mobile, Android or IOS?

The Readme has a lot of ournetworks specific info in it which should probably be generalized.

Also implementing #69 wouldn't be a bad plan.

Also I've planned to do this for a while and now there's an issue with my name on it :P

on rtmp or maybe optional new server

code to activate
acd4eb5

Mirrors will not re-chunk from the RTMP stream, but will:

• pin all .ts and .m3u8 from ipfs-server
• run ipfs http gateways with pubsub

This needs to be added to our Vagrant file as part of rtmp-server Droplet, auth to @ASoTNetworks's streaming cluster for the non-ipfs HLS streams and push RTMP stream to it. @ASoTNetworks prefers pushing rather than have his server pull from that stream.

See: #71 (comment)

Using HLS from nginx-rtmp server it is possible to creates a multiple stream environment (ie multiple room setup)

setting different streamkeys per obs instance, nginx-rtmp will creates seperate STREAMEKEY.m3u8 file and STREAMKEY-nnn.ts files in the hsl directory.

Following updates to process-stream.sh file would need to be done to support this

• create loop to iterate through all m3u8 files in the hls folder
• run processing script against every m3u8 file
• rename current.m3u8 and live.m3u8 files to represent stream names
• if using ipns create IPFS crypo key per stream and use that
• announce multiple streams in player

There are issues not addressed in the v0.1 tag, but they are non-critical / randomly occurring cases that would benefit from being worked on off of a stable v0.1 tag. We expect these to happen over next week:

• Verify streaming with conference setup for 10+ hours @ASoTNetworks
• Investigate occasional black-screen video or jitter frames after ffmpeg restart @darkdrgn2k
• Polish UI for player @Shrinks99 @garrying
• Failures after ~30 chunks when @benhylau streams the MetaMesh clip resulting in "non-monotonous dts in output stream 0:1" all over the logs
• One-site testing at Mozilla @darkdrgn2k @ASoTNetworks @Shrinks99 @benhylau

add following under EACH server directive

    expires           0;
    add_header        Cache-Control private;

i do it following the readme, but get the error, what's the problem? thanks very much!!!

cat  process-stream.log  | grep 2019-05 |awk '{print $6"#EXTINF:"$5"\nhttps://ipfs-gateway.live.mesh.world/ipfs/"$2}' | sed  's/#EXT-X-DISCONTINUITY/#EXT-X-DISCONTINUITY\n/' > /var/www/html/vcr.m3u8

you can play it using
https://domain.com/?live=vcr.m3u8

Sample code for how to embed players on the Our Networks (or any other) website.

shouldn't be to hard to get the base system installed on a debian system instead of terraform using terraform scripts

• copy rtmp-server to /tmp/rtmp-server
• copy shared/video-player to /tmp/video-player
• set permissions

      "chmod +x /tmp/rtmp-server/bootstrap.sh",
      "chmod +x /tmp/rtmp-server/bootstrap-post-dns.sh",
      "chmod +x /tmp/rtmp-server/process-stream.sh",

• run "/tmp/rtmp-server/bootstrap.sh DOMAIN_NAME E-MAIL noop M3U8 address
• Deal with dns entries
• Deal with bootstrap-post-dns.sh
• Deal with predictive naming of interfaces debian9

Strange increase in consumption of disk space noticed

Size of chunks seen to increase from 2-3Mb to 5-6Mb

This happened at Exactly (or almost exactly) 4:00 PM

From the top of the hour you can see the chunks increasing. These are the first 5 from 4:00PM

user@ubuntu:~/Downloads/1$ ffprobe 1.ts 2>&1  | grep bitrate
  Duration: 00:00:16.05, start: 2462.018189, bitrate: 1752 kb/s
user@ubuntu:~/Downloads/1$ ffprobe 2.ts 2>&1  | grep bitrate
  Duration: 00:00:14.05, start: 2478.016733, bitrate: 2709 kb/s
user@ubuntu:~/Downloads/1$ ffprobe 3.ts 2>&1  | grep bitrate
  Duration: 00:00:16.05, start: 2492.018367, bitrate: 2713 kb/s
user@ubuntu:~/Downloads/1$ ffprobe 4.ts 2>&1  | grep bitrate
  Duration: 00:00:14.05, start: 2508.016922, bitrate: 2783 kb/s
user@ubuntu:~/Downloads/1$ ffprobe 5.ts 2>&1  | grep bitrate
  Duration: 00:00:16.05, start: 2522.018556, bitrate: 3226 kb/s

However stream played find

VideoJS has a whole bunch of plugins that could add some useful features to the player and could possibly make #74 easier to actually implement. They can be found here.

Seems like this could make for a few nice easy features & quality of life improvements on the front end, thoughts?

• Use ipfs add -Q to get just the HASH file would be much more cleaner since it just provides the hash, however several places need to be modified to get it to work. ( ref tomeshnet/prototype-cjdns-pi#158 )

This happened on the ffmpeg server after a streaming for a long time. Notice chunk durations:

added QmZvituPPVigWxMMuz8wwbowbTokytSdLh5yA3FjgNUiCo 201807030033-LIVE277.ts 2018-07-03-02-03-05.ts 16.666000
added QmVojZz89P4Cn9zZZcbywsFMBbYHY5Ex3FhKCaVeKJHwci 201807030033-LIVE278.ts 2018-07-03-02-03-21.ts 16.667000
added QmXH5cXWkWcB8iGuV5TWoEfAjPQG2bdESU3c3viDgaH92L 201807030033-LIVE279.ts 2018-07-03-02-03-38.ts 16.667000
added QmUuy3M5v9S6cT8LAntoDN923GDzu2NAPMAGd2hnDjnVw7 201807030033-LIVE280.ts 2018-07-03-02-03-46.ts 8.333000
added QmdamU22Qn8uAvSF17aymFRrPRbeJw7pbaaJBuMTwhkZWn 201807030033-LIVE281.ts 2018-07-03-02-04-03.ts 16.667000
added QmQUC85DwT1piMmwfZ7zRxDX8BwF1pkpKCtyKrSUcAxKwT 201807030033-LIVE282.ts 2018-07-03-02-04-19.ts 16.033000
added QmUVAAEcej3SaQVWJXuwRvHJkMhZnyYkrrEQ9xn98kjMJK 201807030033-LIVE283.ts 2018-07-03-02-04-35.ts 15.700000
added QmUoHNQ47666xgWnSsucXfrfEGAmouk6WCpeKPFD77wYwa 201807030033-LIVE284.ts 2018-07-03-02-04-49.ts 14.467000
added QmfS9tNyURgfDGcN4xfmKCdcG8d63p3GEdGDoBESG9ZX5c 201807030033-LIVE285.ts 2018-07-03-02-05-06.ts 16.666000
added QmcCUvqM9q5faZYkftndGnYYk272Z5hmVXP144XXoJo7HW 201807030033-LIVE286.ts 2018-07-03-02-05-18.ts 11.900000
added QmUA6AQ83ynUJ1QtvoWDZZcgUUBmrktb6127c83isy3ZQV 201807030033-LIVE287.ts 2018-07-03-02-05-32.ts 13.034000
added QmVuxAqZWRvt3rz1DEU42zoeTEVxeiiQY1zZSZEiCP8zjj 201807030033-LIVE288.ts 2018-07-03-02-06-19.ts 15.000000
added QmQnuqLe5WFVpMn2Jmy33mRtR2toXNSgGKs31L6xsdiPdo 201807030033-LIVE289.ts 2018-07-03-02-20-26.ts 0.001956
added Qmadk5JnsqKaAz7fc1W5khyAxCfMq2hH7dcqWAEXhFkYFq 201807030033-LIVE290.ts 2018-07-03-02-20-33.ts 0.002167
added QmeDuvphcHiLxmz56N8KdDp2E4NEWbPec7BAEQ2AnUtSyY 201807030033-LIVE291.ts 2018-07-03-02-20-41.ts 0.002778
added QmQexUqn8ey1vH6NgpZ9Dv3KtTgyg32YBQFwCFF5gBKoyG 201807030033-LIVE292.ts 2018-07-03-02-20-50.ts 0.002778
added QmbVmhL4omnY4iWefzJpZtgQ4rTbRKJADvUWUK4JpsNjd1 201807030033-LIVE293.ts 2018-07-03-02-20-55.ts 0.001878
added QmTqWSqAB7QzVCv1aGRkCkcP1Hi9txkZDykUm6EDue6v6R 201807030033-LIVE294.ts 2018-07-03-02-21-03.ts 0.002778
added QmQ5ovnJcC1ELAeAmei4qUrse2RfypywKcXjrh5A4btUar 201807030033-LIVE295.ts 2018-07-03-02-21-12.ts 0.002778
added QmPrsJo5Wq7mTtimBFdsMuffANkgePGRVZN6bWj2ypF8FH 201807030033-LIVE296.ts 2018-07-03-02-21-20.ts 0.002267
added QmaJsJhfcR8eUrP9p3Mvb5g3MrCTmSeWRuC784z1xgsJ9w 201807030033-LIVE297.ts 2018-07-03-02-21-28.ts 0.002778
added QmXZGExpKqz7DTKpjw9rkKVGfRUsr6H5XtyUaiXvGLDHo5 201807030033-LIVE298.ts 2018-07-03-02-21-33.ts 0.001644
added QmXgQ3x2Q3hDpHJkV5BeYzTaovaR7hSQd5gH4oAs46g9N9 201807030033-LIVE299.ts 2018-07-03-02-21-37.ts 0.001222
added QmYuibHcwSCJ3msb1T6BmSaWHcF5n8EBH8sFpj9BgJSumG 201807030033-LIVE300.ts 2018-07-03-02-21-41.ts 0.001433
added QmauzZEQ2EQjSrR4B6Mi25Kx2JqezTZi4jRAVpHPPniUeb 201807030033-LIVE301.ts 2018-07-03-02-21-44.ts 0.000778
added QmUNqEBeK6ttRdD7bxHWUg7mcgAtvkoAvBe7FAgAnd3VQv 201807030033-LIVE302.ts 2018-07-03-02-21-49.ts 0.001722
added Qmeq3be2iqSuBSmXgkEsEZW2XUCXNGsc2Vh1UZHZEhm7qY 201807030033-LIVE303.ts 2018-07-03-02-21-52.ts 0.000922
added QmWk6XrdGqd4tsE8rBXSntqEs5Xz8asZnepQxXYthYDZaH 201807030033-LIVE304.ts 2018-07-03-02-21-53.ts 0.000589
added QmZV6D7fC1wfGq1SteVFfH4g2E37Pv9pAu4PBfugPsvxAn 201807030033-LIVE305.ts 2018-07-03-02-21-55.ts 0.000400
added QmQTpLQkXo49XYYKpd8RakWcJsr9Cciosx47W9rrevKFtY 201807030033-LIVE306.ts 2018-07-03-02-22-00.ts 0.001833
added Qmc7v1seFBpga8X7AMUWXGnSfoii2Ytu7oxjcfrZMkLeo9 201807030033-LIVE307.ts 2018-07-03-02-22-08.ts 0.002778
added QmRo8bodcCT6aj1kJiCP1EAmgKDuhrwHCwFPRV6SmnQbNE 201807030033-LIVE308.ts 2018-07-03-02-22-12.ts 0.001156
added QmajgeZ6zcfXiKcFaRiwon8dM6UcTXkxCLumqRoS7Vh1ss 201807030033-LIVE309.ts 2018-07-03-02-22-16.ts 0.001522
added QmXcHtiFGS1ME7z5ASS8Tcoo1cRxscZCujWuPsUpuyA7Rr 201807030033-LIVE310.ts 2018-07-03-02-22-21.ts 0.001556
added QmVTcqAFikVmvaZFF2MSVHgG47jpjLc6zg2py5ohg1nP48 201807030033-LIVE311.ts 2018-07-03-02-22-29.ts 0.002778
added QmVG8S5MuDoiQk5hVznwETokSPf5FDvyHMHwAsNuMCKhQR 201807030033-LIVE312.ts 2018-07-03-02-22-38.ts 0.002778
added QmcpQm8eekq8SLASXGHotjXfoLZWsmPtU6NM8cSp8mEWNo 201807030033-LIVE313.ts 2018-07-03-02-22-45.ts 0.002456
added Qme4Km7WvVVvhqM3uhzUpYF7w4zJ4KepXCsditbTfY3UXZ 201807030033-LIVE314.ts 2018-07-03-02-22-50.ts 0.001622
added QmdJZf7AhCnDnB8asF57NUCN41nGL3J5pyYBB3KxdRZLt3 201807030033-LIVE315.ts 2018-07-03-02-22-56.ts 0.002056
added QmSSt3cDNzM5PjVBQqKBmLuoqgkQDZVgSvq4bcmAxCXtYm 201807030033-LIVE316.ts 2018-07-03-02-23-03.ts 0.002122
added QmegcAY3Z1NecZuS5M2g9XH9SYaCMMwB1F2dCLnyuTGhQJ 201807030033-LIVE317.ts 2018-07-03-02-23-10.ts 0.002556
added QmdDjoobxw71XFh9MHyVCuBxtVMmQsz3CgktA6G63KF6gt 201807030033-LIVE318.ts 2018-07-03-02-23-13.ts 0.000800
added QmRB2THaY6SASDC5gpTDP72ZpGfoaRQMjfSmL26zc7v4Qn 201807030033-LIVE319.ts 2018-07-03-02-23-16.ts 0.001056
added Qmd9B795zArARN7b4VTfJHkGeLVumQGdxDedGPvgkwJdaK 201807030033-LIVE320.ts 2018-07-03-02-23-20.ts 0.001544
added QmcJEdDzsuMJgVtDsJXygvtNP9tZccxwp9KVTUZkKxDq5h 201807030033-LIVE321.ts 2018-07-03-02-23-25.ts 0.001356
added QmcpFhrNsz3Qgj1yM8wtx4MgqpMW7gWZV8kBSQ7erfausq 201807030033-LIVE322.ts 2018-07-03-02-23-26.ts 0.000611
added QmVe8C3n2Cofkn9DHwEscMifhXhxsMmjgdBLHgeugAKwE1 201807030033-LIVE323.ts 2018-07-03-02-23-34.ts 0.002633
added QmPDAdh13YvvToAvfKurrdqy38huwthMTZo9X88ZyhkLai 201807030033-LIVE324.ts 2018-07-03-02-23-40.ts 0.001833
added QmcP8mm7VSZuJp4TDJbic9LhKw58q6rioJFyijaKGhVuud 201807030033-LIVE325.ts 2018-07-03-02-23-43.ts 0.001189
added QmV1t7nGwLxx7NtRhAEFuRiUjhZkdrqHoNqx339PPGmihQ 201807030033-LIVE326.ts 2018-07-03-02-23-46.ts 0.000867
added QmPVghtAM9yFru61P9eHUJWQKvj6voUgw2tzrbPe1biL87 201807030033-LIVE327.ts 2018-07-03-02-23-48.ts 0.000733
added QmVi9mMjvrUcZGJ7bu32E3evcrvEnu3GvhKExvLmPqseMN 201807030033-LIVE328.ts 2018-07-03-02-23-52.ts 0.001289
added QmdDvjWErnCszPusrEUXgnfeofBoQc5naAiHjk2nZ5gwdc 201807030033-LIVE329.ts 2018-07-03-02-23-58.ts 0.002022
added QmcfhzKPxR7KHgdy5zgsHkRebKG8WEzCX3kcs9KshrycVi 201807030033-LIVE330.ts 2018-07-03-02-24-05.ts 0.002378
added QmPvyszp2crU8LWE3iphT1jFMDuxCYMqjpoM45XLSzpsHi 201807030033-LIVE331.ts 2018-07-03-02-24-09.ts 0.001344
added QmTJEw9ASCa2hQ14PYZZoa2Z7hDVToqzaqC7svyodnpLFw 201807030033-LIVE332.ts 2018-07-03-02-24-15.ts 0.001756
added QmX84gNkUywxGFXaz6s2tLbabKpN6anaGtbuqZxHZ56YZ6 201807030033-LIVE333.ts 2018-07-03-02-24-19.ts 0.001611
added QmQ5gXFMgEU66CNPzAHhuoNaNTBjdmiStJCRTVTJKdBgLe 201807030033-LIVE334.ts 2018-07-03-02-24-27.ts 0.002478
added QmNTTEg8ec6EDQ2pwbQPp8PGCzfPdCnxHKFeWwHyjXb695 201807030033-LIVE335.ts 2018-07-03-02-24-34.ts 0.002378
added QmWdDaug5XDe24ypQ9tuAdMq2FYFrFw8bXTZ7kaQ24T2Mg 201807030033-LIVE336.ts 2018-07-03-02-24-42.ts 0.002778
added QmcRUGAPtFWrZRMTfVnxnkeyaPAw2gG6F5Wt5xt8FeL8Tu 201807030033-LIVE337.ts 2018-07-03-02-24-50.ts 0.002578
added QmVV8dtznV6m6fQjd81yv5vikaLE5W48KLraQcXkAAoE7E 201807030033-LIVE338.ts 2018-07-03-02-24-56.ts 0.001878
added QmayCpDeS3yVn4PWxRLZZBDvfwDVF1Q9FWiqnKW5a6TFKc 201807030033-LIVE339.ts 2018-07-03-02-24-59.ts 0.000656
added QmSdkTwDsA7ZXek7H46LHHic3zm1gE9DqG7w25xFWC96eL 201807030033-LIVE340.ts 2018-07-03-02-25-00.ts 0.000789
added QmVC9xzBdTvNSvwYDoWQ1BLhwuPpuraE8kiYwxy65nB3ox 201807030033-LIVE341.ts 2018-07-03-02-25-06.ts 0.002011
added QmbGhAdgCxdKRGXk3WWPFXrqxsABP4npULKnYm7Mt9LGLb 201807030033-LIVE342.ts 2018-07-03-02-25-54.ts 15.000000

Then I killed the source stream and that was the end of output. Only after that the m3u8s on both server and mirror synced up. Previously everything including the local ffmpeg generated m3u8 in the live folder was stuck around LIVE287 LIVE288.

This error in ffmpeg logs didn't come up until late into this situation:

Non-monotonous DTS in output stream 0:1;

On the player side, choppy playback and looping content is observed, which is what prompted me to look at logs.

We can move the discussion from ournetworks/2018#26 to here.

Hi all,
I followed the program to do,but I didn't find the new file in my .keys folder.
Guide me.Thank you.
Log :

Progress: [ 98%] [#################.] te-exec): Processing triggers for libc-bin (2.24-11+deb9u3) ...

digitalocean_droplet.rtmp-server (remote-exec): Processing triggers for systemd (232-25+deb9u6) ...
NOTE: If you run ./clean-all, I will be doing a rm -rf on /usr/share/easy-rsa/keys
digitalocean_droplet.rtmp-server (remote-exec): Generating DH parameters, 2048 bit long safe prime, generator 2
digitalocean_droplet.rtmp-server (remote-exec): This is going to take a long time
digitalocean_droplet.rtmp-server (remote-exec): ......
digitalocean_droplet.rtmp-server (remote-exec): .........................................................................................................+.........................................................................+.......................+.............................+......................................................................+..++++++++
digitalocean_droplet.rtmp-server (remote-exec): Using CA Common Name: Fort-Funston CA
digitalocean_droplet.rtmp-server (remote-exec): Generating a 2048 bit RSA private key
digitalocean_droplet.rtmp-server (remote-exec): .............................+++++
digitalocean_droplet.rtmp-server (remote-exec): ...........................+++++
digitalocean_droplet.rtmp-server (remote-exec): writing new private key to 'ca.key'
digitalocean_droplet.rtmp-server (remote-exec): -----
digitalocean_droplet.rtmp-server (remote-exec): Generating a 2048 bit RSA private key
digitalocean_droplet.rtmp-server (remote-exec): ............+++++
digitalocean_droplet.rtmp-server (remote-exec): ..........................+++++
digitalocean_droplet.rtmp-server (remote-exec): writing new private key to 'server.key'
digitalocean_droplet.rtmp-server (remote-exec): -----
digitalocean_droplet.rtmp-server (remote-exec): Using configuration from /usr/share/easy-rsa/openssl.cnf
digitalocean_droplet.rtmp-server (remote-exec): Can't open /usr/share/easy-rsa/keys/index.txt.attr for reading, No such file or directory
digitalocean_droplet.rtmp-server (remote-exec): 140107777611136:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:74:fopen('/usr/share/easy-rsa/keys/index.txt.attr','r')
digitalocean_droplet.rtmp-server (remote-exec): 140107777611136:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:81:
digitalocean_droplet.rtmp-server (remote-exec): Check that the request matches the signature
digitalocean_droplet.rtmp-server (remote-exec): Signature ok
digitalocean_droplet.rtmp-server (remote-exec): The Subject's Distinguished Name is as follows
digitalocean_droplet.rtmp-server (remote-exec): countryName :PRINTABLE:'US'
digitalocean_droplet.rtmp-server (remote-exec): stateOrProvinceName :PRINTABLE:'CA'
digitalocean_droplet.rtmp-server (remote-exec): localityName :PRINTABLE:'SanFrancisco'
digitalocean_droplet.rtmp-server (remote-exec): organizationName :PRINTABLE:'Fort-Funston'
digitalocean_droplet.rtmp-server (remote-exec): organizationalUnitName:PRINTABLE:'MyOrganizationalUnit'
digitalocean_droplet.rtmp-server (remote-exec): commonName :PRINTABLE:'server'
digitalocean_droplet.rtmp-server (remote-exec): name :PRINTABLE:'EasyRSA'
digitalocean_droplet.rtmp-server (remote-exec): emailAddress :IA5STRING:'[email protected]'
digitalocean_droplet.rtmp-server (remote-exec): Certificate is to be certified until Dec 9 04:34:40 2028 GMT (3650 days)
.
.
.
.
Error: Error applying plan:

1 error(s) occurred:

• digitalocean_droplet.rtmp-server: error executing "/tmp/terraform_1732481365.sh": Process exited with status 1

When the RTMP source stream disconnects and reconnects, ffmpeg gets into a bad state. To fix, SSH into ipfs-server:

tail -f process-stream.log # Confirm the log has stalled for 30 s
tail -f /var/log/ffmpeg # Verify ffmpeg is outputting lots of error messages
systemctl restart process-stream.service # Restart stream processing
tail -f /var/log/ffmpeg # Verify ffmpeg is back to normal

Currently the stream choice UI takes a long time and can be quite redundant if there is only one stream source.

• Implement a source selector gear icon & interface (also used for quality if / when that gets implemented)
• Do not show the stream selector when there is only one stream. That one stream is all that should display.
• Autoplay! The player should now default to one stream all the time (IPFS or HTTP, default should be specified in a config or something). Stretch goal to switch to the other automatically if one is not working.

Video.js has a plugin system as well as theme changing abilities that are pretty easy for me to get my head around. I'll probably look into those too.

Related: https://github.com/streamplace/streamplace
SSB thread: %qUEu7ZdN4UodENsUZRX7YsaKDjshKsbXGTB4YGexLHQ=.sha256

We should see if there can be integration points.

current.m3u8

#EXTINF:16.101000,
http://ipfs-server.mesh.world:8080/ipfs/QmUNrg8dZhxUNMXzrF97U6kp8mAR34Gr8d1znFMHYpomeB
#EXTINF:13.799000,
http://ipfs-server.mesh.world:8080/ipfs/QmS8kJNy4wmiwX8a4yzCJB3ah2jcb9XMrFUWg4bi8ybuX3
#EXTINF:,
http://ipfs-server.mesh.world:8080/ipfs/14.767000
#EXTINF:18.299000,
http://ipfs-server.mesh.world:8080/ipfs/QmeoZZdm4ZfMZ1S5LRfkmwxqZ3bTafH9fDPWEHKuo1NQWy
#EXTINF:16.068000,
http://ipfs-server.mesh.world:8080/ipfs/QmXbmGjZemK7D6b5NMUZsPdPXAsANPvwGZJckZyeB7Z21a
#EXTINF:9.866000,
http://ipfs-server.mesh.world:8080/ipfs/QmXfRvMcfVd9XH2gy5VSB1xbGNzQ1i62Va6pRrUPk8nbrN
#EXTINF:14.767000,
http://ipfs-server.mesh.world:8080/ipfs/QmPzghqaMvgg9sGV4uJMWUNjyiMDYQBEn69ChaLEmC93Cf
#EXTINF:16.267000,
http://ipfs-server.mesh.world:8080/ipfs/QmVkeNvSvjFNFTsWQbtVj1v55jMWaKAL48bLTv8xUPufbJ
#EXTINF:14.065000,
http://ipfs-server.mesh.world:8080/ipfs/QmaUxfz9gETwRP8hkY2i1H5rmHppxBmdVLWVFyDBXXEoHa
#EXT-X-ENDLIST

process-stream.log:

added QmUNrg8dZhxUNMXzrF97U6kp8mAR34Gr8d1znFMHYpomeB 201807021737-LIVE49.ts 2018-07-02-17-49-45.ts 16.101000
added QmS8kJNy4wmiwX8a4yzCJB3ah2jcb9XMrFUWg4bi8ybuX3 201807021737-LIVE50.ts 2018-07-02-17-49-59.ts 13.799000
2018-07-02-17-50-14.ts 14.767000
added QmeoZZdm4ZfMZ1S5LRfkmwxqZ3bTafH9fDPWEHKuo1NQWy 201807021737-LIVE52.ts 2018-07-02-17-50-32.ts 18.299000
added QmXbmGjZemK7D6b5NMUZsPdPXAsANPvwGZJckZyeB7Z21a 201807021737-LIVE53.ts 2018-07-02-17-50-48.ts 16.068000
added QmXfRvMcfVd9XH2gy5VSB1xbGNzQ1i62Va6pRrUPk8nbrN 201807021737-LIVE54.ts 2018-07-02-17-50-58.ts 9.866000
added QmPzghqaMvgg9sGV4uJMWUNjyiMDYQBEn69ChaLEmC93Cf 201807021737-LIVE55.ts 2018-07-02-17-51-13.ts 14.767000

See: ournetworks/2018#26 (comment)

The file we want to distribute is the changing live.m3u8 because IPNS isn't reliable, and ipfs pubsub pub <topic> <data> seems to work well.

First we need to enable --enable-pubsub-experiment in the /etc/systemd/system/ipfs.service for all the ipfs nodes. Then on server:

ipfs add live/current.m3u8 | awk '{print $2","}' | ipfs pubsub pub m3u8

On mirrors, run:

ipfs pubsub sub m3u8 > m3u8-sub.log

and in a crontab:

ipfs cat `awk -F, '{ print $(NF-1) }' m3u8-sub.log` > live.m3u8

We decided not to use this for Our Networks since:

• Anyone can pub, we don't have time to implement access control before the conference
• pubsub lacks an interface easy for video player to consume (e.g. http://<gateway>:8080/pubsub/<topic>)
• We know wget is reliable and functionally this is just another way for us to do that without the benefits of IPNS

Hi there, try to run textileio desktop and get below error, any help? thanks in advance.

DB-MBP-MCG8WL:desktop mlu$ go run *.go -v
DEBU[0000] Running app built at
DEBU[0000] Resources folder doesn't exist, restoring resources...
DEBU[0000] Removing /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/resources
DEBU[0000] Restoring resources in /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/resources
DEBU[0000] Starting...
DEBU[0000] Provisioning...
DEBU[0000] Provisioning Astilectron...
DEBU[0000] Removing directory /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron
DEBU[0000] Disembedding vendor_astilectron_bundler/astilectron.zip into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron-v0.27.0.zip...
DEBU[0000] Creating /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor
DEBU[0000] Creating /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron-v0.27.0.zip
DEBU[0000] Disembedding vendor_astilectron_bundler/astilectron.zip
DEBU[0000] Copying disembedded data to /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron-v0.27.0.zip
DEBU[0000] Creating directory /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron
DEBU[0000] Unzipping /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron-v0.27.0.zip/astilectron-0.27.0 into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron
DEBU[0000] Provisioning Electron...
DEBU[0000] Removing directory /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64
DEBU[0000] Disembedding vendor_astilectron_bundler/electron.zip into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64-v1.8.1.zip...
DEBU[0000] Creating /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor
DEBU[0000] Creating /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64-v1.8.1.zip
DEBU[0000] Disembedding vendor_astilectron_bundler/electron.zip
DEBU[0000] Copying disembedded data to /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64-v1.8.1.zip
DEBU[0000] Creating directory /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64
DEBU[0000] Unzipping /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64-v1.8.1.zip into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64
DEBU[0002] Finishing provisioning electron for darwin system
DEBU[0002] Copying /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/resources/icon.icns to /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app/Contents/Resources/electron.icns
DEBU[0002] Replacing in /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app/Contents/Info.plist
DEBU[0002] Replacing in /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app/Contents/Frameworks/Electron Helper EH.app/Contents/Info.plist
DEBU[0002] Replacing in /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app/Contents/Frameworks/Electron Helper NP.app/Contents/Info.plist
DEBU[0002] Replacing in /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app/Contents/Frameworks/Electron Helper.app/Contents/Info.plist
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Electron.app into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/MacOS/Electron into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/MacOS/Textile
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Electron Helper EH.app into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper EH.app
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper EH.app/Contents/MacOS/Electron Helper EH into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper EH.app/Contents/MacOS/Textile Helper EH
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Electron Helper NP.app into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper NP.app
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper NP.app/Contents/MacOS/Electron Helper NP into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper NP.app/Contents/MacOS/Textile Helper NP
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Electron Helper.app into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper.app
DEBU[0002] Renaming /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper.app/Contents/MacOS/Electron Helper into /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/Frameworks/Textile Helper.app/Contents/MacOS/Textile Helper
DEBU[0002] Listening...
DEBU[0002] Executing...
DEBU[0002] Starting cmd /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/electron-darwin-amd64/Textile.app/Contents/MacOS/Textile /var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/vendor/astilectron/main.js 127.0.0.1:50164
DEBU[0002] Astilectron says: {"name":"app.event.ready","targetID":"app","displays":{"all":[{"id":69733248,"bounds":{"x":0,"y":0,"width":1440,"height":900},"workArea":{"x":0,"y":23,"width":1440,"height":822},"size":{"width":1440,"height":900},"workAreaSize":{"width":1440,"height":822},"scaleFactor":2,"rotation":0,"touchSupport":"unknown"},{"id":441019671,"bounds":{"x":-504,"y":-1440,"width":2560,"height":1440},"workArea":{"x":-504,"y":-1417,"width":2560,"height":1417},"size":{"width":2560,"height":1440},"workAreaSize":{"width":2560,"height":1417},"scaleFactor":1,"rotation":0,"touchSupport":"unknown"}],"primary":{"id":69733248,"bounds":{"x":0,"y":0,"width":1440,"height":900},"workArea":{"x":0,"y":23,"width":1440,"height":822},"size":{"width":1440,"height":900},"workAreaSize":{"width":1440,"height":822},"scaleFactor":2,"rotation":0,"touchSupport":"unknown"}},"supported":{"notification":true}}
DEBU[0002] Sending to Astilectron: {"name":"window.cmd.create","targetID":"1","sessionId":"2","url":"file:///var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/resources/app/index.html","windowOptions":{"backgroundColor":"#ffffff","center":true,"height":384,"icon":"/var/folders/j5/d5jntp9s0358y3ffh3nxx_t04nmp6q/T/go-build070166248/b001/exe/resources/icon.png","show":false,"title":"Textile","titleBarStyle":"hidden-inset","width":384}}
DEBU[0002] Stderr says: 2018-10-04 13:54:52.683 Textile[64178:2533576] *** WARNING: Textured window <AtomNSWindow: 0x7fb3f2f4c110> is getting an implicitly transparent titlebar. This will break when linking against newer SDKs. Use NSWindow's -titlebarAppearsTransparent=YES instead.
DEBU[0002] Astilectron says: {"name":"window.event.ready.to.show","targetID":"1"}
DEBU[0002] Astilectron says: {"name":"window.event.did.finish.load","targetID":"1"}
DEBU[0002] Sending to Astilectron: {"name":"window.cmd.show","targetID":"1"}
DEBU[0002] Astilectron says: {"name":"window.event.focus","targetID":"1"}
DEBU[0002] Astilectron says: {"name":"window.event.show","targetID":"1"}
Successfully raised file descriptor limit to 2048.
DEBU[0005] Closing...
ERRO[0005] accept tcp 127.0.0.1:50164: use of closed network connection while TCP accepting
ERRO[0005] read tcp 127.0.0.1:50164->127.0.0.1:50165: use of closed network connection while reading
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4aac308]

goroutine 1 [running]:
textile-go-master/vendor/github.com/textileio/textile-go/gateway.(*Gateway).Start(0x0, 0xc00245e190, 0xf)
/Users/mlu/go/src/textile-go-master/vendor/github.com/textileio/textile-go/gateway/main.go:36 +0x288
main.start(0xc0002cc6c0, 0xc0001d6818, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/mlu/go/src/textile-go-master/desktop/main.go:196 +0x492
textile-go-master/vendor/github.com/asticode/go-astilectron-bootstrap.Run(0x0, 0x5153fa0, 0x5153f98, 0x0, 0x4fd306a, 0x7, 0x4fe47b5, 0x13, 0x4fe3021, 0x12, ...)
/Users/mlu/go/src/textile-go-master/vendor/github.com/asticode/go-astilectron-bootstrap/run.go:172 +0x7af
main.bootstrapApp()
/Users/mlu/go/src/textile-go-master/desktop/bootstrap.go:13 +0x34d
main.main()
/Users/mlu/go/src/textile-go-master/desktop/main.go:50 +0x2a
exit status 2

https://smartshitter.com/musings/2017/12/nginx-rtmp-streaming-with-simple-authentication/

This server pulls from rtmp-server and runs:

• HLS chunking + m3u8 assembling
• ipfs add
• ipns publish with pubsub
• ipfs http gateway with pubsub

This should be ready for this Thursday's virtual mesh testing.

Script to configure the transform using prompts

Work In progress
thoughts?

echo "--------Domain Name--------"
read -p "What is your domain name: " -r
echo -n $REPLY > .keys/domain_name

echo Make sure you point your domain dns servers to
echo ns1.digitalocean.com
echo ns2.digitalocean.com
echo ns3.digitalocean.com


echo "--------Digital Oceans API Token--------"
read -p "What is your API Token form Digital Ocean: "  -r
echo -n $REPLY > .keys/do_token

echo "--------Digital Oceans RSA--------"
echo "Press enter of password request"
ssh-keygen -t rsa -f .keys/id_rsa
echo Add the SSH key to your Digital Ocean account under Settings > Security
echo "------------------"
cat  .keys/id_rsa.pub
echo "------------------"
fingerprint=$(ssh-keygen -l -E md5 -f .keys/id_rsa.pub | awk '{print $2}' | sed 's/MD5://')
echo -n $fingerprint> .keys/ssh_fingerprint
echo Once added the finger print will be $fingerprint



echo "--------Lets Encrypt--------"
read -p "What is your e-mail (for lets encrypt): "  -r
echo -n $REPLY > .keys/email_address


echo "----Installing terraform-----"

wget https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.7_linux_amd64.zip
unzip terraform_0.11.7_linux_amd64.zip
sudo mv terraform /usr/bin


terraform init
# terraform apply

ournetworks/ournetworks.ca@fdcbe89

@darkdrgn2k @ASoTNetworks How about Wednesday night open calls every 2 weeks @appear.in/ournetworks?

@ASoTNetworks can you describe what is required from your end since you are providing the servers?

We briefly discussed license on the chat, but did not select a particular license. Toronto Mesh defaults to GPLv3 but this one is up to @ASoTNetworks and @darkdrgn2k.

Can I get 👍 if you guys are good with GPLv3, or propose otherwise in comments?

OBS device (on-premise)

• Takes video and audio feed
• scp ts chunks to multiple IPFS publishing servers (for redundancy)

IPFS publishing servers (data centre)

• High bandwidth uplink and well-peered into IPFS network
• ipfs add ts chunks received from on-premise device
• ipns publish m3u8 manifest referencing ts chunks
• Request ts and m3u8 from public HTTP gateways to warm their local IPFS cache

Dedicated HTTP gateways (data centre)

• Requires "unmetered" bandwidth cap
• Most people would be watching the streams through these gateways, unless they run a local IPFS instance
• Run one or more dedicated HTTP gateway throughout the conference (so we can ensure sufficient resource allocation as public gateways often get congested)

Archival servers (data centre)

• Does not require "unmetered" bandwidth cap
• Permanent servers to fetch and pin all video assets for archival purpose

Some of these could be physically the same server.

@ASoTNetworks you mentioned chunking could be done in the IPFS publishing servers? Is this consistent with you have in mind?

We have touched on the topic of how real-time this stream is, so I want to start documenting here.

Playing from ipfs-server m3u8 (origin):

Playing from m3u8 on ipfs-mirror:

Playing from m3u8 on ipfs-mirror but ts chunks from ipfs.infura.io:

In all cases observed latency is between 1 to 1.5 min, although I have seen as much as 2 min occasionally.