ionorg / ion-sfu

Pure Go WebRTC SFU

Ion SFU

Go implementation of a WebRTC Selective Forwarding Unit

A selective forwarding unit is a video routing service which allows webrtc sessions to scale more efficiently. This package provides a simple, flexible, high performance Go implementation of a WebRTC SFU. It can be called directly or through a gRPC or json-rpc interface.

Features

• Audio/Video/Datachannel forwarding
• Congestion Control (TWCC, REMB, RR/SR)
• Unified plan semantics
• Pub/Sub Peer Connection (O(n) port usage)
• Audio level indication (RFC6464). "X is speaking"

End to end solutions

ion-sfu is the engine behind several projects. It's designed to be focused, with minimal signaling or external dependencies. It's simple to embed ion-sfu within your service: we include a few examples inside cmd/signal.

For "batteries-included", end-to-end solutions that are easier to deploy, check out:

• LiveKit: Open source platform for real-time communication (SDKs for all major platforms)
• Ion: Real-Distributed RTC System by pure Go and Flutter

Quickstart

Run the Echo Test example

docker-compose -f examples/echotest-jsonrpc/docker-compose.yaml up

Open the client

http://localhost:8000/

SFU with json-rpc signaling

The json-rpc signaling service can be used to easily get up and running with the sfu. It can be used with the corresponding javascript signaling module.

Using golang environment

go build ./cmd/signal/json-rpc/main.go && ./main -c config.toml

Using docker

docker run -p 7000:7000 -p 5000-5200:5000-5200/udp pionwebrtc/ion-sfu:latest-jsonrpc

SFU with gRPC signaling

For service-to-service communication, you can use the grpc interface. A common pattern is to call the grpc endpoints from a custom signaling service.

Using golang environment

go build ./cmd/signal/grpc/main.go && ./main -c config.toml

Using docker

docker run -p 50051:50051 -p 5000-5200:5000-5200/udp pionwebrtc/ion-sfu:latest-grpc

Documentation

Answers to some Frequenty Asked Questions.

Examples

To see some other ways of interacting with the ion-sfu instance, check out our examples.

Media Processing

ion-sfu supports real-time processing on media streamed through the sfu using ion-avp.

For an example of recording a MediaStream to webm, checkout the save-to-webm example.

License

MIT License - see LICENSE for full text

Development

Generate the protocol buffers and grpc code:

1. Best choice (uses docker): make protos.
2. Manually:
   • Install protocol buffers and the protcol buffers compiler. On Fedora dnf install protobuf protobuf-compiler.
   • go get google.golang.org/grpc/cmd/protoc-gen-go-grpc
   • go get google.golang.org/protobuf/cmd/protoc-gen-go
   • protoc --go_out=. --go-grpc_out=. --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative cmd/signal/grpc/proto/sfu.proto

ion-sfu's Issues

Incoming unhandled RTP ssrc

I see this error pops up now and again:

pc ERROR: 2020/09/11 01:42:44 Incoming unhandled RTP ssrc(2854263694), OnTrack will not be fired. single media section has an explicit SSRC

One client only create one PC(udp connection to sfu) for exchanging many other clients media data.

Summary

One client only create one PC(udp connection to sfu) for exchanging many other clients media data.

Motivation

In current design. Let us think one will communicate with another( like 11 live chat room).

A -- SFU --- B

A pub B sub : A ----PC1----> SFU -----PC2 --->B
B pub A sub: A ----PC3---> SFU ----PC4---> B

I will setup 2 udp connections (PC) from A to SFU. If have more clients, the connection will be more.

One solution is like: A ----PC(SSRC-1-A SSRC-1-V SSRC-3-A SSRC-3-V) ---- SFU

improve api suggestion

I learned the code, really cool! simple and powful!

Some suggestion:

• Room=>Session ?
  (one peer can join two different sessions)

• Peer=>WebRTCTransport ?
  (peer derived from transport, there can by many transport: QUICTransport RTPTransport etc..)

• VideoReceiver => VideoTrackReceiver or WebRTCVideoReceiver?

• AudioReceiver => AudioTrackReceiver or WebRTCAudioReceiver?

Failed to set local answer sdp error

This error is thrown:

DOMException: Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Failed to set local answer sdp: Failed to process the bundled m= section with mid='3'.

It seems the m section is missing for mid:3.

Here is the sdp:

sdp: "v=0
↵o=- 3223776293848369481 1599851888 IN IP4 0.0.0.0
↵s=-
↵t=0 0
↵a=fingerprint:sha-256 D2:D7:C5:59:89:44:02:85:F3:55:DA:56:36:D9:8E:2A:36:2B:E3:8E:27:39:D5:CD:6C:FE:9C:A9:4F:7B:B8:00
↵a=group:BUNDLE 0 1 2 3
↵m=audio 9 UDP/TLS/RTP/SAVPF 111
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:0
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:111 opus/48000/2
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=ssrc:875501110 cname:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:875501110 msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f 676e158d-2fef-4df3-9680-db939878d284
↵a=ssrc:875501110 mslabel:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:875501110 label:676e158d-2fef-4df3-9680-db939878d284
↵a=msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f 676e158d-2fef-4df3-9680-db939878d284
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates
↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 125 108 124 123
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:1
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 goog-remb 
↵a=rtcp-fb:96 ccm 
↵a=rtcp-fb:96 nack 
↵a=rtcp-fb:96 nack pli 
↵a=rtpmap:98 VP9/90000
↵a=fmtp:98 profile-id=0
↵a=rtcp-fb:98 goog-remb 
↵a=rtcp-fb:98 ccm 
↵a=rtcp-fb:98 nack 
↵a=rtcp-fb:98 nack pli 
↵a=rtpmap:100 VP9/90000
↵a=fmtp:100 profile-id=2
↵a=rtcp-fb:100 goog-remb 
↵a=rtcp-fb:100 ccm 
↵a=rtcp-fb:100 nack 
↵a=rtcp-fb:100 nack pli 
↵a=rtpmap:102 H264/90000
↵a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
↵a=rtcp-fb:102 goog-remb 
↵a=rtcp-fb:102 ccm 
↵a=rtcp-fb:102 nack 
↵a=rtcp-fb:102 nack pli 
↵a=rtpmap:127 H264/90000
↵a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
↵a=rtcp-fb:127 goog-remb 
↵a=rtcp-fb:127 ccm 
↵a=rtcp-fb:127 nack 
↵a=rtcp-fb:127 nack pli 
↵a=rtpmap:125 H264/90000
↵a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtcp-fb:125 goog-remb 
↵a=rtcp-fb:125 ccm 
↵a=rtcp-fb:125 nack 
↵a=rtcp-fb:125 nack pli 
↵a=rtpmap:108 H264/90000
↵a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
↵a=rtcp-fb:108 goog-remb 
↵a=rtcp-fb:108 ccm 
↵a=rtcp-fb:108 nack 
↵a=rtcp-fb:108 nack pli 
↵a=rtpmap:124 H264/90000
↵a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
↵a=rtcp-fb:124 goog-remb 
↵a=rtcp-fb:124 ccm 
↵a=rtcp-fb:124 nack 
↵a=rtcp-fb:124 nack pli 
↵a=rtpmap:123 H264/90000
↵a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
↵a=rtcp-fb:123 goog-remb 
↵a=rtcp-fb:123 ccm 
↵a=rtcp-fb:123 nack 
↵a=rtcp-fb:123 nack pli 
↵a=ssrc:1701979011 cname:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:1701979011 msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f b51abe50-316f-4fc2-a6de-504846451f95
↵a=ssrc:1701979011 mslabel:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:1701979011 label:b51abe50-316f-4fc2-a6de-504846451f95
↵a=msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f b51abe50-316f-4fc2-a6de-504846451f95
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates
↵m=audio 9 UDP/TLS/RTP/SAVPF 111
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:2
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:111 opus/48000/2
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=ssrc:3372783641 cname:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:3372783641 msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=ssrc:3372783641 mslabel:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:3372783641 label:5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates
↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 125 108 124 123
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:3
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 goog-remb 
↵a=rtcp-fb:96 ccm 
↵a=rtcp-fb:96 nack 
↵a=rtcp-fb:96 nack pli 
↵a=rtpmap:98 VP9/90000
↵a=fmtp:98 profile-id=0
↵a=rtcp-fb:98 goog-remb 
↵a=rtcp-fb:98 ccm 
↵a=rtcp-fb:98 nack 
↵a=rtcp-fb:98 nack pli 
↵a=rtpmap:100 VP9/90000
↵a=fmtp:100 profile-id=2
↵a=rtcp-fb:100 goog-remb 
↵a=rtcp-fb:100 ccm 
↵a=rtcp-fb:100 nack 
↵a=rtcp-fb:100 nack pli 
↵a=rtpmap:102 H264/90000
↵a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
↵a=rtcp-fb:102 goog-remb 
↵a=rtcp-fb:102 ccm 
↵a=rtcp-fb:102 nack 
↵a=rtcp-fb:102 nack pli 
↵a=rtpmap:127 H264/90000
↵a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
↵a=rtcp-fb:127 goog-remb 
↵a=rtcp-fb:127 ccm 
↵a=rtcp-fb:127 nack 
↵a=rtcp-fb:127 nack pli 
↵a=rtpmap:125 H264/90000
↵a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtcp-fb:125 goog-remb 
↵a=rtcp-fb:125 ccm 
↵a=rtcp-fb:125 nack 
↵a=rtcp-fb:125 nack pli 
↵a=rtpmap:108 H264/90000
↵a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
↵a=rtcp-fb:108 goog-remb 
↵a=rtcp-fb:108 ccm 
↵a=rtcp-fb:108 nack 
↵a=rtcp-fb:108 nack pli 
↵a=rtpmap:124 H264/90000
↵a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
↵a=rtcp-fb:124 goog-remb 
↵a=rtcp-fb:124 ccm 
↵a=rtcp-fb:124 nack 
↵a=rtcp-fb:124 nack pli 
↵a=rtpmap:123 H264/90000
↵a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
↵a=rtcp-fb:123 goog-remb 
↵a=rtcp-fb:123 ccm 
↵a=rtcp-fb:123 nack 
↵a=rtcp-fb:123 nack pli 
↵a=ssrc:1812248270 cname:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:1812248270 msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=ssrc:1812248270 mslabel:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:1812248270 label:43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates
↵"

ICE restarts

pion/ice supports restarts. we should too!

Panic when sub to invalid pub

It seems there is a scenario where a router exists for a pub but the transport is undefined. It throws the following error:

panic: interface conversion: transport.Transport is nil, not *transport.WebRTCTransport

goroutine 525 [running]:
github.com/pion/ion-sfu/pkg/node.(*server).subscribe(0x1ff7cb8, 0xc0003ba200, 0x19, 0xc000286020, 0x21727d0, 0x0, 0xc000235400, 0x10)
	/Users/tarrence/go/src/github.com/pion/ion-sfu/pkg/node/subscribe.go:71 +0x10f9
github.com/pion/ion-sfu/pkg/node.(*server).Subscribe(0x1ff7cb8, 0x1a47f00, 0xc0002c0170, 0x1a36240, 0x1ff7cb8)
	/Users/tarrence/go/src/github.com/pion/ion-sfu/pkg/node/internal.go:155 +0x114
github.com/pion/ion-sfu/pkg/proto._SFU_Subscribe_Handler(0x182f360, 0x1ff7cb8, 0x1a44d20, 0xc00015c9c0, 0x1ff7cb8, 0xc000152a00)
	/Users/tarrence/go/src/github.com/pion/ion-sfu/pkg/proto/sfu.pb.go:736 +0xad
google.golang.org/grpc.(*Server).processStreamingRPC(0xc0002bc180, 0x1a489e0, 0xc0008d5080, 0xc000152a00, 0xc00022df20, 0x1fbec20, 0x0, 0x0, 0x0)
	/Users/tarrence/go/pkg/mod/google.golang.org/[email protected]/server.go:1237 +0xcd1
google.golang.org/grpc.(*Server).handleStream(0xc0002bc180, 0x1a489e0, 0xc0008d5080, 0xc000152a00, 0x0)
	/Users/tarrence/go/pkg/mod/google.golang.org/[email protected]/server.go:1317 +0xcd6
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00028c020, 0xc0002bc180, 0x1a489e0, 0xc0008d5080, 0xc000152a00)
	/Users/tarrence/go/pkg/mod/google.golang.org/[email protected]/server.go:722 +0xa1
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/Users/tarrence/go/pkg/mod/google.golang.org/[email protected]/server.go:720 +0xa1
exit status 2

fix examples

pub-from-browser, pub-from-disk, and sub-to-browser gRPC examples need to be updated to the new interface

Single PC per client<->sfu

Instead of having a peer connection per pub/sub, we should have a single pc per client<->sfu and send all tracks over it. that should reduce complexity and port usage

feat(relay): add example

ion-sfu supports sfu <> sfu relay. we should add an example.

we can add a gRPC endpoint that can be called to connect to a remote sfu + session which will automatically start relaying the tracks. ion-avp does something similar already.

Support full WebRTC SettingEngine configuration

The peer connection SettingEngine supports a bunch of parameters that we should allow to be easily configured in the sfu configuration. You can find all the parameters here: https://github.com/pion/webrtc/blob/master/settingengine.go#L18

move renegotiation to data channel

we could do renegotiation in a data channel. then the only out-of-band signaling we would need is the initial exchange

[fix] Remove duplication of addEventListener

Summary

In echotest example's index.html, The lines from 172 to 191 seem to be exactly the same with the lines from 206 to 227.
As they are in the same scope, I guess the same handler is being assigned twice.

Motivation

Tried testing after removing the lines from 172 to 191, and it works fine.

Describe alternatives you've considered

Remove the lines from 172 to 191.

Additional context

If this duplication was intended, sorry for bothering.

Repository: ReadMe of the examples leads to 404

I wanted to checkout the example code on the repository, however every link on the https://github.com/pion/ion-sfu/tree/master/examples result in 404s.

After a bit of searching I found that these probably have been moved to https://github.com/pion/ion-examples/tree/master/ion-sfu.

I think it would be great if you could update the links or just redirect from the ReadMe to the other repository.

Thanks in advance.

We should simplify how we are closing senders. It should be triggered by the `p.pc.RemoveTrack` call, rather than the other way around.

We should simplify how we are closing senders. It should be triggered by the p.pc.RemoveTrack call, rather than the other way around.

According to this comit from Pion/Webrtc we should close the sender when a io.EOF it's recieved from the webrtc.Track#Read(), Currently now we are closing senders only on peer disconnecting /failing

Originally posted by @OrlandoCo in #176 (comment)

[feat]Stats api

Needs a api for expose stats info

subscribe as needed

There is a scene like this:
With 50 people in one room, it is impossible to subscribe to all of them on the mobile.
modile can only decode several streams.

echotest simulcast switch video not work in safari/firefox

always 1280*720 when click switch video ratio

quic transport

support for adding quic transport to a session. can be used for sfu<>sfu relay and sfu<>avp

improve godocs

comments for godocs could be improved basically across the board

Tracelog can get more categories.

When enable trace, the rtp trace logs are too big. So

Make more options to control what logs can be dump.

improve rtcp test coverage

we need testing for rtcp feedback (remb, nack, ect.) and jitter buffer + error cases generally

JitterBuffer throws

Not sure the cause but posting for posterity

2020-07-10 02:22:03.063 INF Router.delSub id=ckcfl9ki0001r01626wgm3o3e
2020-07-10 02:22:03.063 INF webrtc ice closed for mid: ckcfl9ki0001r01626wgm3o3e
2020-07-10 02:22:03.063 INF JitterBuffer.AddBuffer ssrc=1061942830
2020-07-10 02:22:03.063 INF NewBuffer BufferOptions={false 1000}
panic: assignment to entry in nil map

goroutine 2757 [running]:
github.com/pion/ion-sfu/pkg/rtc/plugins.(*JitterBuffer).AddBuffer(0xc000d30690, 0x3f4bf62e, 0xc03f4bf62e)
	/go/src/github.com/pion/ion-sfu/pkg/rtc/plugins/jitterbuffer.go:111 +0xc9
github.com/pion/ion-sfu/pkg/rtc/plugins.(*JitterBuffer).WriteRTP(0xc000d30690, 0xc000851950, 0x0, 0x0)
	/go/src/github.com/pion/ion-sfu/pkg/rtc/plugins/jitterbuffer.go:135 +0x123
github.com/pion/ion-sfu/pkg/rtc/plugins.(*JitterBuffer).AttachPub.func1(0xc000d30690)
	/go/src/github.com/pion/ion-sfu/pkg/rtc/plugins/jitterbuffer.go:94 +0xad
created by github.com/pion/ion-sfu/pkg/rtc/plugins.(*JitterBuffer).AttachPub
	/go/src/github.com/pion/ion-sfu/pkg/rtc/plugins/jitterbuffer.go:83 +0x5a

[json-rpc] support trickle

Right now json-rpc only supports publish and subscribe. We should add support for trickle ice.

feat(lifecycle): use context

i think we could clean up a lot of the stop logic by rather using contexts.

each webrtctransport should have a context that controls the lifecycle of all its components

pub/sub streamer

hi,
I would like to contribute with my suggestion:
I tested many SFU solutions: mediasoup2 and 3, janus, licode, red5pro, jitsi, kurento and they are appear to me too complex compared to a simple old RTMP stream:
You publish a stream streamid and you play the stream streamid
In fact all done with 2 functions

• publishStream(videoDomElement, streamid);
• playStream(videoDomElement, streamid);

May I suggest to use that paradigm for end user developer ?
(hiding all webrtc protocol exchanging)

That would be really simple selective forward unit: I publish streamid and I decide what streamid I want to play

Regards

iOS/Apple, Surface Pro - not able to receive the stream (the root cause provided)

Your environment.

• Browser: Safari 13.1.2 with Chrome on Windows
• Other Information - see below

What did you do?

1. Opened Chrome on Windows and started the stream.
2. Opened Safari or Chrome on macOS and couldn't receive the stream.

What did you expect?

See the stream.

What happened?

I couldn't see it.

After ionorg/ion-sdk-js#28 fix, I was able to experiment a bit more with Safari.

It turns out, that in the SDP Offer Safari has:

(...)
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 125 104
(...)
a=rtpmap:96 H264/90000
(...)

but in the SDP answer we have:
m=video 9 UDP/TLS/RTP/SAVPF 96
(...)
a=rtpmap:96 VP8/90000

For Safari, to get VP8/90000, the SAVPF should be set to 100.

96 works flawlessly with Chrome (on other OS than Mac/iOS), as with Chrome VP8/90000, indeed, it is 96.

To fix that, for maximum compatibility in the future, we probably need to look-up VP8 in the SDP Offer and return the same value to the browser.

Potential solution/fix/help here: pion/webrtc#716 (comment)

support user define track label

Summary

Hope that ion-sfu supports the user-defined label or ID of the track.

I'm new to webRTC, and I am currently using ion-sfu to develop a chat app. I find that for each track the front-end ontracked, it is hard to find which user it belongs to. I need it to support the adjustment to the user's volume. And it may be solved if I can set the label of the track as user id.

I tried to use the mapping from ssrc to label to distinguish the tracks. But the label and ID I tried to print out on backend is different from the track information I received in front-end.

If there is any other solution to this issue, please tell me as well, thanks!

Thanks for reading.

Room Types

Right now we only support a fully connected room where all peers are subscribed to all others by default. We should probably support other types, like broadcast, where only a single peer pubs. What other types type should we support?

panic: send on closed channel

panic: send on closed channel

goroutine 15413 [running]:
github.com/pion/ion-sfu/pkg.(*WebRTCSender).receiveRTCP(0xc002572e40)
	/go/src/github.com/pion/ion-sfu/pkg/sender.go:151 +0x18a
created by github.com/pion/ion-sfu/pkg.NewWebRTCSender
	/go/src/github.com/pion/ion-sfu/pkg/sender.go:65 +0x1c8

simulcast support

integrate pion simulcast

• Update remb loop to add bw state to sender
• sscrc/rid/quality map exposed to sender
• Router forwards all simulcast packets to sender (may be no change)
• Sender inspects simulcast packet and forwards packets for stream best matching current sender bw
• Make sure client is sending abs-time extensions. (required for correct remb)
• Configure abs-time extension in offers

can this SFU be self-hosted?

can this SFU media server be self-hosted for own purpose? bcs I dont want to depend on ion service and i want to have full control on server only by me and not on ion/pion company, as this is an open-source project..can i self host it?

Main ReadMe: Question about port for JSONrpc docker

The current readme says one should use

docker run -p 50051:50051 -p 5000-5020:5000-5020/udp pionwebrtc/ion-sfu:latest-jsonrpc

However when I start this docker container it says (on the command line) that it's listening on port 7000? Shouldn't the exposed port be 7000 then?

E.g.

docker run -p 7000:7000 -p 5000-5020:5000-5020/udp pionwebrtc/ion-sfu:latest-jsonrpc
           ^^^^^^^^^^^^

feat(metrics): expose metrics endpoints

for sfu monitoring, we would like to expose metrics endpoints. we can expose these through go apis initially then through gRPC for consumption by prometheus later.

some initial metrics we should support:

• number of sessions
• number of peers
• number of tracks
• peers per session
• tracks per peer
• ice disconnects/failures/closes
• peer bandwidth metrics
• nack/jitter buffer metrics (drops/misses/ect)
• error counts

let me know if you have any other ideas/suggestions

@steebchen @leewardbound @decanus

Add / Remove Tracks

Summary

A feature to be able to add and remove sending tracks for specific peers.

Motivation

One of the features I require for my use case, is that the ion-sfu server can remove sending tracks for specific users. I need this because an admin user in my application should be able to mute and unmute other members at will, which is currently not possible.

I am opening an issue as requested by @OrlandoCo. CC: @tarrencev whom I have discussed this issue with extensively.

Additional context

• #196

[feat]Support rtx

Chrome will send 2 ssrc for video for rtx. But seems ion-sfu only uses the first ssrc though sdp exchange. Does ion-sfu should support this feature?

Settings IP for sdp candidate

If you use Docker, the IP of the container is added to the sdp and clients located on the host's local network cannot connect. Can you add to the settings so that you can specify which other IP addresses to add to the sdp? otherwise it turns out that the container will only work for the local host

IsVideo shouldn't use payload types

Right now the IsVideo function is determined by payload type. It should use the transport media engine instead (since payload type is dynamic)

https://github.com/pion/ion-sfu/blob/7a2e1a845b1ff786856f9e5a36f1f3f63e5f0511/pkg/rtc/transport/webrtctransport.go#L266

json.Unmarshall error while using sdp from browser for pub-from-browser example

Your environment.

• Version: 0dd6b83
• OS: Arch Linux
• Browser: Google Chrome Version 81.0.4044.129 (Official Build) (64-bit)

What did you do?

I was following the examples code for pub-from-browser

I copied the base64 sdp from from top text are of jsfiddle in expample README.md

Pasted sdp to sdp.txt
then,

cat sdp.txt | go run pub_from_browser/main.go

What did you expect?

I expected to get an another sdp to paste in jsfiddle and start the session

What happened?

Then I get following error traceback:

[wrocket@wrocket-pc pub-from-browser]$ cat sdp.txt | go run main.go

panic: illegal base64 data at input byte 1

goroutine 1 [running]:
github.com/pion/ion-sfu/examples/internal/signal.Decode(0xc000016a00, 0x2124, 0x94e3a0, 0xc000074af0)
	/home/wrocket/git/ion-sfu/examples/internal/signal/signal.go:74 +0xbf
main.main()
	/home/wrocket/git/ion-sfu/examples/pub-from-browser/main.go:30 +0x20d
exit status 2

I found the sdp value in json is string and Sdp in SDPDescription struct in proto3 is []byte.
And because of it we are having issue in json.Unmarshall

video stream break-off

What did you do?

An html interface with ion-sfu and ion-sdk-js had been built which broadcasts a local video stream.
Mostly the default configurations used.

What happened?

The setup for broadcasting was supposed to work fine, since I took steps for normal installation on the main page in each case ( for ion-sfu and for ion-sdk-js respectively).
However the remote video streaming stops abruptly after a while (like in one and half a minute).
What can interrupt the normal flow of a stream?
Should I use a kind of refreshing mechanism to keep the negotiation between the client and the server (ion-sfu) somehow alive?

Here is the log dump of the go build ./cmd/signal/json-rpc/main.go && ./main -c config.toml

[2020-12-09 08:30:23.059] [DEBUG] [57][publisher.go][func1] => Peer ckihg8biw001gwz27lwcjas8m got remote track id: 768930f0-8f70-475e-b025-19b3c870d883 mediaSSRC: 93922767 rid :h streamID: 31bc804d-b160-4375-beb5-5325e1a54fca             
[2020-12-09 08:30:23.059] [DEBUG] [134][interceptor.go][newBuffer] => Setting feedback goog-remb                                                                                                                                              
[2020-12-09 08:30:23.059] [DEBUG] [134][interceptor.go][newBuffer] => Setting feedback transport-cc                                                                                                                                           
[2020-12-09 08:30:23.059] [DEBUG] [134][interceptor.go][newBuffer] => Setting feedback nack                                                                                                                                                   
[2020-12-09 08:30:23.059] [DEBUG] [134][interceptor.go][newBuffer] => Setting feedback nack                                                                                                                                                   
[2020-12-09 08:30:23.059] [DEBUG] [134][interceptor.go][newBuffer] => NewBuffer BufferOptions={1000 0}                                                                                                                                        
[2020-12-09 08:31:04.876] [TRACE] [202][router.go][loopDownTrackRTCP] => sender got nack: TransportLayerNack from 1                                                                                                                           
        Media Ssrc 3734484e                                                                                                                                                                                                                   
        ID      LostPackets                                                                                                                                                                                                                   
        35765   1                                                                                                                                                                                                                             
                                                                                                                                                                                                                                              
[2020-12-09 08:31:04.911] [TRACE] [202][router.go][loopDownTrackRTCP] => sender got nack: TransportLayerNack from 1                                                                                                                           
        Media Ssrc 3734484e                                                                                                                                                                                                                   
        ID      LostPackets                                                                                                                                                                                                                   
        35768   0                                                                                                                                                                                                                             
                                                                                                                                                                                                                                              
[2020-12-09 08:31:19.830] [TRACE] [202][router.go][loopDownTrackRTCP] => sender got nack: TransportLayerNack from 1
  Media Ssrc 3734484e
        ID      LostPackets
        37284   1

[2020-12-09 08:31:34.060] [INFO] [219][peer.go][Close] => RemovePeer ckihg8biw001gwz27lwcjas8m from session test session
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 1344776136 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 2577000426 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 93922767 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [232][receiver.go][readRTCP] => receiver ckihg8biw001gwz27lwcjas8m readrtcp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 3630805290 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 1871866969 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [232][receiver.go][readRTCP] => receiver ckihg8biw001gwz27lwcjas8m readrtcp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 2458631488 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [72][publisher.go][func3] => ice connection state: closed
[2020-12-09 08:31:34.060] [DEBUG] [77][publisher.go][func3] => webrtc ice closed for peer: ckihg8biw001gwz27lwcjas8m
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 1344776136 read rtp eof                                                                                                                                     [0/2381]
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 2577000426 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 93922767 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [232][receiver.go][readRTCP] => receiver ckihg8biw001gwz27lwcjas8m readrtcp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 3630805290 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 1871866969 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [232][receiver.go][readRTCP] => receiver ckihg8biw001gwz27lwcjas8m readrtcp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 2458631488 read rtp eof
[2020-12-09 08:31:34.060] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.060] [DEBUG] [72][publisher.go][func3] => ice connection state: closed
[2020-12-09 08:31:34.060] [DEBUG] [77][publisher.go][func3] => webrtc ice closed for peer: ckihg8biw001gwz27lwcjas8m
[2020-12-09 08:31:34.060] [DEBUG] [209][receiver.go][readRTP] => receiver 3214294019 read rtp eof
[2020-12-09 08:31:34.061] [DEBUG] [66][once.go][doSlow] => Closing sender ckihg8q6o001hwz27zx4ai2g5
[2020-12-09 08:31:34.061] [DEBUG] [166][router.go][loopDownTrackRTCP] => Sender ckihg8q6o001hwz27zx4ai2g5 closed due to: EOF
[2020-12-09 08:31:34.311] [DEBUG] [103][peer.go][func1] => peer ckihg8q6o001hwz27zx4ai2g5 negotiation needed
[2020-12-09 08:31:34.311] [INFO] [112][peer.go][func1] => peer ckihg8q6o001hwz27zx4ai2g5 send offer
[2020-12-09 08:31:34.496] [INFO] [182][peer.go][SetRemoteDescription] => peer ckihg8q6o001hwz27zx4ai2g5 got answer
[2020-12-09 08:33:14.384] [INFO] [219][peer.go][Close] => RemovePeer ckihg8q6o001hwz27zx4ai2g5 from session test session
[2020-12-09 08:33:14.384] [DEBUG] [59][subscriber.go][func1] => ice connection state: closed
[2020-12-09 08:33:14.384] [DEBUG] [66][once.go][doSlow] => webrtc ice closed for peer: ckihg8q6o001hwz27zx4ai2g5

MediaStreamTrack.enabled = false, NACKS

Your environment.

• Version: LATEST
• Browser: LATEST

What did you do?

When video track is disabled, the SFU starts forwarding NACKS continuously.

What did you expect?

No NACKS messages.

What happened?

Maybe it's possible to add a parameter to let know pion that a track it's going to be disabled, or limit rate the NACKS

echotest remote audio mute/unmute

browser: chrome
can hear the audio when click mute
can't hear the audio when click unmute

index.html:
seems the two radio use the same controlRemoteAudio func

SetLocalDescription error: InvalidModificationError: new sdp does not match previous offer

Your environment.

• Version: Latest, Including the pion/WebRTC with renegotiation support
• Browser: Chrome 84.0.4147.135

What did you do?

Run SFU server, when the first person joins, it's ok, but when a second persons join, a renegotiation is fired on the first person transport, and when creating the offer on server and setting it, it gives the following error:

InvalidModificationError: new sdp does not match previous offer

Since the renegotiation failed the first person can't see the second.

What did you expect?

No errors

no plan b fallback as of #84

Your environment.

• Version: 28bb637
• Browser: iOS / XCode / Flutter

What did you do?

Tried using ion-sfu (standalone) with flutter_webrtc (0.2.8 and 0.3.3)

What did you expect?

A successful connection (since https://github.com/pion/ion works just fine with https://github.com/pion/ion-sdk-flutter which uses flutter_webrtc)

What happened?

offer SDP semantics does not match configuration

Reason: #84 disabled the plan b fallback https://github.com/pion/ion-sfu/pull/84/files#diff-dd4ded5678806fb0c41b7ef5413f0dfdL40 and flutter_webrtc seems to use plan b currently

Was this a desired change?

(@tarrencev describes that commit with "update confs", so not entirely clear to me whether there was intent to drop the plan b fallback or not)

(There is a recent WiP PR that might solve this scenario by moving flutter_webrtc to unified plan flutter-webrtc/flutter-webrtc#359 - but the problem scope isn't necessarily constrained to that... or is plan-b dead and long gone everywhere except in flutter_webrtc?)

remove logging `fix` from config

I dont think we need the fix parameter here to be configurable:

https://github.com/pion/ion-sfu/blob/b166816adeaef49cdf4cd719458b57163146d46a/config.toml#L36

Since it is known statically, so it should just be hard coded. Unless I am missing something? When would someone want to configure this?

TWCC support

• Recv-Side TWCC

[fix] Remove unused config variable

Summary

The variable config was assigned, but being used.
In this example, as signaling process is being done via ion-sfu server, the config seems to serve no purpose.

Motivation

As config doesn't play a role in this use case, it can confuse the readers of this example.
Removing it can clarify the role of ion-sfu.

Describe alternatives you've considered

Remove the config assignment part.

Additional context

By the way, thank you for writing such an on-point and readable example.
This one helped me a lot.

Examples use track.onremovetrack but should use stream.onremovetrack

Your environment.

• Version: macOS 10.15.7
• Browser: Firefox 84.0
• Other Information - fixed by changing to stream.onremovetrack and checking event for track kind before removing the video element (because stream.onremovetrack is called twice)

What did you do?

Tested with pubusbtest, open two browser tabs, start publishing on both, close a browser tab and the remotes video element belonging to the closed browser tab is frozen and not removed.

What did you expect?

The video element should be removed correctly when the other party leaves the page or closes the browser tab.

What happened?

The video element remains on the page and shows a frozen picture.

Cut video track sdp as need

Summary

The sfu sdp of each video track has dozens of lines;
It will waste a lot of traffic when renegotiation(for example: a 50 person room)
Most 'a=' lines are useless for video track because one video track only need one codec
It's hard to debug/metric/log for sfu

Motivation

• SFU choose these video track codec and params(rtpmap fmtp rtcp-fb etc..)
• Cut the useless sdp lines make debug/metric/log easy and save traffic when renegotiation

Panic on transport close

2020-07-16 21:17:09.122 INF WebRTCTransport.Close t.ID()=ckcpan12w005c0122fiq0ctfc
2020-07-16 21:17:09.122 INF Router.Close
2020-07-16 21:17:09.122 INF delRouter id=ckcpan12w005c0122fiq0ctfc
2020-07-16 21:17:09.122 ERR JitterBuffer.pliLoop j.Pub.WriteRTCP err=io: read/write on closed pipe
2020-07-16 21:17:09.464 ERR JitterBuffer.rembLoop j.Pub.WriteRTCP err=io: read/write on closed pipe
2020-07-16 21:17:09.757 ERR JitterBuffer.pliLoop j.Pub.WriteRTCP err=io: read/write on closed pipe
2020-07-16 21:17:10.058 INF webrtc ice closed for mid: ckcpan12w005c0122fiq0ctfc
2020-07-16 21:17:10.058 INF WebRTCTransport.Close t.ID()=ckcpankdc005j0122gxd47kfh
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x847350]

goroutine 9155 [running]:
github.com/pion/ion-sfu/pkg/rtc/transport.(*WebRTCTransport).Close(0xc000a50a20)
	/go/src/github.com/pion/ion-sfu/pkg/rtc/transport/webrtctransport.go:146 +0xc0
github.com/pion/ion-sfu/pkg/node.Subscribe.func1(0x6)
	/go/src/github.com/pion/ion-sfu/pkg/node/subscribe.go:53 +0xb6
created by github.com/pion/webrtc/v2.(*PeerConnection).onICEConnectionStateChange
	/go/pkg/mod/github.com/pion/webrtc/[email protected]/peerconnection.go:301 +0x12a

flakey test: TestPeerRemovesRouterWhenRemoteRemovesTrack