Implement more RTCP stuff about mediasoup

[ibc]

NACK processed! Habemus RTP retransmission.

[jmillan]

Below are the RTCP types we are supporting.

For Feedback type "ack" or possitive acknowledgment:

• ack rpsi (bypassed)
• ack app (bypassed)

For Feedback type "nack" or negative acknowledgment:

• nack (locally consumed)
• nack pli (bypasssed)
• nack sli (bypassed)
• nack rpsi (bypassed)
• nack app (bypassed)

For Feedback type "ccm" codec control:

• ccm fir (bypassed)

The following, which are shown in the previous SDPs, are not supported:

• goog-remb
• transport-cc (this one looks promising BTW, but until we generate it we better remove the support from the SDP to avoid a ton of Feedback coming from Chrome.

[ibc]

Lof of RTCP stuff done by @jmillan in #41

[ibc]

Among with other tasks, we must also define which exact RTCP feedback parameters mediasoup supports. For example, current browsers signal this:

• Chrome:

m=audio 51753 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126
a=mid:audio
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000

m=video 64460 UDP/TLS/RTP/SAVPF 100 101 107 116 117 96 97 99 98
a=mid:video
a=rtcp-rsize
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtpmap:101 VP9/90000
a=rtcp-fb:101 ccm fir
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 goog-remb
a=rtcp-fb:101 transport-cc
a=rtpmap:107 H264/90000
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f

• Firefox:

m=audio 52876 UDP/TLS/RTP/SAVPF 109 9 0 8
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=mid:sdparta_0
a=rtpmap:109 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000

m=video 53804 UDP/TLS/RTP/SAVPF 120 121 126 97
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:121 max-fs=12288;max-fr=60
a=mid:sdparta_1
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 goog-remb
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-fb:97 goog-remb
a=rtpmap:120 VP8/90000
a=rtpmap:121 VP9/90000
a=rtpmap:126 H264/90000
a=rtpmap:97 H264/90000

[ibc]

Thanks! I will work on properly setting the supported values.

[ibc]

@jmillan I'm pretty sure you don't need it. However FYI "JSEP" draft also defines RTCP routing in Appendix B.

[ibc]

Next actions:

• Improve/implement proper generation timing for Receiver Reports.
• Locally generate Sender Reports.
• Generate NACK for senders.
• Generate PLI if we detect uplink heavy packet loss.
• Improve NACK generation algorithm: #72
• Move the RTP buffer from the RtpStream class to the RtpSender (this is because, once simulcast/SVC is done, we will handle different sequence numbers for each receiver).
• Regardless how many RTP packets we store in the RtpStream instance, don't satisfy NACK requests for packets older than 200ms (thanks @lminiero).
• Add SDES (CNAME) packet on compound packets with only Receiver Reports.
  • This is a no-sense and won't be done.
• Forward TMMBR and TMMBN (they are dropped right now).
  • It won't be done, see #89 and #93.
• Locally generate REMB feedback messages to let remote senders adapt to the estimated bandwidth.
• Implement draft-holmer-rmcat-transport-wide-cc-extensions (RTP Extensions for Transport-wide Congestion Control).
  • NOTE: This will be done after 1.0.0.

[ibc]

Note for the future: (S)AVPF spec mandates support for TMMBR which is a way to limit the sender bandwidth (REMB does also that), so we should support it.

[ibc]

@jmillan one question please:

Let's imagine that mediasoup is gonna generate its own NACK and TMMBR and send them to remote senders. Which value should be placed into the SSRC of packet sender field of those feedback messages? Also, which SSRC of packet sender is currently set in Receiver Reports and Sender Reports generated by mediasoup?

Such a SSRC of packet sender is not signaled in the SDP (it may be signaled in ORTC or let the browser choose).

[jmillan]

On packets generated by RtpReceiver (ie: Receiver Report), we are setting a SSRC of 0. On packets generated by RtpSender (ie: Sender Report), we are setting the SSRC value that is being sent.

This is the way to go IMHO. Instead of hacking an invented SSRC on RTCP generated by receivers just set it to 0. The receiver of such RTCP has no way to check the SSRC anyway.

[ibc]

Nice and clear.

[ibc]

Habemus NACK generation in master branch: 553f4af

[ibc]

Note regarding receipt of feedback messages (in draft-ietf-rmcat-rtp-cc-feedback):

Compound RTCP packets contain a Sender Report (SR) packet and a
Source Description (SDES) packet, and an RTP Congestion Control
Feedback (RC2F) packet [I-D.dt-rmcat-feedback-message]. Non-compound
RTCP packets contain only the RC2F packet.

So we must be ready to parse feedback messages also in compound packets. But maybe this is already done, is it @jmillan? :)

[jmillan]

So we must be ready to parse feedback messages also in compound packets. But maybe this is already done, is it @jmillan? :)

Absolutely!

[ibc]

Cool, so are those two "packets" implemented?

• RTCP XR Block for Reporting Congestion Control Feedback
• RTP/AVPF Transport Layer Feedback for Congestion Control

[jmillan]

No, not yet.

[ibc]

I'm trying to figure out whether those "standard feedback messages" are already being sent by browsers or not... If so, it's up to the sender (which may be the mediasoup RtpSender) to implement any of the existing sender side bitrate estimation algorithms.

[jmillan]

I'm trying to figure out whether those "standard feedback messages" are already being sent by browsers or not.

They are not being sent as far as I see.

[ibc]

So we basically can deal with draft-alvestrand-rmcat-remb (implemented by Chrome and Firefox) and draft-holmer-rmcat-transport-wide-cc-extensions (implemented by Chrome), both of them discontinued specs, right?

[jmillan]

yes 👍

[ibc]

And we are done here :)