Failed to set local answer sdp error about ion-sfu

[tarrencev]

This occurred when I had on existing participant (A) and then joined with a second (B). The A did not receive B video and the error provided was thrown. Due to this error, A never responded to the offer and the sfu pc was in a state where it no longer called OnNegotiationNeeded when tracks were added/removed. Everything worked fine for B.

[OrlandoCo]

I have never seen this before, can this be related with recent change of caching the transcievers?

[tarrencev]

It is possible. I guess that would mean there was a condition previously where the transceiver was being updated during offer creation to contain the parameters necessary to populate the m= section? Should we assert that an mid= should always have a corresponding m=?

Maybe this check needs to be expanded: https://github.com/pion/webrtc/blob/master/peerconnection.go#L544

Maybe this is related? https://mediasoup.discourse.group/t/create-server-side-consumers-with-paused-true/244

If i understand correctly, the sfu might be writing RTP to the client before the pc is properly negotiated?

[tarrencev]

It seems i am able to reproduce this locally by adding a time.Sleep(time.Second * 5) before sending back the offer in OnNegotiationNeeded

peer.OnNegotiationNeeded(func() {
	log.Debugf("on negotiation needed called for pc %s", peer.ID())
	offer, err := peer.CreateOffer()
	if err != nil {
		log.Errorf("CreateOffer error: %v", err)
		return
	}

	err = peer.SetLocalDescription(offer)
	if err != nil {
		log.Errorf("SetLocalDescription error: %v", err)
		return
	}

	time.Sleep(5 * time.Second)

	err = stream.Send(&pb.SignalReply{
		Payload: &pb.SignalReply_Negotiate{
			Negotiate: &pb.SessionDescription{
				Type: offer.Type.String(),
				Sdp:  []byte(offer.SDP),
			},
		},
	})

	if err != nil {
		log.Errorf("negotiation error %s", err)
	}
})

So it does seem it might be cause by webrtc rtp arriving before the peer connection is properly negotiated

[OrlandoCo]

For the Media Line to be populated the track needs to exist and have a valid codec, so I was thinking of checking that MediaDescriptions were equal to trancievers for Unified plan, but the weird thing is that media attributes are filled but the media name not.

We could loop in the check for all the MediaDescriptions and check that MediaName,Media is not empty. But we need to figure how the MediaName is empty but the attributes doesn't.

[OrlandoCo]

It seems i am able to reproduce this locally by adding a time.Sleep(time.Second * 5) before sending back the offer in OnNegotiationNeeded

peer.OnNegotiationNeeded(func() {
	log.Debugf("on negotiation needed called for pc %s", peer.ID())
.....

So it does seem it might be cause by webrtc rtp arriving before the peer connection is properly negotiated

So the error is in the peer answer?

[tarrencev]

@OrlandoCo I just checked the SDP again and it does contain the correct number of m= lines. So I think it is the issue where track RTP arrives before negotiation is complete. Basically, the SFU sends an offer and RTP from the tracks in the new offer arrive before it has been negotiated. It seems there is some weird behavior in how chrome handles the unassociated track rtp that leads to the error when setting the SFU offer as the remote description

[OrlandoCo]

@tarrencev Is that is the case we can make a Flag in new senders to not accept packets until answer has been recieved

[OrlandoCo]

But I can't figure how this problem relates with the missing M-LINE

[tarrencev]

I was incorrect in parsing it the first time. I dont think we are missing an m= line:

↵o=- 3223776293848369481 1599851888 IN IP4 0.0.0.0
↵s=-
↵t=0 0
↵a=fingerprint:sha-256 D2:D7:C5:59:89:44:02:85:F3:55:DA:56:36:D9:8E:2A:36:2B:E3:8E:27:39:D5:CD:6C:FE:9C:A9:4F:7B:B8:00
↵a=group:BUNDLE 0 1 2 3


↵m=audio 9 UDP/TLS/RTP/SAVPF 111
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:0
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:111 opus/48000/2
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=ssrc:875501110 cname:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:875501110 msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f 676e158d-2fef-4df3-9680-db939878d284
↵a=ssrc:875501110 mslabel:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:875501110 label:676e158d-2fef-4df3-9680-db939878d284
↵a=msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f 676e158d-2fef-4df3-9680-db939878d284
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates


↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 125 108 124 123
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:1
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 goog-remb 
↵a=rtcp-fb:96 ccm 
↵a=rtcp-fb:96 nack 
↵a=rtcp-fb:96 nack pli 
↵a=rtpmap:98 VP9/90000
↵a=fmtp:98 profile-id=0
↵a=rtcp-fb:98 goog-remb 
↵a=rtcp-fb:98 ccm 
↵a=rtcp-fb:98 nack 
↵a=rtcp-fb:98 nack pli 
↵a=rtpmap:100 VP9/90000
↵a=fmtp:100 profile-id=2
↵a=rtcp-fb:100 goog-remb 
↵a=rtcp-fb:100 ccm 
↵a=rtcp-fb:100 nack 
↵a=rtcp-fb:100 nack pli 
↵a=rtpmap:102 H264/90000
↵a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
↵a=rtcp-fb:102 goog-remb 
↵a=rtcp-fb:102 ccm 
↵a=rtcp-fb:102 nack 
↵a=rtcp-fb:102 nack pli 
↵a=rtpmap:127 H264/90000
↵a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
↵a=rtcp-fb:127 goog-remb 
↵a=rtcp-fb:127 ccm 
↵a=rtcp-fb:127 nack 
↵a=rtcp-fb:127 nack pli 
↵a=rtpmap:125 H264/90000
↵a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtcp-fb:125 goog-remb 
↵a=rtcp-fb:125 ccm 
↵a=rtcp-fb:125 nack 
↵a=rtcp-fb:125 nack pli 
↵a=rtpmap:108 H264/90000
↵a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
↵a=rtcp-fb:108 goog-remb 
↵a=rtcp-fb:108 ccm 
↵a=rtcp-fb:108 nack 
↵a=rtcp-fb:108 nack pli 
↵a=rtpmap:124 H264/90000
↵a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
↵a=rtcp-fb:124 goog-remb 
↵a=rtcp-fb:124 ccm 
↵a=rtcp-fb:124 nack 
↵a=rtcp-fb:124 nack pli 
↵a=rtpmap:123 H264/90000
↵a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
↵a=rtcp-fb:123 goog-remb 
↵a=rtcp-fb:123 ccm 
↵a=rtcp-fb:123 nack 
↵a=rtcp-fb:123 nack pli 
↵a=ssrc:1701979011 cname:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:1701979011 msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f b51abe50-316f-4fc2-a6de-504846451f95
↵a=ssrc:1701979011 mslabel:7fa50d2e-7729-47b6-a31d-de54719e9f3f
↵a=ssrc:1701979011 label:b51abe50-316f-4fc2-a6de-504846451f95
↵a=msid:7fa50d2e-7729-47b6-a31d-de54719e9f3f b51abe50-316f-4fc2-a6de-504846451f95
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates


↵m=audio 9 UDP/TLS/RTP/SAVPF 111
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:2
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:111 opus/48000/2
↵a=fmtp:111 minptime=10;useinbandfec=1
↵a=ssrc:3372783641 cname:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:3372783641 msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=ssrc:3372783641 mslabel:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:3372783641 label:5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 5976ef51-9dd6-4390-b37b-51fe5489dbf2
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates


↵m=video 9 UDP/TLS/RTP/SAVPF 96 98 100 102 127 125 108 124 123
↵c=IN IP4 0.0.0.0
↵a=setup:actpass
↵a=mid:3
↵a=ice-ufrag:iBWdqHzmzzaWpvEE
↵a=ice-pwd:vOeolYdeniZMEEHLAzFbRlUffkGjytxX
↵a=rtcp-mux
↵a=rtcp-rsize
↵a=rtpmap:96 VP8/90000
↵a=rtcp-fb:96 goog-remb 
↵a=rtcp-fb:96 ccm 
↵a=rtcp-fb:96 nack 
↵a=rtcp-fb:96 nack pli 
↵a=rtpmap:98 VP9/90000
↵a=fmtp:98 profile-id=0
↵a=rtcp-fb:98 goog-remb 
↵a=rtcp-fb:98 ccm 
↵a=rtcp-fb:98 nack 
↵a=rtcp-fb:98 nack pli 
↵a=rtpmap:100 VP9/90000
↵a=fmtp:100 profile-id=2
↵a=rtcp-fb:100 goog-remb 
↵a=rtcp-fb:100 ccm 
↵a=rtcp-fb:100 nack 
↵a=rtcp-fb:100 nack pli 
↵a=rtpmap:102 H264/90000
↵a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
↵a=rtcp-fb:102 goog-remb 
↵a=rtcp-fb:102 ccm 
↵a=rtcp-fb:102 nack 
↵a=rtcp-fb:102 nack pli 
↵a=rtpmap:127 H264/90000
↵a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
↵a=rtcp-fb:127 goog-remb 
↵a=rtcp-fb:127 ccm 
↵a=rtcp-fb:127 nack 
↵a=rtcp-fb:127 nack pli 
↵a=rtpmap:125 H264/90000
↵a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
↵a=rtcp-fb:125 goog-remb 
↵a=rtcp-fb:125 ccm 
↵a=rtcp-fb:125 nack 
↵a=rtcp-fb:125 nack pli 
↵a=rtpmap:108 H264/90000
↵a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
↵a=rtcp-fb:108 goog-remb 
↵a=rtcp-fb:108 ccm 
↵a=rtcp-fb:108 nack 
↵a=rtcp-fb:108 nack pli 
↵a=rtpmap:124 H264/90000
↵a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
↵a=rtcp-fb:124 goog-remb 
↵a=rtcp-fb:124 ccm 
↵a=rtcp-fb:124 nack 
↵a=rtcp-fb:124 nack pli 
↵a=rtpmap:123 H264/90000
↵a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
↵a=rtcp-fb:123 goog-remb 
↵a=rtcp-fb:123 ccm 
↵a=rtcp-fb:123 nack 
↵a=rtcp-fb:123 nack pli 
↵a=ssrc:1812248270 cname:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:1812248270 msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=ssrc:1812248270 mslabel:c8f68a95-70be-49ec-875e-e2aa0c95c52e
↵a=ssrc:1812248270 label:43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=msid:c8f68a95-70be-49ec-875e-e2aa0c95c52e 43855bb5-8157-464d-8dc3-5cdf754802f6
↵a=sendrecv
↵a=candidate:foundation 1 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 2 udp 2130706431 172.24.0.3 5160 typ host generation 0
↵a=candidate:foundation 1 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=candidate:foundation 2 udp 1694498815 34.105.122.11 5043 typ srflx raddr 0.0.0.0 rport 5043 generation 0
↵a=end-of-candidates

[tarrencev]

@OrlandoCo i wonder if we can wait for the first rtcp packet to start sending rtp. Does a peer send PLI on connect?

[OrlandoCo]

@OrlandoCo i wonder if we can wait for the first rtcp packet to start sending rtp

Does a subscriber would send some kind packet like PLI or something if we dont send packets first?

[tarrencev]

@OrlandoCo I think for video it would work but i guess not for audio. So going to track using the sdp

[OrlandoCo]

This it's the bug report for chrome for this problem, it's accepted and not solved yet Chrome bug. It seems that this problem it's only for chromium browsers, I will test with firefox.

[tarrencev]

interesting. from 2018 so i guess we shouldn't get our hopes up on it getting solved 😐