Coder Social home page Coder Social logo

opus dtx incorrectly judged about mediasoup HOT 7 CLOSED

buptlsp avatar buptlsp commented on June 11, 2024 1
opus dtx incorrectly judged

from mediasoup.

Comments (7)

ibc avatar ibc commented on June 11, 2024

It makes sense. Definitely the OPUS DTX detection in mediasoup is weak.

Honestly we are terribly busy with other mediasoup tasks and I don't think I'm gonna hace time to read the spec and work on this in the short/medium term. @buptlsp is this something you could contribute with a PR and hopefully C++ side tests? Note that we have tests in mediasoup/worker/test/RTC/Codecs folder for video codecs but not for OPUS.

from mediasoup.

ibc avatar ibc commented on June 11, 2024

NOTE: I've been told that in libwebrtc the DTX packet detection is following:

bool IsDtxPacket() const override { return payload_.size() <= 2; }

Would this be ok?

from mediasoup.

ibc avatar ibc commented on June 11, 2024

PR done: #1357

from mediasoup.

buptlsp avatar buptlsp commented on June 11, 2024

as rfc6716 said, there may be 4 types, the length of dtx packet should be:

  • code 0 : one byte toc with no data. payload length = 1.
  • code 1 : one byte toc with no data( 2 * 0 cbr data). payload length = 1.
  • code 2: one byte toc with 1 byte length. payload length = 2.
  • code 3: one byte toc with 1 byte frame count byte. if CBR payload length = 2. if VBR, the payload length should be 2 + M-1 (M is the frame count). but in this case VBR is waste byte, opus codec may just use CBR (I am not sure).

so, I think payload_.size() <= 2 may be right. this judgement may have this problem:

  • code 3 VBR problem as I said.
  • if the ptime is very small, for example, 2.5ms, using code 0, the data may very small to be 1 byte . the total payload length is 1 byte toc + 1 byte data.

The probability of these two situations occurring is too small to be considered.

there is other situation may cause the payload length <=2. the rtp packet may be total padding without data. the payload is not opus packet, so the length may be 2 also. I have discussed this problem with you before. if you have already removed the padding, this problem will not exist.

from mediasoup.

ibc avatar ibc commented on June 11, 2024

there is other situation may cause the payload length <=2. the rtp packet may be total padding without data. the payload is not opus packet, so the length may be 2 also. I have discussed this problem with you before. if you have already removed the padding, this problem will not exist.

This is not a problem since we know the payload size and the padding size separately.

from mediasoup.

ibc avatar ibc commented on June 11, 2024

so, I think payload_.size() <= 2 may be right. this judgement may have this problem:

code 3 VBR problem as I said.
if the ptime is very small, for example, 2.5ms, using code 0, the data may very small to be 1 byte . the total payload length is 1 byte toc + 1 byte data.

So I'm a bit worried here. Can't we know if those cases contain real OPUS data or DTX? If we were an OPUS decoder couldn't we know it by parsing these bytes/bits?

from mediasoup.

buptlsp avatar buptlsp commented on June 11, 2024

if you just use length to judge, It may cause small problem, but it is very easy. the code may by 1 line.
if you check toc of opus, it's can be judged absolutely correct, but a little complicated. the code may be 100 line.

from mediasoup.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.