Extract individual objects? about libmpegh

[SakethSathuvalli]

All I am saying is when You apply a gain of 2.0 or more on this output(pcm_sample = pcm_sample * 0.5), the samples will get clipped.

The sample values exceed the range [-1.0, 1.0] by a very marginal value(< 1e-2). This is due to the implementation and is expected. Also this can happen even if the encoder takes care of clipping.

Thanks!

[SakethSathuvalli]

Hi @sclsj,

Try once with our latest commit - 3b23f65

Thanks!

[sclsj]

Yes, it’s from the raw pcm generated by extren flag. I will try the latest commit. 获取 Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ 发件人: Saketh Sathuvalli ***@***.***> 发送时间: Wednesday, February 1, 2023 2:33:17 AM 收件人: ittiam-systems/libmpegh ***@***.***> 抄送: Jin, Joy ***@***.***>; Mention ***@***.***> 主题: Re: [ittiam-systems/libmpegh] Extract individual objects? (Issue #19) This email originated from outside Davidson College. Use caution, especially with links and attachments. Hi @sclsj<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsclsj&data=05%7C01%7Cjojin%40davidson.edu%7C1df4472e39b24813b1a508db04269704%7C35d8763cd2b14213b629f5df0af9e3c3%7C1%7C0%7C638108336088116109%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5CrfXNLwJXWeYVTFsY3N01oJ4cGQn3CXSfBZycDCe8Q%3D&reserved=0>, Try once with our latest commit - 3b23f65<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fittiam-systems%2Flibmpegh%2Fcommit%2F3b23f65bf7750847e2a46c3092fda9b75ac60a26&data=05%7C01%7Cjojin%40davidson.edu%7C1df4472e39b24813b1a508db04269704%7C35d8763cd2b14213b629f5df0af9e3c3%7C1%7C0%7C638108336088116109%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Gyala4GxIocFodNUFPJR4SkUO4pRI2HDJzZVZQaMGc8%3D&reserved=0> Thanks! ― Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fittiam-systems%2Flibmpegh%2Fissues%2F19%23issuecomment-1411587286&data=05%7C01%7Cjojin%40davidson.edu%7C1df4472e39b24813b1a508db04269704%7C35d8763cd2b14213b629f5df0af9e3c3%7C1%7C0%7C638108336088273194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=L%2FgMExHUlk7htN7eu%2BbWnl3HR6ZP%2FnS52ZCQ4TUsCn4%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJG5FLDXJXHZGH7LDTAAGZDWVIGT3ANCNFSM6AAAAAARE2I234&data=05%7C01%7Cjojin%40davidson.edu%7C1df4472e39b24813b1a508db04269704%7C35d8763cd2b14213b629f5df0af9e3c3%7C1%7C0%7C638108336088273194%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lQBrN08AgcGR%2B%2FHMrVK3nckk5APMqgzHEGq54nqtqYI%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[SakethSathuvalli]

Hi,

Please refer to our https://github.com/ittiam-systems/libmpegh/blob/main/docs/IA-MPEG-H-3D-Audio-Dec-GSG.doc and
https://github.com/ittiam-systems/libmpegh/blob/main/docs/IA-MPEG-H-3D-Audio-Dec-API.doc if any information is available for your use case.

Thanks!

[sclsj]

I'm sorry, I'm not very literate in this topic. I tried to read the guide but can't really understand it and so I can't find revevant information. Am I correct in understanding that this is not something I can do with the command-line program but something that might be possible by using the library?

[SakethSathuvalli]

Hi @sclsj,

Please refer to the entries in tables 3-1 and 3-2 of the API document. These tables have details about the buffers that carry the object audio information when external rendering is enabled. The interfaces are provided as per the specification for MPEG-H and expects the user of this data to have prior knowledge of programming and digital audio formats(PCM data of multichannel).

Thanks!

[sclsj]

Unfortunately I do not know any programming language. Is there a demo or sample code that allows extraction of object data (i.e. MPEG-H 3d mp4 --> adm bwf) anywhere?

[SakethSathuvalli]

Hi @sclsj,

We have recently added a wiki page on external rendering interfaces. Can You please check if it helps you with the issue you are facing?

Thanks!

[sclsj]

Hi @SakethSathuvalli

It definitely helped. However, it seems like I'm getting a lot of popping noises, perhaps due to some problems with the limiter, for example limiter not applied and resulting in heavy clipping. How can I set a lower global gain (for example, multiple every gain for object by 0.5x so it won't clip)? Or maybe disable DRC for objects completely? I can't find an option for either.

[sclsj]

Below is an example. See the first 2 channels that are heavily clipped/limited.

[sclsj]

I think these weird samples are why there are constant popping sounds. It seems like the clipped samples become negative? I'm not sure what's happening but it's definitely because the signal clipped.

[SakethSathuvalli]

I think these weird samples are why there are constant popping sounds. It seems like the clipped samples become negative? I'm not sure what's happening but it's definitely because the signal clipped.

@sclsj, Can You please let know if the waveform shown above is of the audio data file generated using -ext_ren:1 flag ?

[sclsj]

Hi @sclsj,

Try once with our latest commit - 3b23f65

Thanks!

It definitely worked.

However, the objects are still peak limited. How can I set a lower gain for objects?

For the channel output, I messed around with files containing "gain" and lowered the gain (therefore disabling DRC) by modifying all gain in impeghd_peak_limiter.c to 0.1. What would be the corresponding edit that would set a lower object gains (perhaps times a factor of 0.1)?

[SakethSathuvalli]

Hi @sclsj,

The DRC effect will not be applied on the external renderer interfaces PCM output. It appears that the original content itself is peak limited in nature.

Thanks!

[SakethSathuvalli]

Hi @sclsj,

We would like to close this issue if You can confirm that you are able to extract individual objects with the -ext_ren: switch.

Regarding the gain, the decoder does not have a provision to apply desired gain on decoded objects.

Thanks!

[sclsj]

Thank you for all your help! Yes, I was able to use the ext_ren switch. Just want to confirm again that no processing is done to the decoded objects for extren, and all the drc metadata and such are included in the bitstream file?

[sclsj]

Hi @sclsj,

Try once with our latest commit - 3b23f65

Thanks!

I'm still curious why that happened. From my experience you can exceed 0 dB in 24 bits? E.g. if I record in QuickTime the peak usually > 0dB and I have to apply like a -10dB gain. Even if 16 bit (e.g. convert from 24 bit to 16 bit in-place in Audition on file with >0dB peak), shouldn't be audio just be clipped? I thought that's the normal behavior?

And also why would that happen in the first place? Shouldn't the encoded objects be peak normalized/limited in the first place (by the encoder)?

And could you explain how did you fix this (e.g. lower the gain?)?

[sclsj]

Also, it seems like the new version of libmpegh fails to decode 360 RA files from Deezer.

ia_mpeghd_testbench -ext_ren:1 -ifile:/Users/jin/Music/deemix\ Music/YOASOBI\ -\ THE\ BOOK\ 2/07\ -\ もしも命が描けたら.mp4 -ofile:mingdeemix.wav
-ifile:/Users/jin/Music/deemix Music/YOASOBI - THE BOOK 2/07 - もしも命が描けたら.mp4 -ofile:mingdeemix.wav 
                       ITTIAM SYSTEMS PVT LTD, BANGALORE
                             http:\\www.ittiam.com
                      IA_MPEG_H_3D_AUD_DEC_ARM $Rev: 1.2 $
                                        

fatal error: Ittiam mpegh_dec  core coder module :Initialization: : Initialization fatal error

[SakethSathuvalli]

Also, it seems like the new version of libmpegh fails to decode 360 RA files from Deezer.

ia_mpeghd_testbench -ext_ren:1 -ifile:/Users/jin/Music/deemix\ Music/YOASOBI\ -\ THE\ BOOK\ 2/07\ -\ もしも命が描けたら.mp4 -ofile:mingdeemix.wav
-ifile:/Users/jin/Music/deemix Music/YOASOBI - THE BOOK 2/07 - もしも命が描けたら.mp4 -ofile:mingdeemix.wav 
                       ITTIAM SYSTEMS PVT LTD, BANGALORE
                             http:\\www.ittiam.com
                      IA_MPEG_H_3D_AUD_DEC_ARM $Rev: 1.2 $
                                        

fatal error: Ittiam mpegh_dec  core coder module :Initialization: : Initialization fatal error

The failure could be because the stream configurations may not be supported by our decoder. Example the stream could belong to a higher profile level.

[SakethSathuvalli]

Hi @sclsj,
Try once with our latest commit - 3b23f65
Thanks!

I'm still curious why that happened. From my experience you can exceed 0 dB in 24 bits? E.g. if I record in QuickTime the peak usually > 0dB and I have to apply like a -10dB gain. Even if 16 bit (e.g. convert from 24 bit to 16 bit in-place in Audition on file with >0dB peak), shouldn't be audio just be clipped? I thought that's the normal behavior?

And also why would that happen in the first place? Shouldn't the encoded objects be peak normalized/limited in the first place (by the encoder)?

And could you explain how did you fix this (e.g. lower the gain?)?

The fix is related to limiting the upper and lower limits of a 24-bit data. These are referred to as saturation fixes.

[sclsj]

Hi @sclsj,
Try once with our latest commit - 3b23f65
Thanks!

I'm still curious why that happened. From my experience you can exceed 0 dB in 24 bits? E.g. if I record in QuickTime the peak usually > 0dB and I have to apply like a -10dB gain. Even if 16 bit (e.g. convert from 24 bit to 16 bit in-place in Audition on file with >0dB peak), shouldn't be audio just be clipped? I thought that's the normal behavior?
And also why would that happen in the first place? Shouldn't the encoded objects be peak normalized/limited in the first place (by the encoder)?
And could you explain how did you fix this (e.g. lower the gain?)?

The fix is related to limiting the upper and lower limits of a 24-bit data. These are referred to as saturation fixes.

Can you tell me a bit more? I failed to get more information with basic search. Did you just clip the samples?

[SakethSathuvalli]

Hi @sclsj,
Try once with our latest commit - 3b23f65
Thanks!

I'm still curious why that happened. From my experience you can exceed 0 dB in 24 bits? E.g. if I record in QuickTime the peak usually > 0dB and I have to apply like a -10dB gain. Even if 16 bit (e.g. convert from 24 bit to 16 bit in-place in Audition on file with >0dB peak), shouldn't be audio just be clipped? I thought that's the normal behavior?
And also why would that happen in the first place? Shouldn't the encoded objects be peak normalized/limited in the first place (by the encoder)?
And could you explain how did you fix this (e.g. lower the gain?)?

The fix is related to limiting the upper and lower limits of a 24-bit data. These are referred to as saturation fixes.

Can you tell me a bit more? I failed to get more information with basic search. Did you just clip the samples?

Yes, the samples will be clipped to maximum / minimum value if they are found to exceed.

[sclsj]

Got it. What should I do if I don't want it to clip? Is it possible to reduce the gain or to output as 32-bit float, which wouldn't have this problem?

[SakethSathuvalli]

For the individual objects it is not possible to apply gain.

[SakethSathuvalli]

Even when float output is given the values exceeding [-1.0, 1.0] will be clipped. However, support for float output is currently not available

[sclsj]

But is it possible to solve this one way or another? Clipping is still bad, although not nearly as bad as inverted samples.

[sclsj]

Even when float output is given the values exceeding [-1.0, 1.0] will be clipped. However, support for float output is currently not available

I'm asking about float since QuickTime Player on macOS records in 24-bit float and every time I record most of the samples are > 0dB. I just do a -12dB volume reduction and it's ready to be used.

[SakethSathuvalli]

A not so clean way is to tap the output before clipping. This has to be done in the code. Please check the fix given for saturation overflow. It can help you with the buffer containing audio data that is not clipped.

Thanks!

[sclsj]

Emm... I just replaced that with pcm_sample = pcm_sample * 0.5f and it seemed to work... Other than increased noise floor, what problems might that cause?

[SakethSathuvalli]

When You try to apply gain on the output again the clipping behavior will be seen again when the data is stored as 16/24 bit PCM. Other than this I don't see any other implications.

Thanks!

[sclsj]

I don't think I understood you. Could you please explain more? Are you saying that if this 0.5 gain is not low enough for certain files, the ext_ren pcm will still be clipped?

Also, why would the encoded objects be greater than 0dB? Shouldn't the encoder clip any values exceeding [-1.0, 1.0]? I don't understand why, if the encoded objects can be greater than 1.0, the decoded output can't be with a significantly higher bitrate. I'm sorry if this question seems silly.

Thank you for all your help!

[sclsj]

Hello again! You locked #38 due to off-topic, so I'm unable to delete or hide my comment. You have my consent to delete or hide my comment.