negative-size-param reported by asan when decoding a sample about libmpegh

[SakethSathuvalli]

Hi @miartad - Yes we are able to reproduce the issue with clang-14 and on MSVS as well. The root cause for the issue is truncation error happening in floating point arithmetic of a module. An initial(/quick) fix that we see is going for a bigger container format (from FLOAT32 to FLOAT64) for few of the variables. However, We are looking for other optimal solutions for scenarios like this. This will take some time at our end.

Thanks for Your patience and also for bringing this issue to our notice.

[Vamsi100858]

Hi @miartad , Can you please let know the compiler used and platform (x86/x64/armv7) on which this crash is observed ? We tried with clang-8 and not able to observe the crash. Thanks

[amiartus]

Hi @Vamsi100858

I have tried with clang 15 and gcc 11, both show the error when sanitize address parameter is added to cmake.

~/projects/libmpegh/b$ /usr/bin/cc --version
cc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0

~/projects/libmpegh/b$ clang --version
Ubuntu clang version 15.0.6

~/projects/libmpegh/b$ ubuntu-report
This is the result of hardware and optional installer/upgrader that we collected:
{
  "Version": "22.04",
  "OEM": {
    "Vendor": "Dell Inc.",
    "Product": "Precision 14 5470",
    "Family": "Precision"
  },
  "BIOS": {
    "Vendor": "Dell Inc.",
    "Version": "1.4.2"
  },
  "Arch": "amd64",

[amiartus]

@Vamsi100858 were you able to reproduce the issue?

[SakethSathuvalli]

Hi @miartad,

We have added a simple fix (of all the things that we tried internally) for this issue, since its been some time this issue is added here. Thanks for Your patience!

--
Saketh

[amiartus]

hi @SakethSathuvalli I checked the main branch and the issue is fixed there by a389d71

fatal error: Ittiam mpegh_dec  core coder module :Initialization: : Initialization fatal error

the fix in #46 is not necessary to resolve this issue