Different behavior on amd64 and arm about constriction

[actcwlf]

Thanks for your reply! Adding a small regularization is a good idea. As in previous comment, I don't have access to that device for now. If I get more detailed information, I will update the information here.

[robamler]

Thank you for reporting! On arm, are you using the precompiled constriction package from pypi (for new Macs with arm64 chips) or did you compile constriction from source? If you compiled from source, did you compile in --release mode? The error message that you see should appear in a debug build but it should not appear in a release build (and the official packages on pypi should all be release builds unless I messed something up in the CI).

In case you indeed compiled from source: I'd be happy to provide precompiled python packages for your platform going forward, but I'd need someone to test them, at least initially. Would you mind telling me what platform you're developing for and would you consider testing a precompiled python package on it?

Background

The second symbol in your example is encoded with a Laplace distribution with scale parameter b = 0.0, which is not a well-defined distribution (the pdf of a Laplace distribution is $p(x\mid\mu,b)=\frac {1}{2b}e^{-|x-\mu |/b}$, so $b=0$ would lead to a division by zero). The Laplace distribution is implemented in the probability crate, which checks for b > 0.0 using a should! macro, which is defined to be equivalent to debug_assert!. Unless special compiler flags are used, debug_assert! only checks in debug builds and is ignored in release builds.

Notes to Self

• We should at least document that the Laplace distribution requires b > 0.0, and maybe the python API should check for it even in release builds. In fact, the provided example works on amd64 only by coincidence. It would fail to reconstruct the original message if we changed the second symbol in message to anything other than 10 because probability—justifiably—does not provide a valid quantile function for Laplace distributions with b = 0.0. [update on 2023-11-25: I can't reproduce this anymore. Maybe I tried it out in debug mode myself when I tested this originally]
• I guess the readme should document that, when compiling python packages for production use from source, one should compile in --release mode. This may be obvious to rust developers but these instructions are mainly for developers who are at home in python land and not necessarily very familiar with rust.

[actcwlf]

Thanks for your reply!

I compiled from source following

compile in release mode (i.e., with optimizations) and run the benchmarks: cargo bench

and

build the Python module: poetry run maturin develop --features pybindings

Because I am pretty new to rust, I am not very sure whether I compiled the package in release mode at that moment.

Unfortunately I dont have the original device to test the solution now. If the behavior is reproduciable on other platform in debug mode, I think this may be the main reason.

By the way, when $b=0$, can I suppose the decode result is $\mu$ ? Is this a correct workaround?

[actcwlf]

I make another test on amd64

import constriction
import numpy as np

message = np.array([10, 5, 6], dtype=np.int32)

entropy_model = constriction.stream.model.QuantizedLaplace(
            -20, 20 + 1
        )

encoder = constriction.stream.queue.RangeEncoder()
encoder.encode(message, entropy_model, np.array([10., 10., 10.]), np.array([10., 0., 0.]))

compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

decoder = constriction.stream.queue.RangeDecoder(compressed)
decoded = decoder.decode(entropy_model, np.array([10., 10., 10.]), np.array([10.,  0., 0.]))

print(decoded)
assert np.all(decoded == message) # (verifies correctness)

the result is

compressed representation: [2042752312  556892819]
(in binary: ['0b1111001110000011110110100111000', '0b100001001100011000001010010011'])
[10  5  6]

May be there is some fallback when $b=0$?

[robamler]

You are right, leakily quantized Laplace distributions with $b=0$ do actually work in release builds. I can't reproduce my tests where these failed, maybe I accidentally tested in debug mode myself.

I compiled from source following
[...]

build the Python module: poetry run maturin develop --features pybindings

Because I am pretty new to rust, I am not very sure whether I compiled the package in release mode at that moment.

This compiles in debug mode (i.e., it will result in a python package that contains more checks and fewer run-time optimizations). This explains the difference in behavior between your two builds.

(For completeness, if you want to compile in release mode, use: poetry run maturin develop --features pybindings --release; I updated the readme accordingly.)

I'm a bit unsure what the conclusion should be. On the one hand, one could interpret a Laplace distribution with $b=0$ as a delta distribution, and one could argue that a delta distribution should work with our leaky quantizer in most cases (see below) because the quantizer integrates the delta distribution over a finite interval, and the leakiness adds some small nonzero probability mass even if the integral does not contain the delta peak. And it indeed seems to work in experiments.

On the other hand, there are two reasons against allowing $b=0$ for Laplace distributions:

• parity with the rust API: while all provided precompiled python modules are compiled in release mode, users of the rust API will probably develop in debug mode, and there the probability crate (which is not under my control) forbids $b=0$ (and I think that's the right thing to do for a general purpose probability crate).
• There's an edge case where quantizing a delta distribution is not well-defined: when the delta peak is right at the boundary between two bins. In this case, when the leaky quantizer integrates over one of the two adjacent bins, it's not clear whether the peak should be counted towards the probability mass. As far as I can tell, the way the Laplace distribution is implemented in the probability crate, it should evaluate 0.0 / 0.0 for this case on this line, which should result in NaN for both debug and release builds. Weirdly, I can't manage to trigger this issue in practice, so it seems like it's not an issue, but I want to understand why it's not an issue before relying on it.
  [update (2023-11-26): in this edge case, the delta peak is counted entirely towards the right bin. The NaN result from Laplace::distribution is cast into an integer here and here, which results in 0 as per rust reference so that the cdf still evaluates to zero at the delta peak. It still feels dangerous to rely on this behavior, especially if the easier solution is to just add a tiny amount of regularization to $b$.]

Request for feedback

To help me decide, could you give me some more background on why you tested with a Laplace distribution with $b=0$? Did this come up in a real use case or were you deliberately coming up with edge cases for stress testing? In other words, would it suffice for your use case if constriction enforced $b>0$ and if this requirement was documented in the python API reference, or would it make your life easier if $b=0$ was accepted and interpreted as a delta distribution?

[actcwlf]

Thanks for your explanations! I am doing some research on Cool-Chic and encountered the case $b=0$. For my use case, interpreting a Laplace distribution as a Delta distribution is useful when $b=0$. But I think these two reasons against allowing $b=0$ are reasonable. Perhaps adding some options is a good alternative? For example, add a strict=true option in constriction.stream.model.QuantizedLaplace to enforce the check that $b>0$ and strict=false for more flexible interpretation.

[robamler]

Thanks for the background information. I like the idea about a strict=true argument to QuantizedLaplace, but I decided against it for the following reasons (this list is not meant as criticism, it's just to document and communicate my reasoning):

• I think it would add unnecessary complexity to the python API where a simpler solution exists (see below);
• it would still break the promise that the python API is a subset of the rust API, and would thus prevent users from porting python prototypes to exactly equivalent rust, at least for rust debug builds; and
• it relies on somewhat brittle behavior that might break, e.g., with future versions of the probability crate (which are out of my control).

Proposed Solution

The problem of $b=0$ can be easily avoided by adding some small regularization (e.g., 1e-16) to the scale parameter. I added this recommendation to the python API documentation (e.g., here), and I added unit tests to assert that quantizing continuous distributions with even much smaller scales behaves as expected without any numerical issues.

Regarding Precompiled Packages for arm

Unrelated to the specific issue reported here, please let me know if you'd be willing to test a precompiled python package for your computer architecture so I can upload official pip packages to pypi going forward. This would make things easier also for other people with your compute architecture. I'd need to know your host triple to do this (i.e., what rustc --version --verbose says after "host").

[actcwlf]

Hi, I have got the original device, and I would like to know if you can provide a precompiled version now. I would be happy to do some testing. The system info is

rustc 1.77.1 (7cf61ebde 2024-03-27)
binary: rustc
commit-hash: 7cf61ebde7b22796c69757901dd346d0fe70bd97
commit-date: 2024-03-27
host: aarch64-unknown-linux-gnu
release: 1.77.1
LLVM version: 17.0.6

The device is a Jetson nano and the recommanded version of python is 3.6. I found release v0.2.4 support 3.6 but failed to install it. Is it feasible to compile a newer version to the platform?
Thanks in advance!