fwcd / mixxx-bisect

Small tool for finding Mixxx regressions on Linux, macOS and Windows by binary searching over prebuilt snapshots

Mixxx Bisect

A small tool for finding regressions in Mixxx, inspired by mozregression. The tool binary searches over a commit range and lets the user tag automatically downloaded Mixxx snapshots with good/bad to identify the commit introducing the regression.

Note

The mixxx.org downloads server only provides binary snapshots for macOS and Windows, therefore m1xxx is the only supported snapshot repository on Linux.

Usage

To search a specific range of commits, run:

mixxx-bisect -g <good commit> -b <bad commit>

To search the entire range of available snapshots, run mixxx-bisect without arguments.

Development

To set up a development environment, create a venv with

python3 -m venv venv
. venv/bin/activate

Then, install the package along with its dependencies in editable mode:

pip3 install -e .

mixxx-bisect should now be on the PATH in the venv.

mixxx-bisect's Issues

Improve CLI around specifying explicit ranges

Currently, the user manually has to check the available commits in https://downloads.mixxx.org/snapshots/main. We could improve this by e.g. letting the user search a window of the latest n snapshots via a flag.

Modularize into package

We should modularize the regression finder into a proper (installable) Python package and move the default download/mountpoint into ~/.local/state (but keep it customizable). While we're at it, perhaps we'd want to give the project a more concise name, e.g. mixxxregression?

Add support for searching for regressions over `m1xxx` builds

This could be done e.g. by adding m1xxx as a submodule, by querying the GitHub API or both.

Add ability to run single commits

Just as a small convenience.

Investigate caching downloaded snapshots

We could e.g. decide to cache recent snapshots, or the first few layers of the binary search tree (though regressions will likely be located in different parts of the snapshot space, the only comprehensive solution would likely be to cache all snapshots, which would require a lot of space)

Make sure that disks unmount successfully before proceeding

While macOS seems to be able to happily mount images on top of each other, we should probably take care of this:

$ df -h
...
/dev/disk7s1    145Mi  133Mi   23Mi    86%    2686  4294964593    0%   /Users/<user>/git/mixxx-regression-finder/mnt
/dev/disk15s1   145Mi  133Mi   23Mi    86%    2685  4294964594    0%   /Users/<user>/git/mixxx-regression-finder/mnt
/dev/disk16s1   156Mi  133Mi   25Mi    85%    2806  4294964473    0%   /Users/<user>/git/mixxx-regression-finder/mnt
/dev/disk17s1   158Mi  133Mi   25Mi    85%    2821  4294964458    0%   /Users/<user>/git/mixxx-regression-finder/mnt

The reason for why these couldn't be unmounted in the first place was possibly due to mixxxdj/mixxx#11516.

Investigate how Linux could be supported

Mixxx distributes (binary) nightly builds for Linux for Ubuntu via a PPA¹². Unfortunately, it seems as if Launchpad removes older packages³, so we might have to find some other source for the packages. Mixxx does build Debian packages as part of their GitHub Actions CI, these artifacts, however, expire after a while.

For the purposes of this tool, we could look into setting up a clean Debian chroot and installing the package along with its dependencies there. Since dependencies may change, we should investigate how this could be done in a performant way (though Qt 5 and 6 should already be separate packages, so we might be able to get away with always installing the newest versions of the dependencies for the requested snapshot).

Footnotes

1. https://mixxx.org/download/#testing ↩

2. https://launchpad.net/~mixxx/+archive/ubuntu/nightlies ↩

3. https://askubuntu.com/a/375838 ↩

Check architecture

We should make sure to only use supported architectures, and ideally in their order of preference (arm64 > x86_64 on Apple Silicon Macs, x86_64 on Intel Macs).

Verify sha256sum after download

Just to make sure that the file was not corrupted during download.