lichen-community-systems / flocking-core Goto Github PK

We now having a working CI system using Github Actions. The next step is to create a process that will build release artifacts, package them up, and post them to the Github release page whenever code is pushed to a tag.

Ultimately, I think it will be easiest to co-locate all libflock-related libraries in the same repository, configured as a monorepo (or workspace in Cargo terms, I think) for, at least:

• libflock itself
  • the fully platform-agnostic signal generation library
• the platform-specific Hosts (responsible for handling input and output devices, opening/saving/decoding/encoding files, etc.)
  • CPAL
  • Bela
  • wasm Web Audio API

The Daisy boards are increasingly used for hardware musical instruments and Eurorack modules. While there are third-party Rust bindings available, it looks like they are still in the relatively early stage of development. My impression is that the best approach is to develop an Environment implementation in C++ using libDaisy, and to use Flocking via bindgen-created C bindings.

In order to determine if this is viable and sustainable, we should create a simple Environment implementation based on one of Daisy's examples, and implement a stub function in Flocking in order to establish the appropriate build supports for calling Rust code from C++. We'll also want to choose a platform on which to manage the cross-compilation process in CI (I'm tempted to use macOS for symmetry with my own current development environment, but Linux would be fine too).

The realtime portions of Flocking—namely, the unit generators and graph evaluator—should be located in a separate library crate the can be built across multiple platforms and includes support for both C and wasm bindings.

At the moment, there's no README, license, or anything else to orient people to the goals of this project. We should create documentation for these, and then move ownership of the repository to @continuing-creativity.

At this point, we've encountered a number of issues with ASIO support in CPAL:

1. It causes panics at runtime when enumerating WASAPI devices and then ASIO devices
2. It makes local builds more difficult, because the ASIO SDK and LLVM have to be installed
3. It's not clear how to conditionally include its required dependencies only on Windows, which otherwise causing builds to fail on Apple Silicon

Given this, it simply doesn't seem worth supporting ASIO until the whole system is more mature and (hopefully) our third-party dependencies have sorted out some of their bugs.

This new version of Flocking will have to be carefully partitioned into the portions of the system that are designed to run without a memory allocator or standard library (particularly, the unit generators and graph evaluator) and the portions that provide dynamic runtime supports (e.g. parsing and interpreting Composition JSON files, instantiating unit generators and wiring them dynamically, etc.).

Given that Cargo doesn't seem to provide good support for mixing no-std modules with those that require the standard library, we should divide up the repository into multiple directories that can contain fully separate Cargo workspaces/builds as needed.

This issue relates to #21.

CPAL supports ASIO as a host on Windows, but it requires installing the ASIO SDK and LLVM, and specifying a compile feature. We should add support for this.

For various reasons, I've come to the conclusion that Rust isn't the right fit for this project. Most prominently, the difficulty of writing graph-based systems in Rust, particularly without a full std environment, as well as the complexity of managing integration with Daisy's GCC-compiled C++ code, represent significant barriers.

Since Flocking's unit generators are written in a very C-like dialect of JavaScript, and with C's stability and ubiquity, plain old seems like the best option for a radically interoperable system like libflock.

A user should be able to specify generic audio settings such as sample rate, number of channels, buffer size and block size. They should also be able to specify Environment-specific settings such as the host to use (if appropriate) and which input and output devices to use.

In most cases, these will need to be treated as hints to Flocking, since not all Environments will support these being customizable (especially the web, which can't support user-specified input/output devices or sample rates, and which may provide constraints on the size of buffers).

Currently, we panic (using the expect() method) when a Host does not have any devices, or if it doesn't have default input or output devices registered. This will be rare in practice, but at very least Github's Windows-based Actions Runners have no audio devices setup. We should gracefully note the lack of devices and exit successfully.

This may be an issue with Big Sur or with running on M1 hardware, I'm not sure. However, when I try to build flocking-cpal, I get the following error

The following warnings were emitted during compilation:

warning: couldn't execute `llvm-config --prefix` (error: No such file or directory (os error 2))
warning: set the LLVM_CONFIG_PATH environment variable to the full path to a valid `llvm-config` executable (including the executable itself)

error: failed to run custom build command for `coreaudio-sys v0.2.6`

Caused by:
  process didn't exit successfully: `~/code/flocking-core/target/debug/build/coreaudio-sys-ba8cad63afa3ce94/build-script-build` (exit code: 101)
  --- stdout
  cargo:rerun-if-env-changed=COREAUDIO_SDK_PATH
  cargo:rustc-link-lib=framework=AudioToolbox
  cargo:rustc-link-lib=framework=AudioToolbox
  cargo:rustc-link-lib=framework=CoreAudio
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS
  cargo:warning=couldn't execute `llvm-config --prefix` (error: No such file or directory (os error 2))
  cargo:warning=set the LLVM_CONFIG_PATH environment variable to the full path to a valid `llvm-config` executable (including the executable itself)

  --- stderr
  thread 'main' panicked at 'libclang error; possible causes include:
  - Invalid flag syntax
  - Unrecognized flags
  - Invalid flag arguments
  - File I/O errors
  - Host vs. target architecture mismatch
  If you encounter an error missing from this list, please file an issue or a PR!', /Users/colin/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.53.3/src/ir/context.rs:573:15
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This seems to have to with the ASIO feature being turned on by default, and the Windows-specific specification of an older version of clang in flocking-cpal's Cargo.toml. Why this is the case only on M1, I'm not sure.

While our CI system can easily build the system on Windows, it can't be reused by developers who are building on their local machine. We should factor out that code into a script that can be run as part of building anywhere.

At least on some platforms, it should be possible to set up a virtual audio device so that we can write tests that verify that audio is correctly being written to a device—and hopefully even read.

On macOS, Blackhole should do the trick and seems to be installable with homebrew so hopefully will work within a Github Actions runner. On Linux, ALSA seems to have the ability to create virtual devices. On Windows, ASIO is our ideal host, but I'm not sure what's available there for virtual devices (and which can be installed with Chocolatey).

It seems like ffmpeg should be available on all these platforms and could be used to record the output from the virtual device or play back audio files into it.

At the moment, libflock-cpal is a configured as a binary crate, but it's ultimately intended to be used as a library by others who are building applications. To this end, we should convert the project to a library, and move the implementation that is currently in main() into a public function and create an example for it. Cargo seems to have nice support for building examples that are separate from a library but contained in the same repository.

My goal with the libflock-related repositories is to have a fully automated build, testing, and release process. Whenever someone creates a PR or pushes to a branch, the build and test process should be run. Whenever a tag is pushed, the release artifacts for each platform should be created and published.

Currently this project is set up as a Cargo workspace. This means that all common dependencies are shared. However, the cross-platform essential "core" of flocking-core (in the flocking directory) is intended to support usage in a no-std environment. However, since (as I understand it) features in Cargo are additive, this means that the whole workspace has to either support no-std or not, as there are cases otherwise where a transitive dependency (in our case, CPAL) may require a common dependency (Serde) with std=true which will override its inclusion in flocking with std=false. Presumably in a release build, this would work fine, but I've already experienced problems with Serde between the two environments. The best solution I can think of is just to take the flocking crate out of the workspace and manage it separately with it's own self-contained Cargo workflow.