michaelholmes4 / nncomp

Real-time recurrent neural networks for audio plugins

NNComp

This is the code base for the Honours thesis project submitted to the University of Queensland by Michael Holmes 2022.

Update 2023

New GUI!!
Now the plugin has a modern look and the weights of each neural network can be visualised in real-time.

Acknowledgements

The following papers were highly influential in guiding this project:

• Efficient neural networks for real-time modeling of analog dynamic range compression by Christian Steinmetz and Joshua Reiss.
• Real-time black-box modelling with recurrent neural networks by Alec Wright, Eero-Pekka Damskägg, and Vesa Välimäki

Introduction

This repo can be used to train RNN, LSTM and GRU neural networks and convert these networks into efficient C++ code for use in audio plugins.

The trained models from the thesis project can be downloaded here.

An audio plugin was created using iPlug2 and can be downloaded in the releases tab. *Note for best sound quality please run the plugin at a sample rate of 48kHz. Additional sample rates to be added in the future.

Contents

This repo is split into 2 modules: Training and Plugin. Detailed usage instructions are available inside each module.

Training

Code for training and testing the PyTorch models. A script is supplied for converting these models into C++ headers to use with the Plugin module.

Plugin

The iPlug2 project file is supplied along with quick C++ implementations that can be used in other projects.

nncomp's Issues

Thesis

Hi @michaelholmes4 ,

Cool work! I was wondering if your thesis document would be made available online.

Allocating on the RT Audio thread

This is looking really nice now!

I just noticed this though:

NNComp/Plugin/NNComp/NNComp.cpp

Line 116 in d700d83

gr[i] = new sample[nFrames];

It would be better to preallocate as much memory as you might need in OnReset(), rather than allocating and freeing memory in ProcessBlock()

Nice!

Great to see you're using iPlug2 for such a cool project :-)

If you need any help e.g. setting it up so it builds on github actions, feel free to reach out

best

oli