Coder Social home page Coder Social logo

christofmuc / jammernetz Goto Github PK

View Code? Open in Web Editor NEW
49.0 8.0 9.0 15.35 MB

OpenSource internet jam sessions, aka Networked Music Performance software.

License: GNU Affero General Public License v3.0

C 1.27% C++ 84.95% Dockerfile 0.51% Batchfile 0.38% CMake 9.49% Shell 0.70% Python 2.70%
juce-application nmp networked-music-performance juce audio-streaming audio-applications audio-recorder

jammernetz's Introduction

release Build Status

Introduction

JammerNetz is an Open Source system for "networked music performance" (NMP), also known as "jamming over the internet". It consists of a lightweight server software, and a little UI client software that can use your audio devices and stream multiple uncompressed high-quality audio channels to other participants as a common mixdown.

The design choice is that if you have a modern DSL or fibre connection there is no need to degrade your music experience using CODECs mostly designed for speech transmission as used in some other internet jam solutions. We are happily using JammerNetz since over two years for our weekly sessions with synths, voice, and electric guitar.

Of course be aware that the main influence on the quality has your internet provider and your choice where you are running the server. If you have high quality fibre and a good and fast ASIO-capable audio interface, you can expect 50ms total air-to-air latency, which we think is awesome. Of course, if some participants are bound for cable internet and are trying to play together across all of Germany, even running the server in a suitable AWS instance in Frankfurt close to the internet's main hub will not get you much better than 70-80 ms total audio latency (including AD and DA conversion in your interface), which we feel is still worth the effort! Do not try to use WiFi.

Features

JammerNetz is quite feature rich, the following are the main items:

  • It allows you to host a jam session on a server, allowing a configurable number of clients to participate.
  • It allows for channel configuration to send mono or stereo channels, and also a "send-only" channel type for using microphones without hearing your own voice. This makes for great communication during the session.
  • Send multiple audio channels per client to the server, e.g. your synthesist can send the keyboards and his voice separately.
  • Does automatic hard-disk recording of your session to local disk on each client in a lossless compressed FLAC file. After the session, everybody has a record to revisit.
  • Does automatic MIDI recording in case it detects any incoming MIDI notes, thereby logging all keys played into a MIDI file for later revisit ("what did I play? Sounds great!")
  • Features a built-in instrument tuner display showing you the detected note and cents for each channel, so it is easy and quick to get everybody on the same A.
  • BlowFish encryption based on a shared secret, so you are not sending data unsecured through the internet. We don't claim this is state of the art and probably not enough bits of encryption, but better than sending unencrypted audio data. This certainly is a point for improvement.

Screenshot

Here is a screenshot

Limitations

It should be noted that due to the design of the system, we have a few limitations or restrictions that other systems might not have. We believe that we have made sensible trade-offs, but your milage may vary:

  1. All clients need to run on the same sample rate (48000 is set in the source, but you might want to switch to 44100 in file BuffersConfig.h).
  2. The network packets are set to 128 buffer size, which we feel is the best trade off between number of packets per second, MTU, and latency. The Audio device can run in different / biffer buffer sizes now, but be aware that this might increase latency and require bigger jitter buffers on the server (command line parameter on the server!).
  3. As we are aiming for lowest-possible latency, you should really use an audio device with ASIO drivers on Windows, even if Windows Audio in different modes is offered. Stay away from DirectSound. Mac CoreAudio works as well very nicely, as does Jack on Linux.

Usage

We provide installers for Windows and Mac client to download here in the release section. The server executable is also installed by the Windows installer for you to test it e.g. locally, but for real life application you'd need to run the server somewhere in the cloud.

you will need some experience in compiling a C++ application and starting an AWS (or Azure or self-hosted or...) instance and deploying the Linux build of the server there to run it. Depending on the interest in this system, we might be able to provide more help.

Building the software

Supported platforms

We use JUCE, a cross-platform library with support for all major platforms, but we have tested the client software at the moment only on Windows 10 using MS Visual Studio 2019 and macOS 10.15, and the server on an up-to-date Ubuntu Linux 20.04 LTS. Other platforms might work as well, but might require some fiddling and fixing.

Downloading

Clone with submodules from github

git clone --recurse-submodules -j8 https://github.com/christofmuc/JammerNetz

The recursive clone with submodules is required to retrieve the following additional modules already into the right spot inside the source tree:

  1. We use the magnificent JUCE library to immensly reduce the amount of work we have to do.
  2. the nice JUCE module ff_meters from ffAudio to display the level meters for each channel.
  3. Q, a highly interesting modern C++ DSP library we use for the instrument tuning/pitch detection. Go check it out, it's really cool!
  4. Infra, a little helper library required by Q.
  5. Flatbuffers, a C++ serialization library we use for parts of the network protocol.

As we don't want to send any unencrypted audio data through the internet, we use a simple BlowFish encryption scheme to make sure that only authorized people join the jam session. More on that below.

Installing more dependencies with Conan

We are moving towards Conan for dependency management, and in order to include the pdcurses library on Windows, just run the following conan install command before building from within the JammerNetz top-level-directory:

conan install -if Builds\Windows -s build_type=Release --build missing .

In case you do not have conan installed, have a look at their documentation and download page. It is a great tool!

Building on Windows

We use modern CMake 3.14 and Visual Studio 2019 for C++. Make sure to have both of these installed. Newer Visual Studios might work as well, you can select them as generators in CMake.

Optionally, if you want to produce a Windows-style installer for your band members: We always recommend the InnoSetup tool, really one of these golden tools that despite its age shines on and on. Download it and install it, it will automatically be picked up and used by the build process.

Using CMake and building JammerNetz client and server is a multi-step build:

cd third_party\flatbuffers
cmake -S . -B Builds -G "Visual Studio 16 2019" -A x64
cmake --build Builds --config Release
cd ..\..
cmake -S . -B Builds\Windows -G "Visual Studio 16 2019" -A x64
cmake --build Builds\Windows  --config Release

The build will take a few minutes, and produce Release versions of Client and Server software, as well as a client installer in case you have InnoSetup installed before kicking off. The installer executable is created as <JammerNetzDir>\Builds\Client\jammernetz_setup_x.x.x.exe

To test it, you can launch the server on your local machine with

<JammerNetzDir>\Builds\Server\Release\JammerNetzServer.exe -k [nameOfSecretsFile]

Simply use the "connect to local server" checkbox of the client.

Building on macOS

We tested on macOS Mojave 10.15:

First install your prerequisites with brew:

brew install cmake gtk+3 glew

Then run

cd third_party/flatbuffers
cmake -S . -B LinuxBuilds
cmake --build LinuxBuilds -j8
cd ../..
cmake -S . -B Builds/macOS -DCMAKE_BUILD_TYPE=Release
cmake --build Builds/macOS -j8

Building the server for Linux on Windows

Most likely, you're not going to run your server on a Windows machine but prefer a Linux cloud machine located at some strategic position in the Internet.

Cross-platform building Linux server on Windows 10 using Docker

Thanks to Docker, it has never been easier to do cross-platform development on Windows. If you don't have it, get yourself Docker Desktop and experience the power!

Building the Linux version from command line

After installing Docker Desktop, we're ready to do a cross-platform build for Linux.

First, what we are going to do is to create a docker image for the build machine we are going to use. For that, open a command line, cd into the directory <JammerNetzDir>\Server\docker\buildmachine and run the command

buildBuildmachine.bat

You can build the Linux version from a Windows command line by just running the provided batch file inside the buildmachine directory. You need to specify the JammerNetz top level directory in the funky Docker volume syntax with forward slashes and a double slash for the drive letter. E.g. the Windows directory D:\Development\JammerNetz needs to be specfied as //d/Development/JammerNetz:

makeLinux.bat //d/Development/JammerNetz

This is so the Linux virtual machine will mount the source directory from the host, and directly place it's build output on the host, so the virtual machine is immediately removed again after the build is finished.

You will find the output of the Build machine in the directory <JammerNetzDir>\Server\Builds\LinuxMakefile\build. The server executable is just called "JammerNetzServer" and is ready to run on a matching Linux machine.

Deploying the Linux build to a real Ubuntu server

The build above produced a Linux executable. If you have an Ubuntu server running you want to use, e.g. an Amazon EC2 instance, you can copy the executable to the server e.g. with scp, then dial into the machine with ssh and launch it. For example (from within the <JammerNetz> directory, use the IP of your server of course):

scp Builds\LinuxBuilds\Server\JammerNetzServer [email protected]:.

Then dial into your server, using ssh or putty, and make sure to have all runtime prequisites installed. With that done, you can just launch the server:

apt-get install -y libtbb-dev libasound2-dev libjack-dev
./JammerNetzServer -k [nameOfSecretsFile]

The server should start up and announce its presensence with a happy

Server listening on port 7777
Starting JammerNetz server, using CTRL-C to stop

All clients should be able to connect to the server via its IP address.

Native Linux builds

If you are working on Linux, make sure to have all development prerequisites installed!

Debian/Ubuntu

On a fresh Debian 9 machine, that would e.g. require the following installs:

sudo apt-get -y install g++ libasound2-dev libcurl4-openssl-dev libfreetype6-dev libncurses-dev libjack-dev libx11-dev libglew-dev mesa-common-dev webkit2gtk-4.0

Fedora

For a Fedora-based distribution like Amazon Linux 2, you would use yum to install the dependencies:

sudo yum install alsa-lib-devel libcurl-devel freetype-devel ncurses-devel jack-audio-connection-kit-devel libX11-devel  mesa-libGL-devel webkitgtk4-devel glew-devel

Compilation

With those installs and the recursive git clone from above, cd into the cloned directory and run cmake with the following commands:

cmake -S . -B builds -G "Unix Makefiles"
cd third_party/flatbuffers
cmake -S . -B LinuxBuilds -G "Unix Makefiles" -D FLATBUFFERS_CXX_FLAGS="-Wno-error"
cmake --build LinuxBuilds/
cd ../..
cmake --build builds

This should have created a server binary as builds/Server/JammerNetzServer and a client binary as builds/Client/JammerNetzClient.

To launch the server, just type

./builds/Server/JammerNetzServer -k [nameOfSecretsFile]

and it shall listen on port 7777. If you omit the -k you can use unencrypted traffic, then the client also needs a blank secrets file path. Recommended for testing only.

The gcc version seems to matter, I am testing with a vanilla Ubuntu 18.04 LTS installation which comes with gcc 7.5.0 out of the box.

The encryption secrets

Earlier versions had the 72 random bytes required for the Blowfish encryption compiled into the executable. But now the secrets are read from the command line by the server, and the client has a browse to... feature to select a file with the secret key. The encryption is symmetric, so both the server and all clients need to have the same secrets file, and the secure distribution of the key is left to the user.

To generate the shared secret, create a file e.g. named RandomNumbers.bin and specify this in the command line when launching the server, and distribute to the clients for selecting in the UI.

For example, you can use an external source like https://www.random.org/bytes/ to generate 72 random bytes, or use a more trustworthy key source as you like, or on Linux and Mac use your random device:

head -c 72 /dev/urandom > JammerNetz-secret.bin

Similar systems

We had used the great Jamulus system before developing our own system, and JammerNetz certainly has been inspired by this great piece of software. Please note that JammerNetz was created in pre-pandemic times, and also before Jamulus got its extra surge in community activity.

We also made some substantial design and architecture changes over Jamulus, justifying a new development instead of contributing to the Jamulus codebase. Most importantly, while Jamulus is using Qt as a cross-platform library, JammerNetz uses JUCE, massively reducing the lines of code required.

Licensing

As some substantial work has gone into the development of this, I decided to offer a dual license - AGPL, see the LICENSE.md file for the details, for everybody interested in how this works and willing to spend some time her- or himself on this, and a commercial MIT license available from me on request. Thus I can help the OpenSource community without blocking possible commercial applications.

Contributing

All pull requests and issues welcome, I will try to get back to you as soon as I can. Due to the dual licensing please be aware that I will need to request transfer of copyright on accepting a PR.

Special thanks

Special thanks go to our contributors, namely Viktor for helping in creating the first build for macOS!

About the author

Christof is a lifelong software developer having worked in various industries, and can't stop his programming hobby anyway.

jammernetz's People

Contributors

christofmuc avatar johan12345 avatar zalez avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jammernetz's Issues

Floating point assert in zero_crossing::peak_pulse()

The debug version asserts after a few hours in the std::max() function, in a piece of code that shouldn't assert. I am using Visual Studio 2017.

This is what I see:

SourceCode asserting with "invalid comperator":

template<class _Ty>
_Post_equal_to_(_Left < _Right ? _Right : _Left)
_NODISCARD constexpr const _Ty& (max)(const _Ty& _Left, const _Ty& _Right)
_NOEXCEPT_COND(_NOEXCEPT_OPER(_Left < _Right))
{	// return larger of _Left and _Right
    if (_Left < _Right)
    {
        _STL_ASSERT(!(_Right < _Left), "invalid comparator");
        return (_Right);
    }
return (_Left);
}

Float values displayed in Debugger:

_Left	5.04023665e-05	const float &
_Right	0.000307894428	const float &

32 bit Int cast of them:
(int)&_Left 944989985 int
(int)&_Right 966880484 int

Cleary there is no reason for the CPU to calculate _Left is smaller than _Right and _Right is smaller than left.

Have you encountered any of this? I post it here because this is where it happened, but it looks more like a compiler or CPU bug. The assembler looks good, it does nothing else than

00007FF7D063F5A5  mov         rax,qword ptr [_Left]  
00007FF7D063F5AA  mov         rcx,qword ptr [_Right]  
00007FF7D063F5AF  movss       xmm0,dword ptr [rax]  
00007FF7D063F5B3  comiss      xmm0,dword ptr [rcx]  
00007FF7D063F5B6  ja          std::max<float>+3Ah (07FF7D063F5BAh)  
00007FF7D063F5B8  jmp         std::max<float>+0A5h (07FF7D063F625h)  

Where I'd say the comiss can't be that bad.

CPU: i9-9900K

I tried googling this but didn't succeed. I filed this with the Q library, let's see if they have any idea.

Reproducing it with just calling std::max() on the inputs above is not successful, there must be some more state in the FPU to trigger this.

Could add better error concealment

Currently, I either repeat a package or insert silence.

Several better algorithms exists, list of papers:

Windows Audio devices pretend they work, but then don't

This is really strange - when you select some Windows Audio Devices like a cheap USB headset, Windows Audio claims they can do 48000 KHz at buffer size 128 - but then they don't.

The software opens the audio device just fine, but the measured sample rate suddenly can be as low as 12000 Hz, or just above 30000 Hz. This indicates a different buffer size was chosen after all.

I need to either fix the buffer size problem, or detect that the Sample rate measured is way off and stop even try pretending it works.

Linux compile error (when compiling oneTBB)

[ 49%] Linking CXX static library libds_look_and_feel.a [ 49%] Built target ds_look_and_feel [ 50%] Building CXX object third_party/oneTBB/src/tbb/CMakeFiles/tbb.dir/allocator.cpp.o [ 50%] Building CXX object third_party/oneTBB/src/tbb/CMakeFiles/tbb.dir/arena.cpp.o In file included from /usr/lib/gcc/x86_64-linux-gnu/11/include/x86gprintrin.h:89, from /usr/lib/gcc/x86_64-linux-gnu/11/include/immintrin.h:27, from /root/JammerNetz/third_party/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:33, from /root/JammerNetz/third_party/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:26, from /root/JammerNetz/third_party/oneTBB/src/tbb/task_dispatcher.h:20, from /root/JammerNetz/third_party/oneTBB/src/tbb/arena.cpp:17: /usr/lib/gcc/x86_64-linux-gnu/11/include/waitpkgintrin.h: In function ‘void tbb::detail::r1::prolonged_pause()’: /usr/lib/gcc/x86_64-linux-gnu/11/include/waitpkgintrin.h:53:1: error: inlining failed in call to ‘always_inline’ ‘unsigned char _tpause(unsigned int, long long unsigned int)’: target specific option mismatch 53 | _tpause (unsigned int __A, unsigned long long __B) | ^~~~~~~ compilation terminated due to -Wfatal-errors. gmake[2]: *** [third_party/oneTBB/src/tbb/CMakeFiles/tbb.dir/build.make:90: third_party/oneTBB/src/tbb/CMakeFiles/tbb.dir/arena.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:712: third_party/oneTBB/src/tbb/CMakeFiles/tbb.dir/all] Error 2 gmake: *** [Makefile:166: all] Error 2

Have to evaluate practicality of pitch detection with real instruments. A single oscillator works fine.

DOC: Hint on creating a random secret

In the GitHub README a hint on how to create 72 bytes of random data could be added. The following command works well on macOS and linux:

head -c 72 /dev/urandom > JammerNetz-secret.bin

For easy distribution of a secret to users: how about a feature that calculates some sort of checksum on a file, for example an image. So if a jpg is addressed with the -k option, it reverts to that. Providing someone with a key is then as simple as sharing a picture with them. Small picture files generally don't get recompressed by WhatsApp or Facebook, and can be easily uploaded to and downloaded from forums, web pages and other sources. The entropy would suffice and people could use logo's and other human recognizable content as the 'key' to the server!

Got package I couldn't decipher from "client's IP" - somebody trying to break in?

Hi,
I got this message on the console when I try to conncet a client:
Got package I couldn't decipher from "client's IP" - somebody trying to break in?
Maybe a shared secret pb ? RandomNumbers.bin were created under /common with the same shared secret on server side and client side before compilation...
Have you got any idea about the pb ?

Want to be able to control local mix

The current design does force all participants to hear - except for drops - exactly the same. For some use cases, this might not be required/desired, so there could be an option to override volume sliders from the general mix with my own taste.

Want local monitoring

Instead of the "Send" function, which only uploads yourself but you will not hear yourself from the server, we should add a local monitoring function that mixes the locally recorded audio into the queue, with adjustable volume and adjustable delay.

SERVER: Port 0 is accepted when no "=" sign is used in specifying command line param

When starting the linux server compiled from the latest GitHub sources, and started with
JammerNetzServer -k JammerNetzSecret_202105140952.bin --port 22125 the output shows Server listening on port 0

Tried different port numbers, also didn't work.

I can't connect to the server, so using the --port switch may be breaking the server altogether?

Using Ubuntu 20.04 and the compilation instructions in the Github readme.

Choirs: Large number of participants. Would like to have submixers

The UI currently is not suited for a larger number of participants. a) the quality info data is not really needed by default (could be a break-out window?), and you might want to group people info submixers.

If we can mix others (not only ourselves), this could be a pure UI feature in moving multiple volume sliders at once.

Want to specify which port to use

Currently port 7777 is hard coded, but if we want to launch multiple servers (rooms) on one machine, it would be nicer to specify the port at startup. Of course, the client then needs to allow this as well.

On Windows, client should show button to configure driver

because it can be arbitrarily hard to explain to somebody on how to navigate to the ASIO settings of the driver on a Windows machine. I myself can't find it when I search.

virtual bool AudioIODevice::hasControlPanel 

and

virtual bool AudioIODevice::showControlPanel 

Compilation on Linux

Hi,
Compilation was fine with GCC 7.5.0 (Ubuntu 18.04), but failed with GCC 10.2.0 (Archlinux).
With GCC 10.2.0, flatbuffers' compilation fails with an Werror. It can be overided by:

  • adding FLATBUFFERS_CXX_FLAGS cmake option (see here) ;
  • calling CMake with this option in <JammerNetz>/third_party/flatbuffers: cmake -S . -B LinuxBuilds -G "Unix Makefiles" -D FLATBUFFERS_CXX_FLAGS="-Wno-error".

Perhaps informations could be given here about your GCC version ?

Rename Input Channel

Hello , first , nice job !
Would it be possible to add possibility to rename input channel ?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.