revolupowered / one-voip-godot-4

One voip plugin to rule them all

VoIP extension for Godot 4

Add voice chat to your game in minutes!

Features:

• Uses native godot bus effects and audio streams ✅
• Opus compression ✅
• Jitter buffer 🚧
• Echo cancelling 🚧

Builds for:

• Windows ✅
• Linux ✅
• Mac ✅
• Android 🚧
• Web 🚧

How to Use

Install webrtc-native by downloading the latest release and dragging the webrtc-native folder into your godot project's root folder

Install one-voip by downloading the latest release and dragging the one-voip folder into your godot project's root folder

Spawn in a new bus in your project's audio mixer for your microphone, mute the bus (unless you want to listen to yourself), and add an VOIPInputCapture.

Add an AudioStreamPlayer somewhere in the scene, add a new AudioStreamMicrophone as the stream, set the Bus to the one you made (e.g. in the demo it's called "Mic") and turn on autoplay.

To send audio, connect a function to the packet_ready signal on the effect:

var idx = AudioServer.get_bus_index("Mic")
var effect = AudioServer.get_bus_effect(idx, 0)
effect.packet_ready.connect(_on_packet_ready)

And send the packet using any method you like in the _on_packet_ready function.

To receive audio, instantiate an AudioStreamVOIP every time a peer connects, add it to some kind of audio stream player, and MAKE SURE you play it. When you receive a packet, figure out what user sent it, and then push it to their AudioStreamVOIP:

stream.push_packet(packet)

Check out the demos for a full example.

If you use the demo ensure that AudioStreamPlayer uses unique AudioStreamVOIP making it local to scene.

Compiling

Windows

Install CMake and Visual Studio build tools (MinGW will also work by modifying the opus path in SConstruct, instructions are for MSBuild)

In thirdparty/opus: cmake -Bbuild

In thirdparty/opus/build: msbuild Opus.sln /p:Configuration=Release

In the project root: scons

This will build to demo_rtc/bin

Linux

Possible Dependencies

In thirdparty/opus: cmake -Bbuild -DCMAKE_POSITION_INDEPENDENT_CODE=ON

In thirdparty/opus/build: make

In the project root: scons

Mac

Not sure on the status of whether unsigned GDExtensions can be loaded into MacOS, but it builds, so here are the instructions anyways

!! You may need to set driver/mix_rate=48000 in the [audio] section of project.godot

In thirdparty/opus: cmake -Bbuild

In thirdparty/opus/build: make

In the project root: scons platform=macos arch=x86_64 or scons platform=macos arch=arm64

Web (STILL BROKEN ON GODOT 4)

Install emscripten (chocolatey can help with dependencies: choco install emscripten)

In thirdparty/opus: emcmake cmake -Bbuild

In thirdparty/opus/build: make

In the project root: scons platform=javascript

one-voip-godot-4's Issues

Add to AssetLib

If this project works it would be great to have it on the AssetLib so everyone can know it exists and use it.

AudioStreamVOIP doesn't output any sound

i have troubles using this librairy so i test in my project, all thing was working except the fact than AudioStreamVOIP doesn't output any sound, so i test some demo project thatuse your addons but in every project no audio was output,i'm on godot 4.2 i hope it's a real issue and not just me who is stupid, thanks for the help, here's the code and the node so you can see :

Feature request: End of stream sample, and buffer size query

The jitter buffer prints out the following errors it would be useful to expose programatically: "TOO MANY SAMPLES" and "NOT ENOUGH SAMPLES".
https://github.com/RevoluPowered/one-voip-godot-4/blob/main/src/spsc_jitter_buffer.cpp#L21

VOIP channels don't need to be constantly streaming between all players since we talk in bursts not usually at the same time.

The basic interface is a Push To Talk, like on a walkie-talkie, where the stream starts and stops with a pressed button. The microphone threshold feature simply automates pressing this PTT button for you. Therefore, there would be an End-Of-Stream packet which, if pushed into the jitter buffer, should suppress the "NOT ENOUGH SAMPLES" error and cause it to sleep until the stream resumes.

Also, if I've got a backlog of packets (or a recorded message from a player), I'd like to be able to push these samples into the jitter buffer in batches without triggering the "TOO MANY SAMPLES" error, so knowing the remaining capacity of the queue would help.

Often the PTT or threshold clips people's speech at the front end because they don't pause, so I imagine one would hold onto a short 0.5 second ring-buffer of samples that would be sent forward at the trigger point so as effectively back-date it. This would be another reason for overstuffing the buffer.

Weird crackles in audio

Hi,
I tried using your VOIP over P2P with Epic Online Services as a relay but the audio that is transferred to the other client has a lot of "crackles" if I can explain it that way.

Here is what it sounds like.
https://github.com/RevoluPowered/one-voip-godot-4/assets/116175704/391ad4b2-58fb-46c7-a51b-962c484d748a

Here is what my mic sounds like.
https://github.com/RevoluPowered/one-voip-godot-4/assets/116175704/000441e5-cb03-445f-89f6-79abdda25fc2

It may not be an issue related to the VOIP but I have no idea what is causing it so I thought I should ask.
Also, it may not be related but the console of godot get spammed with this message :

NOT ENOUGH SAMPLES - frames: 512 num_read: 0
NOT ENOUGH SAMPLES - frames: 512 num_read: 0
NOT ENOUGH SAMPLES - frames: 512 num_read: 0
NOT ENOUGH SAMPLES - frames: 512 num_read: 0

Maybe it's intentional, but I'd rather ask just in case.

macOS support?

Is that possible to add macOS support?

Android support?

I was wondering, what might the ETA on an android build for this be? Also just to be 100% clear, this would not build to Oculus for VR because it doesn't build for Android, correct? I think there would be a lot of utility in adding VR support for this library.

one-voip gdextension?

I've been trying and failing miserably to get a voip system that I'm happy with using Godot and GodotSteam. Both the old godot-voip plugin for Godot 3 (which I updated myself for Godot 4) and Steam Voice either would get increasingly stuttery with more players connected or just had stutters in general respectively. I came across this project and it seems really promising.

In the README and demo folder, there seems to be references to a one-voip gdextension plugin, but there are no releases nor extension that I can find. Is this project still active? If so, is there a gdextension plugin like I expected, or do I have to compile myself?

one-voip throws std::bad_cast on connect on ANY distro of linux

On trying to run a game with the extension, godot throws an std::bad_cast error (more details given later)

Input Threshold for reduced background noise?

I've tried using this plugin, and it works great! However, I noticed there is no way to set an input threshold, and by default too much background noise for my liking is caught.

I don't know how speex and opus would handle this, but I would assume there is some way in _sample_buf_to_packet to have an input threshold value checked against to stop sending packets (or send empty packets?) if the maximum value doesn't exceed the input threshold.

I am suggesting this because I am right at the end of finals and can't implement this myself, and it is not a feature that can be easily implemented on the GDScript side of things.

Bad sound with 3+ clients

Hey! I've been testing the plugin with my friends and seems like it's fine when 2 clients are talking
But when it comes to 3+ the voice pitch becomes very robotic and a lot of noise introduces
You can reproduce this with running 3 instances in godot

Do you know anything on this? Can I fix it somehow?
I've tried various bitrates and changing buffer length, no luck :(

PS. Fix is to set AudioStreamVOIP as local to scene #17

Not enough samples debug message

I actually made this awesome package work with my project. The only problem is my message output is getting spammed with these debug messages: "NOT ENOUGH SAMPLES - frames: 512 num_read: 0"
Any idea why that could be happening? The message is located in spsc_jitter_buffer.cpp but I do not understand the surrounding code correctly...

Allow jitter buffer to be implemented in GDScript

I think the current use of a jitter buffer obfuscates what should be a fairly transparent process that would allow for synchronicity with animations, events and visemes.

Capturing:
If you expose the VOIPInputCapture::_sample_buf_to_packet() function to GDScript then we can use a base AudioEffectCapture object and feed the VOIPInput object that wraps the Opus library like so:

while voipinputcapture.get_frames_available() >= 441:
    var samples = voipinputcapture.get_buffer(441)
    var opuspacket = voipinputcapture._sample_buf_to_packet(samples)
    transmit(opuspacket) # or packets.append(opuspacket)

For a bonus, _sample_buf_to_packet() could Repacketize multiple opus frames into a single opus packet when the length of samples is a multiple of 441. (By the way, 441 audio frames is 10ms of sound at 44100Hz, which is resampled up to 480 audio frames at 48000Hz required by the Opus library.)

The reason the current design of send_test_packets -> emit signal packet_ready is unsatisfying is that we lose track of which packet corresponds to which time window since it's going out to a different callback function instead of returning to the caller.

The output Opus stream stutters for the very simple reason that there is a mismatch between _process() frame rate (60fps) and audio encoding rate (100fps).

Playback

I exposed a copy of the push_packet() function that returned the uncompressed samples:
PackedVector2Array AudioStreamVOIP::spush_packet(const PackedByteArray& packet)

The GDScript code for managing and playing the incoming stream is:

setup:

var staticvoipaudiostream = ClassDB.instantiate("AudioStreamVOIP")
$AudioStreamPlayer.stream = ClassDB.instantiate("AudioStreamGenerator")
$AudioStreamPlayer.play()
var playbackthing = $AudioStreamPlayer.get_stream_playback()


processing:

opuspacketsbuffer = [ ... ]  # filled from the networkthat
while playbackthing.get_frames_available() > 441 and len(opuspacketsbuffer):
    var frames = staticvoipaudiostream.spush_packet(opuspacketsbuffer.pop(0))
    playbackthing.push_buffer(frames)

Ther docs warn that AudioStreamGeneratorPlayback.push_buffer() is slow in GDScript, so the implementation of AudioStreamVOIP is probably good, if we also have the a get_opus_frames_available() to tell us how many empty slots are left in the jitter buffer.

It would also be a good idea to know how many frames are left in the buffer so we don't get the "NOT ENOUGH SAMPLES - frames:" error. However I can't see a good function in the Godot libraries to base it on, other than float _get_playback_position().

Additionally, features like Forward Error Encoding (where it can fill in for missing opus packets) have to be managed outside of this library since the opus packets don't contain sequence numbers, so you have to tell the library when a packet is missing.

There's also a DTX (Discontinuous Transmission) feature that puts out 400ms long frames when there is silence. Otherwise the Opus library assumes that everyone is talking all the time everywhere like it's a video conference where the only purpose of being online is to talk. This is not how we play networked games, where there something other than just talking as an activity, and a VOX (Voice operated push to talk) system would be more appropriate, as well as being kinder on the bandwidth, and considerably more scalable.

Audio Stream VOIP doesn't instantiate playback

I have tried implementing this in my Godot project using (version 4.2 stable) but I can't get it to work. When I press play, I get an error saying that the Audio Stream VOIP must implement the "instantiate playback" method.

Here is the error:

E 0:00:05:0557   instantiate_playback: Method must be implemented!
  <C++ Error>    Method/function failed. Returning: Ref<AudioStreamPlayback>()
  <C++ Source>   servers/audio/audio_stream.cpp:187 @ instantiate_playback()
E 0:00:05:0557   play: Failed to instantiate playback.
  <C++ Error>    Condition "stream_playback.is_null()" is true.
  <C++ Source>   scene/3d/audio_stream_player_3d.cpp:580 @ play()

I'm very confused by this error since the method seems to be implemented at least from what I can tell. My guess is that I messed something up since I'm not very used to compiling C++ projects.