[BUG] Audio degradation after kasmvnc base about docker-webtop

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[j0nnymoe]

What browser are you using?

[BlueCobbleStone]

What browser are you using?
i'm using brave

[thelamer]

In general Chromium based browsers should just work. A lot depends on the connection quality, host, and client specs.

What we do for audio is different than most implementations of remote audio as we are making the assumption of lower latency and larger beefier hosts that are not in a shared environment.

The default and only audio option is 32 bit float PCM which is sending the raw PCM data from Pulseaudio at about 3 megabit when their is actual data to send. This is lossless crystal clear audio when it works, but there is also no room for extreme latency, dropped packets, or host/client load getting in the way.

Firefox in particular seems to have a problem on the server side when it goes from zero audio to playing where it ships a huge chunk of audio data for the initial second of the playback as it seems to fall behind a bit from silence while it is still playing. This translates to a pop and crackle client side as we are simply processing the data as soon as it comes in. The same thing can happen with other programs like VLC, but again should be limited to the start of playback or when returning from complete silence.
It is a delicate balance that is meant to prevent a series of issues that arise from using transcoded audio streams including:

• Sync issues- Audio and video will get out of sync after some time when a session is being used as the audio track never stops and can be not processed properly clientside or load from the video rendering can get in the way.
• No Null Data- When transcoding to a web native audio format like aac/mp3/etc the stream is constant when audio is turned on, there is no way to detect for null packets and the bitrate will be consistent.
• Browser compatibility- This really boils down to Safari from a support standpoint but playing back an audio track in Safari has been problematic use web native formats. It seems to layer on it's own buffering when it detects that it is needed which in turn translates to out of sync audio for the client.

In general the audio is not going to be perfect for everyone, but if you use a chromium based browser clientside and don't use firefox in the container you will have a much better experience.
The code itself is dead simple:

Pull from pulseaudio socket: https://github.com/linuxserver/kclient/blob/master/index.js#L154-L166
Play PCM audio when we get a chunk from the server: https://github.com/linuxserver/kclient/blob/master/public/js/pcm-player.js

If this does not work for you, using these images in the Kasm Workspaces application (which we have a container for https://github.com/linuxserver/docker-kasm ) Uses the audio system that I described earlier by transcoding the audio into a transport stream with ffmpeg and shipping that to the browser. In general this will be more reliable for extreme remote scenarios but will run into the other points I described above.

In the end the decision to use this lossless high bandwidth audio format was made because these images are geared for self hosting ignoring sane bandwidth and latency limitations you might run into in a "production" deployment.

[BlueCobbleStone]

Thanks for all that information.
I forgot to precise that I'm running the container on my local network (same computer).
I tested running chromium in the container with alpine-kde image and I still have worse audio quality than before kasm-base switch.
Sorry but i don't have any other evidence than "I can hear it" when playing youtube videos, music etc ... It seems obvious to me but it's maybe not.

[thelamer]

I am absolutely obsessed with fidelity (thus the 32 bit float audio and lossless desktop compression) This is a 24 bit flac and it sounds exactly like it was playing natively because that is essentially what we are doing with the PCM forwarding we are doing in the browser: (now keep in mind this has been re-encoded to aac for the mp4)

audio.mp4

So if you are not talking about pops and crackles than I am afraid we are not on the same page here. This implementation absolutely mops the floor with the old images.

[BlueCobbleStone]

I'm not talking about crackles It feels like the sound has been compressed to a lower quality.
If I Listen the music you'r playing (btw nice song :) on your video and on youtube "https://www.youtube.com/watch?v=To86UdOx0A8" It doesn't feel the same.

[thelamer]

Bits are bits, and I am simply pushing the bits for playback into the web browsers audio processing interface particularly at the absolute maximum quality possible 32 bit float pcm https://www.sounddevices.com/32-bit-float-files-explained/

In any case I hope I have explained the audio subsystem sufficiently, I am afraid there is nothing I can do to make this sound any better as everything is already turned up to 11.

[BlueCobbleStone]

I think I have a clear example,
You can install the following packages apk add alsa-utils alsa-plugins-pulse
Then, open the terminal in the container and use this command (Be careful very loud sound) : speaker-test
It sounds completely different

Edit: Idk maybe the pulseaudio configuration is the problem

[thelamer]

@BlueCobbleStone I think I have this nipped in the bud now, it turns out I messed up the bitshift in the PCM player and was pumping the left channel to both left and right, it has been fixed in the base I am building Ubuntu kde now.