juice-labs / juice-labs

Juice Community Version Public Release

Join our Discord Server for ideas, questions, and feedback

See Welcome to Juice GPU-over-IP

We're offering up our Community Version here for your use, which is governed by these Terms and Conditions.

What is Juice?

Juice is GPU-over-IP: a software application that routes GPU workloads over standard networking, creating a client-server model where virtual remote GPU capacity is provided from Server machines that have physical GPUs (GPU Hosts) to Client machines that are running GPU-hungry applications (Application Hosts). A single GPU Host can service an arbitrary number of Application Hosts.

Client applications are unaware that the physical GPU is remote, and physical GPUs are unaware that the workloads they are servicing are remote -- therefore no modifications are necessary to applications or hardware.

Why Juice?

GPU capacity is increasingly critical to major trends in computing, but its use is hampered by a major limitation: a GPU-hungry application can only run in the same physical machine as the GPU itself. This limitation causes extreme local-resourcing problems -- there's either not enough (or none, depending on the size and power needs of the device), or GPU capacity sits idle and wasted (utilization is broadly estimated at below 15%).

By abstracting application hosts from physical GPUs, Juice decouples GPU-consuming clients from GPU-providing servers:

1. Any client workload can access GPU from anywhere, creating new capabilities
2. GPU capacity is pooled and shared across wide areas -- GPU hardware scales independently of other computing resources
3. GPU utilization is driven much higher, and stranded capacity is rescued, by dynamically adding multiple clients to the same GPU based on resource needs and availability -- i.e. more workloads are served with the same GPU hardware

---

Please go to Welcome to Juice GPU-over-IP for the full picture.

juice-labs's Issues

Error setting certificate file

Helo,

When running juice I get the following error. I am running on Rocky Linux 8. I am not sudo (nor should the program require it).

2024-02-08 14:59:28.774 client.cpp:519 1cbbf4 I] Build: 2023.08.10-2103.0633b794                                                                             
2024-02-08 14:59:28.800 client.cpp:834 1cbbf4 I] Requesting session from 76.146.231.104:43210                                                                
2024-02-08 14:59:28.872 client.cpp:848 1cbbf4 F] Unable to connect to server, error setting certificate file: /etc/ssl/certs/ca-certificates.crt 

In addition, it would be nice if Juice could be clear when writing to stdout about which messages come from it, by having something like a [JUICE] prefix. I thought for a while that this error was coming from the software juice was calling.

I should mention that the agent shows Warning: TLS is disabled, data will be unencrypted.

Combine local and remote GPU

How can I combine local and remote GPU? Is it even supported?

I tried running server on local and remote and adding their IPs into the config. But nvidia-smi shows just one gpu. If I run agent only on remote, it also shows only the remote gpu and hides the local one.

Blender crashes rendering default cube

c:\Work\JuiceServer\agent>node dist\bin\main.js --launcher ..\Renderer_Win.exe --port 43210
Juice Agent, version undefined
Copyright 2021-2022 Juice Technologies, Inc.
Hostname: win10vm
Controller: undefined
listening on :::43210
info: GPU UDP update broadcast disabled.
::ffff:192.168.0.10 - - [26/Dec/2022:02:49:33 +0000] "GET /connect HTTP/1.1" 200 58 "-" "-"
error: cae6b7be-e3e5-4c1a-8ac1-5036cfeaf429 exited signal null
::ffff:192.168.0.10 - - [26/Dec/2022:02:51:11 +0000] "GET /connect HTTP/1.1" 200 58 "-" "-"
error: 43f6878f-68c0-40a0-aa76-9e089594060c exited signal null
::ffff:192.168.0.10 - - [26/Dec/2022:02:59:48 +0000] "GET /connect HTTP/1.1" 200 58 "-" "-"
error: e59cf94b-1aed-4713-8fec-37f6077fc772 exited signal null

Using version 2022-12-23-2154-7c6e2773 on both client and server.

Server: Windows 10, 3060 ti
Client: Ubuntu 22

Reproduction: start blender, choose CUDA for the gpu acceleration, hit F12 to start a render of the default cube in cycles. Mine crashes around iteration 1744.

juice.log is empty - you're buffering the output, so it's lost in a crash. I'll make a separate bug for that.

Any particular instructions to pool multiple gpu's ?

Hi,
As a home user, just to test out some inference and large models, i have 2 gpu's lying around, but in different form factors (one in desktop, one in a laptop). Is there a way for this driver abstraction to use both of these gpu's?

Non-nvidia GPUs

Are there any plans to support non-Nvidia GPUs? Like AMD RDNA/CDNA-based ones, or Intel Arc, oneAPI? Also, are there any plans to support GPU virtualization via DPUs (e.g., Nvidia Bluefield or Pensando)?

On a separate note, is the project still actively developed, or is it in a "maintenance-only" mode at this point?

RDMA (RocE) support

Wouldn't that fit perfect into your project?

Probably not much effort to implement support of communication via verbs beside TCP/IP.

Cus, RDMA capable hardware is so cheap nowadays.

Terra Invicta doesn't appear to be caching

Seeing 0.00% in the Juice HUD for caching performance. Grabbing Terra Invicta to confirm and resolve.

MultiGPU Issues

I only have 2 GPUs, but 4 are listed. I cannot get this system to work with multiple GPUs, with a single GPU it works fine, but with multiple GPUs, the program I'm using doesn't work. It works fine when I use one GPU with Juice, as well as working fine with 2GPUs and not using Juice

The program is automatic1111.

the error I get is Traceback (most recent call last): File "C:\ai_tools\stable-diffusion-webui-1\launch.py", line 39, in <module> main() File "C:\ai_tools\stable-diffusion-webui-1\launch.py", line 30, in main prepare_environment() File "C:\ai_tools\stable-diffusion-webui-1\modules\launch_utils.py", line 314, in prepare_environment raise RuntimeError( RuntimeError: Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check Press any key to continue . . .

juice.log is opened with buffering on linux, all logs lost in crashes.

You can trivially check that it's buffered - when a juiced program is running juice.log is 0 bytes, as soon as it exits the logs are written out.

You need to explicitly fflush() or use setvbuf() to disable buffering on the output handle.

Licence warning

Hi there,

I see that the entire repository is using MIT Licence which is a copyleft licence, but in the wiki you mentions in ToS that commercial usage is prohibited.

Well both doesnt work together.

NB: MIT Licence allow private/modification/commercial and closed source product usages.

I am not sure which one got priority licence in repository or your custom ToS but they must be aligned together and should not introduce conflicts.

My 2 cents.

Edit: removed copyleft licence.

Slow performance over gigabit lan

My GPU gets a GeekBench 6 score of 88,506, but when I test it over a gigabit lan with juice server/client I only get a score of 12,238. Isn't it supposed to get closer to the raw hardware score?

How to launch minecraft? (or: allow forking processes to work with the client)

Minecraft typically launches the "launcher". There you can select a version and click on a button "play".
I assume in the background it's doing something like "spawn child process".
Because at that point juicify stops working. So it correctly handled the "launcher", but when the launcher spawned a sub-process, it would not juicify these (based on the fps-stats I assume it doesn't).

Is it possible to add support for this? Ie: keep juicify running as long as at least one child-process is still running?

I think this will also help with Steam or Epic games as they also launch a "forked" process upon clicking "Play".

Thanks

Messages banner visual standard

Follow Prototype partner proposed for the UI

Current:

Expected:

Components:

Prototype link:

https://www.figma.com/file/Bf0iqp7KFDkjjMrLM4hv9I/Trial-Tier-Application---New-UI-Prototype-(Juice)?type=design&node-id=2510%3A8392&mode=design&t=CSpVFROj64xo0gVK-1

Mobile and Mac support

When will this be available?

License

So I read the license but I'm still a bit confused because to me there are two ways to take it.

1. Don't sell the ability to provide remote gpu's

Or/and

2. Don't use compute provided by a juice gpu for anything that will make money.

---

So use case 1:
I install juice on my gpu machines and then use applications which use those gpus to create some content that I eventually make money on directly or as result of displaying said content.

Use case 2:
I use juice to route gpu compute from servers in paying for to a user facing web application that charges in a per use fashion.

And 3.
The same as 2 except it's subscription based and there is a free teir.

Which of these use cases are okay?

FFmpeg nvidia encoder does not work through Juicify

There are a number of video tools that use Ffmpeg for input and output, and I'm having troubles with one of them: Topaz Video AI. I raised the issue on their forum (in this thread: https://community.topazlabs.com/t/running-video-ai-in-a-virtual-machine/53387/2 ) and I was asked to test if Ffmpeg could work as a standalone tool.

Indeed it doesn't fare so well. When the hardware encoder on the GPU is engaged, ffmpeg through juicify fails. The message being issued is typically:

[av1_nvenc @ 0000025ed351c340] Cannot load cuGetErrorString
exit status 0xc0000005
And logging the output of Juicify at level Trace doesn't add anything I can see of value.
Topaz Video AI spits out the log file attached here:
logsForSupport.zip

Is there a way that this can be fixed? It's hard to ask every video tool maker that uses Ffmpeg to step around this issue.

error connecting

hi, i tryed to run client and server on localhost, but it's not work because agent.exe triing to connect to 192.168.1.255, why?
process agent.exe don't listen 43210 port, only Renderer_Win.exe listen 43210 port, very strange...

Renderer_Win.exe results:

when i tryed to make ./juicify nvidia-smi:

MacOS support [Request]

What about supporting mac os so we can accelerate mac os virtual machines that are hosted on windows?

Make client connectivity issues more visible

Krieger writes: It would be nice if juicify.exe could attempt to hit the server and print a message saying it's network unreachable. I thought it was broken, turns out my vpn had disconnected so i couldn't reach the juice server.

Renderer_Win: release of code?

Hi,

I was looking into this project and although it seems promising, it is missing AMD support (see also ticket #92).
As such I started to dig into the code to see how I could add this feature, but it looks like this source tree has only the communication protocols between the server & client. (I'm no hero in go, so I might be wrong)

Is it possible to release this as well so I could at the very least try and get it to work on my Radeon card?

Thanks!

[Feature request] Client driver for paravirtualization

A suggestion for installing a server GPU simulator on the client as a client driver, so that the client thinks that it has the requested GPU locally. This should eliminate the need for juicify prefix for any command in that specific scenario, as the GPU would look like it is installed on the client side, provded that the same GPU is never to be actually installed on the client side so there will be no driver clash.

where are the c lib come from

do not find the source code of the c lib. for example libjuicejuda.so.

Project dead?

Hi,

To make it quick: the latest release was in August 2023 and the last commit is from October 2023.
Is this project dead or still being worked on?
Thanks