Can we please have support for writing plugins in something other than python? about decky-loader

[ILOVEPIE]

should probably be using some sort of remote procedure call protocol. I would do this over UNIX sockets simply because it’s going to be more secure.

…

On Mon, Jun 6, 2022 at 2:27 PM TrainDoctor ***@***.***> wrote: Reopened #78 <#78> . — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACB45SVU5QWRBTLK36JX5LVNZUNFANCNFSM5X562HEA> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[JohnnyCrazy]

I also think UNIX sockets is the way to go. Security aside, it's probably also the fastest and most low-overhead solution.

Regarding security, I'm not 100% sure how UNIX sockets work in detail, but a shared secret which is passed to the plugin server process via its environment variables could work? Would we even need to encrypt the traffic?

[ILOVEPIE]

Remote Procedure Call would allow methods to be added to the API easily and there are already standard cross-language Remote Procedure Call libraries for communicating over UNIX or TCP/IP sockets. Another advantage is that If a plugin wants to add their own API that other plugins could use a RPC system would easily allow that. One option is gRPC which was part of the ProtocolBuffers project.

[TrainDoctor]

Work has recently started on support for websockets. Currently python is still needed to start up a plugin's custom backend written in other languages but once websockets are fully implemented, python will no longer be a requirement for interacting with the Decky Loader's backend.

[ILOVEPIE]

I would say a Remote Procedure Call API over Unix Sockets is the best solution.

[FrogTheFrog]

Hello, I found myself being blocked by the current back-end limitations and for what it's worth I agree with @ILOVEPIE and vote for gRPC.

[TrainDoctor]

This is something that's actively planned. It's going to be even better than explicit support of a specific language. it's going to be a system where any language where you can send information through a TCP socket will be supported.
I'll be closing this issue as this is planned to be supported.

[TrainDoctor]

Re-opening this issue to help foster discussion about how inter-process communication could/will work and usage temp Unix sockets vs other types as two examples of good points brought up by @JohnnyCrazy.

[marios8543]

The loader already uses unix sockets under the hood, for IPC between the loader process and the isolated plugin processes, so in theory you only need to remove the python module run and add a subprocess to an external binary to do this. Most of the work regarding communication is already there

[marios8543]

What are the advantages over the current solution with a barebones unix socket ? We only really use it for one thing and that is native plugin methods

[Samuel-B-D]

Just throwing stuff in the air, but another completely different approach could be to allow loading of wasm plugins by integrating wasmtime runtime into decky-loader
https://bytecodealliance.org/articles/wasmtime-1-0-fast-safe-and-production-ready

[NGnius]

The front-end is a browser so it's already got WASM support baked in. The best course of action would be to provide some sort of API to break out of the browser sandbox (safely) if Decky wanted to go that route. Adding another WASM runtime seems unnecessary unless the CEF version is too out of date or something (it's not; USDPL uses it fine).

But to actually touch on the central discussion: Decky already has support for custom back-ends, with USDPL being the only one that I'm aware of that's used by plugins. It's not using any standard like RPC, and only supports Rust back-ends atm, but it has zero reliance on Decky for back-end activities (aside from actually starting the back-end). The problem is communicating from the front-end to the back-end, and that can be solved just as well by third-party stuff so I don't really think it's necessary for Decky to have it. Obviously I'm biased because I am the third-party in this, but I think leaving it up to the plugins devs is the best course of action when there's no single obvious solution.

[james2432]

I would advise against using gRPC, you need to use HTTP/2.0 and protobufs have massive overhead vs a tcp socket

[ILOVEPIE]

GRPC doesn’t have to be over HTTP you can do Protobufs over tcp. Heck valve uses Protobufs for the source engine networking code.

…

On Tue, Oct 4, 2022 at 12:36 AM James ***@***.***> wrote: I would advise against using gRPC, you need to use HTTP/2.0 and protobufs have massive overhead vs a tcp socket — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACB45SFQHODAZJE2GO74ATWBPNAVANCNFSM5X562HEA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ILOVEPIE]

And nobody said we were using TCP sockets were using UNIX sockets

…

On Tue, Oct 4, 2022 at 12:36 AM James ***@***.***> wrote: I would advise against using gRPC, you need to use HTTP/2.0 and protobufs have massive overhead vs a tcp socket — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACB45SFQHODAZJE2GO74ATWBPNAVANCNFSM5X562HEA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[james2432]

then couldn't a random path be passed to the plugin being started in the sysargs?

like /tmp/[some random string here such as a guid]

then the backend would be responsible for opening the unix socket and listening for event/some protocol between decky loader and plugins

[AAGaming00]

Work has recently started on support for websockets. Currently python is still needed to start up a plugin's custom backend written in other languages but once websockets are fully implemented, python will no longer be a requirement for interacting with the Decky Loader's backend.

Clarifying, WebSockets are for communicating between Decky itself and the Decky frontend (and plugin frontends.) This does however solve one of the main things blocking custom non-py backend communication, since it will allow for bidirectional frontend-backend communication.

[leroycep]

What about using a systemd service for backend services? Take the method used for running syncthing in game mode in this reddit post, and then have plugins communicate using HTTP requests or websockets.

[NGnius]

Decky is already a systemd service, it can spawn custom backends. Installing additional services just makes it more annoying to uninstall

[TrainDoctor]

This has been largely accomplished through the custom backend effort some time ago. Closing.