giongto35 / cloud-morph

Decentralize, Self-host Cloud Gaming/Application

Decentralized, Self-hosted Cloud Gaming service on WebBrowser.
Video Demo: https://youtu.be/eFR7fLy9z6E
Technical Document: https://github.com/giongto35/cloud-morph/wiki
Landing Page: https://cloudmorph.io
CloudRetro: https://github.com/giongto35/cloud-game: Sister project, Cloud Gaming Service For Retro Games

Introduction

CloudMorph is a decentralized, self-hosted cloud gaming/cloud application platform. Users can quickly spawn up a cloud gaming service with minimal configuration. By leveraging the ease of deployment, CloudMorph goal is to build a decentralized cloud application network with app providers and consumers.
CloudMorph copes with different technical challenges from OS to Cloud, such as Low-Latency Streaming, Video/Audio encoding pipeline and optimization, Windows application Virtualization in headless server, OS event simulation, NAT traversal, P2P network structurization, etc.
Unlike CloudRetro, a completed Cloud Gaming solution for Retro Game run on its dedicated cloud infrastructure, CloudMorph decentralized hosting works to users with any Windows Games/Applications by a generic and modularized solution.

Discord: Join Us

Changelog

• 26/9/2021: Able to run in Windows/ Mac
• 19/6/2021: Modularize Frontend as Widget. All parts are independent

Demo

Video Demo: https://www.youtube.com/watch?v=fkOpOQ-HwFY

Screenshot	Screenshot
Diablo II-US	Starcraft
RoadRash	Browse and Switch games

CloudMorph Demo

• Cloud Diablo SG (Demo of Collaborative play Diablo running on Singapore server using CloudMorph).
• Cloud Diablo US (Demo of Collaborative play Diablo running in US server). Switch applications using the sidebar on the left.

Usecases

For Consumers.

• Play any provided application directly from Browser without Installation.
• i.e, clouddiablo.com is for playing Diablo on Browser collaboratively. When we have less time to finish a game, let's do it together?

For Providers

• Playable Ads: Instead of Video Teaser, Game/Application providers can let users interact with the application.
• Discoverable: Any application that joined the network will be surfaced to users over CloudMorph channel.

For Developers

• Experience playing/hosting Cloud Gaming on their own.
• Plugable Cloud gaming module: The cloud gaming core is packaged and virtualized to be extendable to different tech stacks. E.g Python, Java ...

Getting Started

Windows (WIP)

Running in Sandbox (Recommended)

To setup Window Sandbox, Turn on Virtualization in Bios and Enable Windows Sandbox Feature. Tutorial https://techgenix.com/install-configure-and-use-windows-sandbox/

1. Using setup-sandbox.ps1 to download and install necessary packages (FFMPEG) in sandbox image (winvm/pkg)
2. go run server.go

Running without Sandbox

Without Sandbox, environment is not isolated, so mouse + keyboard simulation will target your main mouse + keyboard. If you play on the same machine, so you will experience your mouse is moving away.

1. Install globally dependencies: FFMPEG. Doesn't need to put in sandbox folder as 1
2. go run server.go

Mac/Ubuntu

Running locally

1. Install Dependecies: Docker/Go. Or just setup.sh
2. go run server.go

Running remotely

• Run setup_remote.sh 111.111.111.111 inside ./script, 111.111.111.111 is the address of your host. What you will get your application hosted on your remote machine. More details are in Deployment section below.

Debug in case you need

• If your run is succesful, there will be a Docker running in background. Inside Docker there are 5 apps is running and their logs are suffix with _err _out

Design

Goal:

1. Cloud gaming Philosophy: The application run in the remote cloud instance. Video/Audio is streamed to users in the most optimal way. User interaction needs to have the lowest latency.
2. Cross-platform compatibility: The service is accessible in web-browser, the universal built-in that can fit multiple platforms like Desktop/Mobile. No console, plugin, external app, or devices are needed.
3. Deployment Simplicity: There is no API/ interface integration required to set up the integration. One line command to finish the deployment.
4. Mesh network: Providers-Consumers over Peer To Peer communication. After joining the network, Provider's Application is discoverable and immediately launched with one selection.
5. Modularizable: A concise technical stack to develop/deploy for cloud gaming/ cloud application service.
6. Scalable: Able to scale on headless machines cluster horizontally.

CloudApp Core

1. When a Web Service starts, Application Container, named "CloudApp Core", is spawned. Inside the Container there are Application + Virtual Display/Audio + Windows Event Simulation Utility. Multiple Containers can be spawned on demand.
2. Web Service is in charge of Webapp interface, P2P communication. When a user/client connects the service, the service will initialize a WebRTC connection between the client and service. This project uses WebRTC Pion, which is a fantastic library for WebRTC in Golang.
3. Input captured from Client is sent to Web Service using WebRTC Data Channel (UDP).
4. Web Service sends received input events to Virtual Machine over a socket.
5. The utility (syncinput.exe) listens to the input events and simulates equivalent Windows OS events to Wine Application through WinAPI.
6. Application screen/ Audio is captured in a Virtual Display Frame Buffer (XVFB)/ Virtual Audio (PulseAudio), which is later piped to FFMPEG.
7. FFMPEG encodes the Video Stream to RTP (VPX/H264) stream and Audio Stream to Opus stream.
8. Overall, "CloudApp Core" module receives Input as WebSocket event and Output as RTP stream. It is packaged in Container with the interface declared at core/go/cloudapp.

Decentralize

• Discovery service is a centralized service, backed by etcd. In this flow, Client periodically query this service to get list of joinable host and show them in sidebar.
• For Provider, if the configuration in config.yaml includes discoveryHost attribute, application will be discoverable to user.

Detailed Technology

wiki

Development

The service is built using Golang, C++, and Linux X11 utility tools (Xvfb, ffmpeg). You can set up all dependencies with setup.sh. After that, you can run the go server with

• go run server.go

Access to your local at

• localhost:8080

Note: the wine application runs inside Docker. You can run it without docker by changing run-wine.sh to run-wine-nodocker.sh in server.go for easier debugging.

Explore and Contribute

• Wiki Deep Dive Into Codebase

Deployment

First, we need an Ubuntu instance with a public network firewall (No firewall rule for P2P communication NAT traversal). You can get a VPS from any provider (like AWS, DigitalOcean) We prepare below in the same directory

Required:

1. config.yaml: app config, the app configuration.
2. apps: the folder contains the app we will deploy. It is later mapped to winvn/apps in the remote instances. For example, DiabloII. If your application is already inside the synced wine environment at 2, ex "Program Files", we can skip it. We need to configure config.yaml, apppath to point to the correct app path.
3. setup_remote.sh: a copy of the script in winvm/script to deploy your application to server.

Optional:
4. wine: whole wine folder from .wine. If there is no wine folder, the deployment uses the default .wine from installation.

After that, we run setup_remote from inside the folder:

• setup_remote.sh $ip. Ex: ./setup_remote.sh 159.89.146.77
• Tutorial Video: https://www.youtube.com/watch?v=w8uCkfZdHVc

Deployment Example

• script/example contains example applications configuration. Note: /apps is left empty due to copyright.

Deployment with Lutris

• Lutris eases the installation of a game on Linux. The recommended flow is to install game with Lutris and copy produced wine environment in .wine folder to Cloud Morph.

Road Map - Request for Help

• UI improvement
• Full Dockerization. Only core is packaged, server is not run in Container.
• Port C++ Window API to Rust.
• GPU acceleration. - Integrate with FFMPEG job.
• Multiplex application sessions. Currently, only collaborative mode is supported, which serves all application sessions from the same single instance.
• Performance optimization.
• Web Mobile controller support. Only mouse click is simulated.

cloud-morph's Issues

Audio support?

Maybe I missed this but is there currently support for audio? The diablo demo has no sound. I'm using the Brave browser.

Running without Sandbox failed

PS E:\test\cloud-morph> go run server.go
go: github.com/gofrs/[email protected]+incompatible: Get "https://proxy.golang.org/github.com/gofrs/uuid/@v/v4.2.0+incompatible.mod": dial tcp 108.177.97.141:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
go: downloading github.com/gorilla/mux v1.8.0
go: downloading github.com/gorilla/websocket v1.5.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/gofrs/uuid v4.2.0+incompatible
go: downloading github.com/pion/rtp v1.7.13
go: downloading github.com/pion/interceptor v0.1.11
go: downloading github.com/pion/webrtc/v3 v3.1.41
go: github.com/gofrs/[email protected]+incompatible: Get "https://proxy.golang.org/github.com/gofrs/uuid/@v/v4.2.0+incompatible.mod": dial tcp 108.177.97.141:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

This is MASSIVELY exciting, if there is anything I can do tell me

Just saw this from cloud-game README, @giongto35 this is so cool!

I have a bunch of games I bought from GoG that work great on Linux, but I can't play locally. I have gigabit internet so this is going to be something I use all the time.

Really excited to see where this goes.

Road Rash Demo Freezes at 'Save Game As...'

The Road Rash demo froze when trying to use the 'Save Game As...' menu item from the Restroom. This feature opens a Windows save dialog box to save a game file.

When selecting 'Save Game As...', the save dialog opened correctly. When trying to interact with the dialog to save a file, the dialog disappeared from the screen and the focus was changed back to the game menu. the screen was frozen on the game Restroom menu and no user input was accepted.

Official page game loading forever

I see multiple dots moving in circles.

dosbox_pure_libretro.so keyboard mapping

my end goal is to get dosbox games running in the same way that the other cores work.

I successfully installed dosbox_pure_libretro.so as a new core and added a game to the games folder
adjusted config.yaml, started up the server and worker and got a game running.

however it seems the keyboard/controller does not map to what the dosbox core expects.

imho I would expect that web/js/input/input.js and web/js/input/keyboard.js have a role to play in the client side of the controller support.

no clue about the server side though.

Display the real-time latency

I wonder if the latency from the web client to the server could be displayed during the play, since it might be a useful indication for users who are suffering a big lag and wondering why.

execing run-client.sh panic: exit status 126

I am deploying this project on Ubuntu18.04 and I've done steps one to four. But when I run "go run server.go", there are some problems:

2020/10/22 15:39:38 Starting monitoring server at :3535
2020/10/22 15:39:38 Profiling is enabled at :3535/debug/pprof
2020/10/22 15:39:38 Spawn server
2020/10/22 15:39:38 listening wine at port 9090
2020/10/22 15:39:38 execing run-client.sh
panic: exit status 126

goroutine 1 [running]:
github.com/giongto35/cloud-morph/pkg/core/go/cloudgame.(*ccImpl).launchGameVM(0xc00007f9e0, 0x138c, 0xc000024c80, 0x17, 0xc00002a980, 0xd, 0xc00002a9b0, 0x9, 0x406900, 0x140)
cloud-morph/pkg/core/go/cloudgame/cloudgame.go:141 +0x340
github.com/giongto35/cloud-morph/pkg/core/go/cloudgame.NewCloudGameClient(0xc000024c80, 0x17, 0xc00002a980, 0xd, 0xc00002a9b0, 0x9, 0x0, 0xc00002aa10, 0xd, 0xc00002a9e0, ...)
cloud-morph/pkg/core/go/cloudgame/cloudgame.go:75 +0x1db
github.com/giongto35/cloud-morph/pkg/core/go/cloudgame.NewCloudService(0xc000024c80, 0x17, 0xc00002a980, 0xd, 0xc00002a9b0, 0x9, 0x0, 0xc00002aa10, 0xd, 0xc00002a9e0, ...)
cloud-morph/pkg/core/go/cloudgame/service.go:304 +0x1ac
main.NewServer(0xf490c0)
cloud-morph/server.go:230 +0x44e
main.main()
cloud-morph/server.go:296 +0xb8
exit status 2

Will be there a way to use native linux applications

Hello,

At first thanks for this project, I was looking for some cloud gaming solution but made with opensource. It's more a question than a real issue but do you think it would be possible to use the same process for running a native linux application/game instead of Windows applications with Wine?

Cheers.

Typo in wiki: Wine is not a emulator

Please fix mention about Wine. Wine is not a emulator (even acronym in the name mentions it)

Difficulties encountered when running in Windows（without sandbox）

I run this project in Windows 10 (without sandbox),and runs in Google Chrome .The picture in the web page is not right, and the mouse position is also not synchronized.

here is the config.yaml：

Unity games support

Hi,
We made a Windows game with Unity. When deployed to cloud morph, the game starts but the input does not work (the game locally works with mouse and keyboard and with gamepad).
We enabled some logs in the game and we also made a simpler game only to test the inputs, but we get a black screen.

Any suggestion? Are Unity games supported/Are there some tips when building the game (i.e. x86 vs x64, supported graphics APIs, etc.)? Thanks in advance.

panic: listen udp :5004: bind: address already in use

Hi there. I am starting to test this. Use Ubuntu 20.04 on Windows. I did some pre-installations to get it working, as always. The issue is that when it is running, it reaches a point where it shows me a Firewall window, but the whole process stops, This is the message.

2021/08/27 00:35:07 Launched application VM
2021/08/27 00:35:07 Setup Video Listener
panic: listen udp :5004: bind: address already in use

goroutine 1 [running]:
github.com/giongto35/cloud-morph/pkg/core/go/cloudapp.(*ccImpl).newLocalStreamListener(0xc0001044b0, 0x138c, 0x1, 0xc0000ec860)
        /home/evan/cloud-morph/pkg/core/go/cloudapp/cloudapp.go:232 +0x26b
github.com/giongto35/cloud-morph/pkg/core/go/cloudapp.NewCloudAppClient(0xc0000ec860, 0x6, 0xc0000ec870, 0xb, 0xc0000ec890, 0x7, 0x0, 0xc0000ec8c0, 0xd, 0xc0000ec8a7, ...)
        /home/evan/cloud-morph/pkg/core/go/cloudapp/cloudapp.go:91 +0x375
github.com/giongto35/cloud-morph/pkg/core/go/cloudapp.NewCloudService(0xc0000ec860, 0x6, 0xc0000ec870, 0xb, 0xc0000ec890, 0x7, 0x0, 0xc0000ec8c0, 0xd, 0xc0000ec8a7, ...)
        /home/evan/cloud-morph/pkg/core/go/cloudapp/service.go:276 +0xf7
github.com/giongto35/cloud-morph/pkg/core/go/cloudapp.NewServerWithHTTPServerMux(0xc0000ec860, 0x6, 0xc0000ec870, 0xb, 0xc0000ec890, 0x7, 0x0, 0xc0000ec8c0, 0xd, 0xc0000ec8a7, ...)
        /home/evan/cloud-morph/pkg/core/go/cloudapp/server.go:71 +0x26c
main.NewServer(0xfcaee0)
        /home/evan/cloud-morph/server.go:211 +0x4fd
main.main()
        /home/evan/cloud-morph/server.go:314 +0xb8
exit status 2

Sick

Hey, just want to say: amazing work! keep up the great work! 💯

Webrtc video/audio/text chat from client device somehow?

I know this is a stretch for this project.

I have some projects that I use webrtc video and audio chat in the experience. Is it possible to cloud stream the game/experience but still have access to user hardware for webrtc video and audio chat?

Just curious and no pressure. Thanks!

[Discussion/Question] Non-cloud variant and easier customisation

So this might be a bit off topic, but still I would like to discuss this topic:

My idea would be to use this project as an In-home-streaming solution (just like Nvidia GameStream or Steam Link etc.).
While this is of course to some degree already possible, there are still some scenarios that are not covered natively, for example:

1. A PC-Setup would not be headless.
2. Non-Docker Setup:
   • either just native on the computer
   • or alternatives like LXD could be used
3. No need to copy things around -> just use the local game installations (assuming it's on Linux of course).
4. (optional idea) Control the desktop (instead of just one application).

While I am sure that users who are programmers themselves, could change the source code to adjust it for other purposes, it is quite difficult to understand everything for other people.

So the potential solution would be to have easier configurable settings/structures etc., that would allow the users to setup the above mentioned scenarios more easily.

I can of course understand if you say that this is not the scope of this project, but I wanted to mention it, as I think it would be useful for many people.

Thx 🙂 .

Keyboard doesn't work for some games

Happens with VirtualCop, Samurai Showndown ...

Domain down

Hmm?

Is there already gpu support?

Can it be compiled on mac os?

Sorry to disturb, I'm the rookie in cloud gaming/streaming media programming. I'm using Mac OS, but it doesn't work by your step. Here is the result.

It stacks in "2020/09/18 17:15:50 Launched application VM", Then has no outputs. I checked docker logs, here is the result.

Can you give me some advice? I'm newer in this field, but I want to study.

[Feature Request] Controller support

See here for example: https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API/Using_the_Gamepad_API

Get "http://178.128.125.200:7700/get-apps": EOF

Get "http://178.128.125.200:7700/get-apps": EOF

Failed to run on Windows

I follow by the step to run on windows. But go run server.go will stop at the log Setup Video Listener.

I found that the powershell command powershell -ExecutionPolicy Bypass -F run-sandbox.ps1 apps/spider sol.exe spider 800 600 -w windows is not working. This powershell command has no log showed and no SandBox window

How does v-GPU work？

It supports v-gpu？if not，how does it support multi-doker if many user use one server to play？
I am thinking about whether i can use nvdia‘s v-gpu to support my server.

No work

It has been showing a black screen on DII since today, is it normal?
I was able to run around a bit with a necromancer and then, after the page reload, it all went black. It was possible to play if multiple tabs were open, in which case the first one would stream the game.

Nil server

Found a small bug with clouddiablo: it returns some nil server atm.

Is it possible to run server on windows natively?

My OS : Windows 10 1909 x64