petrasrec / ws-scrcpy

Web client prototype for scrcpy.

Forked version of https://github.com/NetrisTV/ws-scrcpy

[FORKED] Changes

Only one emulator per server

The change is oriented to kubernetes setup, when one emulator will have a scrcpy server assigned to it as a side container, I have modified the code to make the whole UI be oriented for one emulator control.

New UI design for better UX

Refactored whole UI to look more professional, new screen when scrcpy can't connect to the emulator via adb.

Added support for user email

Tracking active connections to the scrcpy server. Fetching email from x-goog-authenticated-user-email header. If this header is not found then email will be assigned as localhost.

Exporting prometheus metrics

Added metrics for:

• web_socket_active_connections
• input_bytes
• input_frames
• dropped_frames
• decoded_frames
• websocket_latency

All metrics have labels based on player name and user email that is using this instance.

Minor changes

• Moved all video player settings to a config.
• Added navbar at the top.
• Emulator screen is rendered on the same main page instantly.
• Default player is preselected and a new dropdown added if user wants to change the player.
• Added browser-sync integration. For hot reload development, run npm run dev and wait for the browsersync tab to open (localhost:3001).

Original readme

Web client for Genymobile/scrcpy and more.

Requirements

Browser must support the following technologies:

• WebSockets
• Media Source Extensions and h264 decoding;
• WebWorkers
• WebAssembly

Server:

• Node.js v10+
• node-gyp (installation)
• adb executable must be available in the PATH environment variable

Device:

• Android 5.0+ (API 21+)
• Enabled adb debugging
• On some devices, you also need to enable an additional option to control it using keyboard and mouse.

Build and Start

Make sure you have installed node.js, node-gyp and build tools

git clone https://github.com/NetrisTV/ws-scrcpy.git
cd ws-scrcpy

## For stable version find latest tag and switch to it:
# git tag -l
# git checkout vX.Y.Z

npm install
npm start

Supported features

Android

Screen casting

The modified version of Genymobile/scrcpy used to stream H264-video, which then decoded by one of included decoders:

Mse Player

Based on xevokk/h264-converter. HTML5 Video.
Requires Media Source API and video/mp4; codecs="avc1.42E01E" support. Creates mp4 containers from NALU, received from a device, then feeds them to MediaSource. In theory, it can use hardware acceleration.

Broadway Player

Based on mbebenita/Broadway and 131/h264-live-player.
Software video-decoder compiled into wasm-module. Requires WebAssembly and preferably WebGL support.

TinyH264 Player

Based on udevbe/tinyh264.
Software video-decoder compiled into wasm-module. A slightly updated version of mbebenita/Broadway. Requires WebAssembly, WebWorkers, WebGL support.

WebCodecs Player

Decoding is done by browser built-in (software/hardware) media decoder. Requires WebCodecs support. At the moment, available only in Chromium and derivatives.

Remote control

• Touch events (including multi-touch)
• Multi-touch emulation: CTRL to start with center at the center of the screen, SHIFT + CTRL to start with center at the current point
• Mouse wheel and touchpad vertical/horizontal scrolling
• Capturing keyboard events
• Injecting text (ASCII only)
• Copy to/from device clipboard
• Device "rotation"

File push

Drag & drop an APK file to push it to the /data/local/tmp directory. You can install it manually from the included xtermjs/xterm.js terminal emulator (see below).

Remote shell

Control your device from adb shell in your browser.

Debug WebPages/WebView

/docs/Devtools.md

File listing

• List files
• Upload files by drag & drop
• Download files

iOS

Experimental Feature: is not built by default (see custom build)

Screen Casting

Requires ws-qvh available in PATH.

MJPEG Server

Enable USE_WDA_MJPEG_SERVER in the build configuration file (see custom build).

Alternative way to stream screen content. It does not require additional software as ws-qvh, but may require more resources as each frame encoded as jpeg image.

Remote control

To control device we use appium/WebDriverAgent. Functionality limited to:

• Simple touch
• Scroll
• Home button click

Make sure you did properly setup WebDriverAgent. WebDriverAgent project is located under node_modules/appium-webdriveragent/.

You might want to enable AssistiveTouch on your device: Settings/General/Accessibility.

Custom Build

You can customize project before build by overriding the default configuration in build.config.override.json:

• INCLUDE_APPL - include code for iOS device tracking and control
• INCLUDE_GOOG - include code for Android device tracking and control
• INCLUDE_ADB_SHELL - remote shell for android devices (xtermjs/xterm.js, Tyriar/node-pty)
• INCLUDE_DEV_TOOLS - dev tools for web pages and web views on android devices
• INCLUDE_FILE_LISTING - minimalistic file management
• USE_BROADWAY - include Broadway Player
• USE_H264_CONVERTER - include Mse Player
• USE_TINY_H264 - include TinyH264 Player
• USE_WEBCODECS - include WebCodecs Player
• USE_WDA_MJPEG_SERVER - configure WebDriverAgent to start MJPEG server
• USE_QVH_SERVER - include support for ws-qvh
• SCRCPY_LISTENS_ON_ALL_INTERFACES - WebSocket server in scrcpy-server.jar will listen for connections on all available interfaces. When true, it allows connecting to device directly from a browser. Otherwise, the connection must be established over adb.

Run configuration

You can specify a path to a configuration file in WS_SCRCPY_CONFIG environment variable.

Configuration file format: Configuration.d.ts.

Configuration file example: config.example.yaml.

Known issues

• The server on the Android Emulator listens on the internal interface and not available from the outside. Select proxy over adb from the interfaces list.
• TinyH264Player may fail to start, try to reload the page.
• MsePlayer reports too many dropped frames in quality statistics: needs further investigation.
• On Safari file upload does not show progress (it works in one piece).

Security warning

Be advised and keep in mind:

• There is no encryption between browser and node.js server (you can configure HTTPS).
• There is no encryption between browser and WebSocket server on android device.
• There is no authorization on any level.
• The modified version of scrcpy with integrated WebSocket server is listening for connections on all network interfaces (see custom build).
• The modified version of scrcpy will keep running after the last client disconnected.

Related projects

• Genymobile/scrcpy
• xevokk/h264-converter
• 131/h264-live-player
• mbebenita/Broadway
• DeviceFarmer/adbkit
• xtermjs/xterm.js
• udevbe/tinyh264
• danielpaulus/quicktime_video_hack

scrcpy websocket fork

Currently, support of WebSocket protocol added to v1.19 of scrcpy

• Prebuilt package
• Source code