lcfyi / websocket-live-streamer

A proof-of-concept for using WebSockets to send real-time webcam data to a client. Runs at ~0.1s latency.

Livestreaming webcam through WebSockets

This is a proof-of-concept demo of using WebSockets to stream a video camera feed through WebSockets, so any browser can access it.

At the moment, it is very work-in-progress. The stream is coming in as the raw JPEG binary, which is then read as a blob that's loaded into an Image(), which is then drawn onto an HTML5 canvas.

The latency is about ~0.1s, but with relatively high CPU usage (NEW! Down from ~50% to ~20% CPU usage)

To demo this, make sure you have opencv-python and websockets installed, and you have Python 3.6+.

Demo

1. Ensure you have the prerequisites, try pip install -r requirements.txt
2. Run python server.py
3. Visit localhost:8000 to view the stream. A corresponding window will also be loaded with the current time.

Running this in a Raspberry Pi

This currently works pretty well; I'm noticing ~0.4s latency between frames, but it runs at 30fps otherwise.

To try it out, make sure you have the v4l2 drivers installed. Then, enable it with sudo modprobe bcm2835-v4l2. You can add it at boot by adding bcm2835-v4l2 to /etc/modules.

websocket-live-streamer's Issues

Why couldn't i change the Websocket port?

Thank for sharing bro,

I have a question, is the 8585 port of ws protocol is fixed?

I have tried changing it to other ports, but it only runs on 8585 port. How could I change it?
Because I have many IP Camera need live streaming.
I plan to create each of them one websocket service, and of cource, different port. If the 8585 port is fixed and can not change, is there any solution to serve many Camera with Websocket?

Thank you so much.

Websocket streaming via JS to the Python Server

Hi @lcfyi . Firstly, what a brilliant project and the approach is just cogent. I had a series of doubts though.

1. I want to setup a similar app on my DO-droplet, so opening an OpenCV camera is throwing me an exception saying "can't open camera by index". Thus, I decided, I would go for a normal JS Webpage that takes the webcam stream from any web browser and anywhere (getUserMedia()), and I also have a sample self-signed SSL Certificate attached to the server. I want to thus send this webcam stream (which is working on the webRTC side btw) to the python backend.
2. How do I take the camera input from JS webpage and send it to the python backend using the same 'python-websockets' procedure that you've used? Also, I am assuming that websockets is a convenient way to connect this JS webcam to python's OpenCV, how can I pass it though?
3. So far, I am able to send a text from an HTML form to the python's websocket server; I just want to send the camera stream in the same way to the backend. The moment I am able to do so, I can then process the image frames that I'll be recieving from the webcam stream.

Anyway, do let me know if there's a way and also if I missed on anything. I am also attaching my JS, HTML and Websocket code for your reference.
A. wsclient.js

var webSocket   = null;
var ws_protocol = null;
var ws_hostname = null;
var ws_port     = null;
var ws_endpoint = null;
function onConnectClick() {
    var ws_protocol = document.getElementById("protocol").value;
    var ws_hostname = document.getElementById("hostname").value;
    var ws_port     = document.getElementById("port").value;
    var ws_endpoint = document.getElementById("endpoint").value;
    openWSConnection(ws_protocol, ws_hostname, ws_port, ws_endpoint);
}
function onDisconnectClick() {
    webSocket.close();
}
function openWSConnection(protocol, hostname, port, endpoint) {
    var webSocketURL = null;
    webSocketURL = protocol + "://" + hostname + ":" + port + endpoint;
    console.log("openWSConnection::Connecting to: " + webSocketURL);
    try {
        webSocket = new WebSocket(webSocketURL);
        webSocket.onopen = function(openEvent) {
            console.log("WebSocket OPEN: " + JSON.stringify(openEvent, null, 4));
            document.getElementById("btnSend").disabled       = false;
            document.getElementById("btnConnect").disabled    = true;
            document.getElementById("btnDisconnect").disabled = false;
            var video = document.querySelector("#videoElement");

            if (navigator.mediaDevices.getUserMedia) {
              navigator.mediaDevices.getUserMedia({ video: true })
                .then(function (stream) {
                  video.srcObject = stream;
                  alert("Your webcam is connected!")
                })
                .catch(function (err0r) {
                  console.log("Something went wrong!");
                });
            }
        };
        webSocket.onclose = function (closeEvent) {
            console.log("WebSocket CLOSE: " + JSON.stringify(closeEvent, null, 4));
            document.getElementById("btnSend").disabled       = true;
            document.getElementById("btnConnect").disabled    = false;
            document.getElementById("btnDisconnect").disabled = true;
        };
        webSocket.onerror = function (errorEvent) {
            console.log("WebSocket ERROR: " + JSON.stringify(errorEvent, null, 4));
        };
        webSocket.onmessage = function (messageEvent) {
            var wsMsg = messageEvent.data;
            console.log("WebSocket MESSAGE: " + wsMsg);
            if (wsMsg.indexOf("error") > 0) {
                document.getElementById("incomingMsgOutput").value += "error: " + wsMsg.error + "\r\n";
            } else {
                document.getElementById("incomingMsgOutput").value += "message: " + wsMsg + "\r\n";
            }
        };
    } catch (exception) {
        console.error(exception);
    }
}
function onSendClick() {
    if (webSocket.readyState != WebSocket.OPEN) {
        console.error("webSocket is not open: " + webSocket.readyState);
        return;
    }
    var msg = document.getElementById("message").value;
    webSocket.send(msg);
}

B. client.html

<!DOCTYPE html>
<html>
<title>Webcam</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="Content-Security-Policy"  content="connect-src * 'unsafe-inline';">
<script src="wsclient.js"></script>
<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Montserrat">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<style>
body,h1,h2,h3,h4,h5,table,tr,td,th,p,div,a, {font-family: "Montserrat", sans-serif}
table    { border: 0px solid black; }
input    { width: 510px; }
select   { width: 300px; }
textarea { width: 513px; border: 0px solid black; }
#btnConnect    { width: 120px; height: 30px; border: 0px; border-radius: 6px; background-color: #047FD4; color: white; font-weight: bold;}
#btnDisconnect { width: 120px; height: 30px; border: 0px; border-radius: 6px; background-color: #047FD4; color: white; font-weight: bold;}}
#btnSend       { width: 120px; height: 30px; border: 0px; border-radius: 6px; background-color: #047FD4; color: white; font-weight: bold;}}
#container{
  margin: 0px auto;
  width: 500px;
  height: 370px;
  border: 10px slategrey solid;
}
#videoElement{
  width: 500px;
  height: 370px;
  background-color: white;
}
</style>
<body>
<div class="w3-content w3-text-dark-grey" style="max-width:1500px">
<br><br><br>
<div class="w3-clear">
<header class="w3-center w3-margin-bottom">
  <h1><b>Web Socket for Webcam</b></h1>
  <div class="w3-center container">
    <table align="center">
        <tr>
            <td width="200px">WS Protocol</td>
            <td>
                <select id="protocol">
                    <option value="ws" selected="selected">ws</option>
                    <option value="wss">wss</option>
                </select>
            </td>
        </tr>
        <tr>
            <td>WS Hostname</td>
            <td><input type="text" id="hostname"/></td>
        </tr>
        <tr>
            <td>WS Port</td>
            <td><input type="text" id="port"/></td>
        </tr>
        <tr>
            <td>WS Endpoint</td>
            <td><input type="text" id="endpoint"/></td>
        </tr>
     </table>
  </div>
  <p><b>Click on "connect" to start with the webcam streaming.</b></p>
  <p class="w3-padding-16"><button id="btnConnect" onclick="onConnectClick()">Connect</button></p>
</header>
</div>

<div class="w3-container">
  <div class="w3-center container">
    <video autoplay="true" id="videoElement">
    </video>
  </div>
  <div class="w3-center container">
  <br><br><p class="w3-padding-16"><button id="btnDisconnect" onclick="onDisconnectClick()">Disconnect</button></p>
  </div>
  <div class="w3-center container">
    <table align="center">
      <tr>
          <td width="200px">Message</td>
          <td><input type="text" id="message"/></td>
      </tr>
      <tr>
          <td></td>
          <td>
              <input id="btnSend" type="button" value="Send Message" disabled="disabled" onclick="onSendClick()">
          </td>
      </tr>
    </table><br/>
    <textarea id="incomingMsgOutput" rows="10" cols="20" disabled="disabled"></textarea>
  </div>
</div>
</body>
</html>

C. server.py

import asyncio
import websockets

async def hello(websocket, path):
    name = await websocket.recv()
    print (f"< {name}")

    greeting = f"Hello {name}!"

    await websocket.send(greeting)
    print (f"> {greeting}")

start_server = websockets.serve(hello, "0.0.0.0", 1000)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()