Coder Social home page Coder Social logo

local-first-web / relay Goto Github PK

View Code? Open in Web Editor NEW
91.0 3.0 12.0 2.68 MB

A tiny relay server that bridges two WebSocket connections, allowing the clients to talk directly to each other. (Formerly known as ๐ŸŸ Cevitxe Signal Server.)

TypeScript 98.80% JavaScript 1.20%
relay-server peer-to-peer distributed-systems local-first decentralized-applications distributed-system cevitxe taco

relay's Introduction

@localfirst/relay logo

@localfirst/relay is a tiny service that helps local-first applications connect with peers on other devices. It can run in the cloud or on any device with a known address.

Deploy to: Glitch | Heroku | AWS | Google | Azure | local server

Why

Getting two end-user devices to communicate with each other over the internet is hard. Most devices don't have stable public IP addresses, and they're often behind firewalls that turn away attempts to connect from the outside. This is a connection problem.

Even within a local network, or in other situations where devices can be reached directly, devices that want to communicate need a way to find each other. This is a problem of discovery.

What

This little server offers a solution to each of these two problems.

1. Discovery

Alice can provide a documentId (or several) that she's interested in. (A documentId is a unique ID for a topic or channel โ€” it could be a GUID, or just a unique string like ambitious-mongoose.)

diagram

If Bob is interested in the same documentId, each will receive an Introduction message with the other's peerId. They can then use that information to connect.

2. Connection

Alice can request to connect with Bob on a given documentId. If we get matching connection requests from Alice and Bob, we pipe their sockets together.

diagram

How

Server

From this repo, you can run this server as follows:

pnpm
pnpm start

You should see something like thsi:

> @localfirst/[email protected] start local-first-web/relay
> node dist/start.js

๐ŸŸ Listening at http://localhost:8080

You can visit that URL with a web browser to confirm that it's working; you should see something like this:

Running server from another package

From another codebase, you can import the server and run it as follows:

import { Server } from "@localfirst/relay/Server.js"

const DEFAULT_PORT = 8080
const port = Number(process.env.PORT) || DEFAULT_PORT

const server = new Server({ port })

server.listen()

Client

This library includes a lightweight client designed to be used with this server.

The client keeps track of all peers that the server connects you to, and for each peer it keeps track of each documentId (aka discoveryKey, aka channel) that you're working with that peer on.

import { Client } from "@localfirst/relay/Client.js"

client = new Client({ peerId: "alice", url: "myrelay.somedomain.com" })
  .join("ambitious-mongoose")
  .on("peer-connect", ({ documentId, peerId, socket }) => {
    // `socket` is a WebSocket

    // send a message
    socket.write("Hello! ๐ŸŽ‰")

    // listen for messages
    socket.addEventListener("data", event => {
      const message = event.data
      console.log(`message from ${peerId} about ${documentId}`, message)
    })
  })

โš  Security

This server makes no security guarantees. Alice and Bob should probably:

  1. Authenticate each other, to ensure that "Alice" is actually Alice and "Bob" is actually Bob.
  2. Encrypt all communications with each other.

The @localfirst/auth library can be used with this relay service. It provides peer-to-peer authentication and end-to-end encryption, and allows you to treat this relay (and the rest of the network) as untrusted.

Server API

The following documentation might be of interest to anyone working on the @localfirst/relay Client, or replacing it with a new client. You don't need to know any of this to interact with this server if you're using the included client.

This server has two WebSocket endpoints: /introduction and /connection.

In the following examples, Alice is the local peer and Bob is a remote peer. We're using alice and bob as their peerIds; in practice, typically these would be GUIDs that uniquely identify their devices.

/introduction/:localPeerId

  • :localPeerId is the local peer's unique peerId.

Alice connects to this endpoint, e.g. wss://myrelay.somedomain.com/introduction/alice.

Once a WebSocket connection has been made, Alice sends an introduction request containing one or more documentIds that she has or is interested in:

{
  type: 'Join',
  documentIds: ['ambitious-mongoose', 'frivolous-platypus'], // documents Alice has or is interested in
}

If Bob is connected to the same server and interested in one or more of the same documents IDs, the server sends Alice an introduction message:

{
  type: 'Introduction',
  peerId: 'bob', // Bob's peerId
  documentIds: ['ambitious-mongoose'] // documents we're both interested in
}

Alice can now use this information to request a connection to this peer via the connection endpoint:

/connection/:localPeerId/:remotePeerId/:documentId

Once Alice has Bob's peerId, she makes a new connection to this endpoint, e.g. wss://myrelay.somedomain.com/connection/alice/bob/ambitious-mongoose.

  • :localPeerId is the local peer's unique peerId.
  • :remotePeerId is the remote peer's unique peerId.
  • :documentId is the document ID.

If and when Bob makes a reciprocal connection by connecting to wss://myrelay.somedomain.com/connection/bob/alice/ambitious-mongoose, the server pipes their sockets together and leaves them to talk.

The client and server don't communicate with each other via the connection endpoint; it's purely a relay between two peers.

Deployment

Deploying to Glitch

You can deploy this relay to Glitch by clicking this button:

Remix on Glitch

Alternatively, you can remix the local-first-relay project.

Deploying to Heroku

This server can be deployed to Heroku. By design, it should only ever run with a single dyno. You can deploy it by clicking on this button:

Deploy

Or, you can install using the Heroku CLI as follows:

heroku create
git push heroku main
heroku open

Deploying to AWS Elastic Beanstalk

Install using the AWS CLI:

eb init
eb create
eb open

Deploying to Google Cloud

Install using the Google Cloud SDK:

gcloud projects create my-local-first-relay --set-as-default
gcloud app create
gcloud app deploy
gcloud app browse

Deploying to Azure

Install using the Azure CLI:

az group create --name my-local-first-relay --location eastus
az configure --defaults group=my-local-first-relay location=eastus
az appservice plan create --name my-local-first-relay --sku F1
az webapp create --name my-local-first-relay --plan my-local-first-relay
az webapp deployment user set --user-name PEERID --password PASSWORD
az webapp deployment source config-local-git --name my-local-first-relay
git remote add azure https://[email protected]/my-local-first-relay.git
git push azure main
az webapp browse --name my-local-first-relay

AWS Lambda, Azure Functions, Vercel, Serverless, Cloudwatch Workers, etc.

Since true serverless functions are stateless and only spun up on demand, they're not a good fit for this server, which needs to remember information about connected peers and maintain a stable websocket connection with each one.

License

MIT

Prior art

Inspired by https://github.com/orionz/discovery-cloud-server

Formerly known as ๐ŸŸ Cevitxe Signal Server. (Cevitxe is now @localfirst/state)

relay's People

Contributors

herbcaudill avatar icidasset avatar okdistribute avatar orta avatar ryanhlewis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

relay's Issues

Server crash

Sometimes, I get this error in the relay server when doing production testing with fun edge cases, such as clearing data from a device and disconnecting from the relay unexpectedly. In a real app this would look like someone uninstalling the app/clear cache while their peer is actively sending websocket messages & trying to connect to them.

I don't think that the relay should ever throw an error and crash, is there a way to catch all of these errors and log them? Would a PR be welcome on this?

Error: WebSocket is not open: readyState 2 (CLOSING)
    at WebSocket.send (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/websocket.js:314:19)
    at Server.sendIntroduction (/Users/okdistribute/dev/backchannel/node_modules/@localfirst/relay/dist/Server.js:95:74)
    at WebSocket.<anonymous> (/Users/okdistribute/dev/backchannel/node_modules/@localfirst/relay/dist/Server.js:74:34)
    at WebSocket.emit (events.js:315:20)
    at Receiver.receiverOnMessage (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/websocket.js:720:20)
    at Receiver.emit (events.js:315:20)
    at Receiver.dataMessage (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/receiver.js:414:14)
    at Receiver.getData (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/receiver.js:346:17)
    at Receiver.startLoop (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/receiver.js:133:22)
    at Receiver._write (/Users/okdistribute/dev/backchannel/node_modules/express-ws/node_modules/ws/lib/receiver.js:69:10)

Error in README documentation about introduction message format

In the README documentation at: https://github.com/local-first-web/relay#introduction-endpoint-introductionlocalid

It was documented that the introduction JSON message format is:

{
  type: 'Join',
  join: ['ambitious-mongoose', 'frivolous-platypus'], // documents I have or am interested in
}

This causes the following behavior on the server, where every connecting client is matched (since documentIds for every connection is undefined):

  lf:relay:58888 received introduction request client1 +2m
  lf:relay:58888 introductionConnection client1 +0ms
  lf:relay:58888 introduction request: { type: 'Join', join: [ 'docId-1', 'client1' ] } +41ms
  lf:relay:58888 received introduction request client2 +6s
  lf:relay:58888 introductionConnection client2 +0ms
  lf:relay:58888 introduction request: { type: 'Join', join: [ 'docId-1', 'client2' ] } +38ms
  lf:relay:58888 sending introductions client2 client1 [ undefined ] +1ms

The introduction JSON message element name should be "documentIds" instead of "join":

  lf:relay:58888 received introduction request client1 +48s
  lf:relay:58888 introductionConnection client1 +0ms
  lf:relay:58888 introduction request: { type: 'Join', documentIds: [ 'docId-1', 'client1' ] } +48ms
  lf:relay:58888 received introduction request client2 +6s
  lf:relay:58888 introductionConnection client2 +0ms
  lf:relay:58888 introduction request: { type: 'Join', documentIds: [ 'docId-1', 'client2' ] } +39ms
  lf:relay:58888 sending introductions client2 client1 [ 'docId-1' ] +1ms

Clarification for unofficial client usage

Your Github repo has recently caught my eye, and I've been looking to dabble with it.

However, I am working with C# and Unity, not exactly Javascript. Therefore, I am not using your built Client and instead opting for simple WebSocket connections. This has all been fine and dandy, and I've managed to connect clients to their websockets, but not to one another.

Therein, my questions lie. I went through your source code a number of times to figure this out, but as a last resort, I am requesting clarification from you. Here are my questions:

  1. How does the DocumentID get sent in the Introduction Request?

It is clear that the Server seems to take the Introduction Request and uses it to find information such as username and documentId. The username part is immediately visible: /introduction/username. However, the DocumentID is not. I have tried attaching it as a header to the request, under the name "Join", but that did not seem to work. I have tried attaching it as a parameter in the URL, as /introduction/username?join=document, but that failed too. I even created a KeyValue JSON message with the key "Join" to send, and still, nothing.

  1. How does the Connection Endpoint function?

You say the Client and Server do not communicate with one another on the connection endpoint. How exactly does this function? Is the client given some public IP of another client and some open port?

  1. Is there support for multiple people?

I am not talking large-scale support, 20+, etc. but rather for even small groups. If the server is a relay server, how large can it expand? It seems that you iterate "two" over and over again within the documentation, but within your code you explicitly call and allow for multiple DocumentIds, which seems to suggest there is support for multiple clients connected to one another, in a rather strange fashion.

Thanks, and I wish you all the well in this endeavor.

Heartbeat Size

As far as I can tell using this relay, connections are made and stay alive for approx. 55 seconds before they are deemed "idle" and closed. This is prevented by the use of the heartbeat signal made every five seconds, as seen in your code.

const HEARTBEAT = JSON.stringify({ type: 'Heartbeat' })
this.heartbeat = setInterval(() => socket.send(HEARTBEAT), 5000)

My question is, if this is a relay server and intended to have minimal load and bandwidth between the connections, would it not be better to send off a Heartbeat message as something incredibly short (1 bit?).

It doesn't even have to be proper JSON, as it would simply be ensuring that a connection exists.

The heartbeat receiver and the default unknown data receiver both essentially return the same output:

case 'Heartbeat':        // nothing to do       
this.log('โ™ฅ')        
break

default:        
break

Am I overthinking this? Is the small bit difference between a "{ type:'Heartbeat' }" string and a 0/1 negligible?
Does the understanding behind knowing a heartbeat signal is being received outweigh the tiny increase in possible performance or bandwidth?

Thanks,

Multiplex

Looking at the code, I can't tell -- does each document create it's own websocket connection? Is there any interest in a feature that multiplexes document events over a single websocket connection to the relay?

The benefits are improved performance when a user has many documents open, especially on mobile devices.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.