Coder Social home page Coder Social logo

realmq / realmq-platform Goto Github PK

View Code? Open in Web Editor NEW
57.0 5.0 8.0 2.53 MB

RealMQ :rocket: is an open source real-time messaging platform with focus on integrability, privacy, security and performance.

Home Page: https://realmq.com

License: Open Software License 3.0

JavaScript 97.91% Shell 2.01% Dockerfile 0.08%
mqtt real-time pubsub telecommunication platform iot signaling

realmq-platform's Introduction

RealMQ Logo

RealMQ Platform Build Status FOSSA Status Join the community on Spectrum

RealMQ πŸš€ Open Source Real-time Messaging Platform

RealMQ is a free and open source backbone for real-time communication. Our core values are performance and integrability. The platform is built around a convenient REST-API to manage access control, data channels and users/devices. Messages, events and data are exchanged via the battle proven MQTT protocol which is lightweight, fast and well established in the world of IoT. Clients connect to our highly scalable MQTT broker that takes care of delivering messages to all connected subscribers. On top of this slim but flexible core you can build awesome realtime products and services around use-cases like:

  • Device to device communication
  • Exchange signals / send and receive realtime events
  • Broadcast messages or events to segmented users
  • Aggregate and dispatch data
  • Emit sensor/IoT Device data
  • Chat

Features

REST API

We follow the API first approach, as we see our API as the central component in our architecture. Our mission is to build a developer friendly API with industry standard patterns and tools like RESTful design principles and an OpenAPI specification. It goes without saying that all requests are TLS encrypted and authentication is backed by security tokens (Bearer authentication).

Message Persistence

Message history can be enabled and fine tuned on channel level. Once enabled, all messages published to the channel will be kept for a configured amount of time. Persisted messages can be retrieved via API history resource.

Explore code samples

Using RealMQ via Node SDK Using RealMQ via Web SDK

// Create a new channel that keeps messages for 2 weeks 
const channel = await realmq.channels.create({
  history: '2 weeks'
});

// Retrieve the history (last 20 messages)
const history = await realmq.history.query({
  channel,
  limit: 20
});

Using RealMQ via CURL

curl -X POST -H 'Authorization: Bearer token...' -H 'Content-Type: application/json' \
     -d '{"history": "2 weeks"}' \
     https://api.realmq.com/client/v1/channels
     
curl -H 'Authorization: Bearer token...' \
     https://api.realmq.com/client/v1/channels/:channelId/history?limit=20

Custom Ids

It’s just a little detail that manifests the seamless integration into your stack: All API resources can be operated with optional custom ids. read more

Explore code samples

Using RealMQ via Node SDK Using RealMQ via Web SDK

// Create resources with custom ids
const channel = await realmq.channels.create({ id: 'custom-channel-id' });
console.log(channel.id); // => custom-channel-id

// Retrieve resources with custom ids
const channel = await realmq.channels.retrieve('custom-channel-id');

// Or get an id auto generated
const channel = await realmq.channels.create();
console.log(channel.id) // => 20f62e87-e689-4a11-bcf3-a78026fffd85

Using RealMQ via CURL

# Create resources with custom ids
curl -X POST -H 'Authorization: Bearer token...' -H 'Content-Type: application/json' \
     -d '{"id": "custom-channel-id"}' \
     https://api.realmq.com/client/v1/channels

# Retrieve resources with custom ids
curl -H 'Authorization: Bearer token...' /
     https://api.realmq.com/client/v1/channels/custom-channel-id

# Or get an id auto generated
curl -X POST -H 'Authorization: Bearer token...' \
     https://api.realmq.com/client/v1/channels

Realms

The RealMQ platform comes with built in multi-tenancy / project support. You have the ability to operate multiple, isolated data containers. Our so-called Realms make it a breeze to setup clean separation of different applications and launch on-the-fly environments (like prod, test, demo). Realms are completely transparent to clients, as we are able to route all RTM and API calls into the correct Realm via auth tokens. For Realm administration and access management you can operate directly against a dedicated administration api.

SDKs

We provide SDKs for NodeJS and the Browser. Other major platforms will follow. The SDKs aim for seamless integration and hence provide you with a convenient interface for interacting with our platform and are first class helpers for building reactive realtime apps.

Explore code sample

Using RealMQ via Node SDK

import RealMQ from '@realmq/node-sdk';
// Initialize with auth token
const realmq = RealMQ('sub_kg2...');

// Publish messages of any format
realmq.rtm.publish('some-channel', {
  status: 'Exited!'
});

// Subscribe to real-time updates
realmq.rtm.subscribe('some-channel', msg => {
  console.log('Horay! New message received:', msg);
});

// Retrieve subscriptions
const subscriptions = await realmq.subscriptions.list();

Security & Encryption

The RealMQ platform comes with fine grained access control capabilities, that can be managed via API. We don’t enforce a specific content type for the messages that are sent between clients. It’s totally up to you to decide what the payload contains. This allows for easy integration of any kind of end-to-end encryption for messages. On top of our TLS this establishes an additional security layer.

Roadmap

πŸ₯‡ Help up prioritize our backlog with your votes and comments on the features you are most exited about.

  • CLI Tool: Operate the RealMQ platform from the command line. (#86)
  • Public Cloud Service: Free developer tier and pay-as-you-go subscriptions. (#87) Reach out to [email protected] for beta access.
  • Dashboard: A web app for account-, realm management and monitoring. (#88)
  • Webhooks: Integrate RealMQ via http/webhook (#85)
  • Channel Pipes: Message routing & enrichment (#89)
  • Multi-Channel: SMS, email, push integration (#90)

πŸ‘ And you are more then welcome to start a discussion and suggest a new feature.

Configuration

The app can be configured via environment variables, that can also be injected via .env file. For a complete list of configuration options see config docs.

Run

The RealMQ platform comes with a docker compose setup aimed for local development.
To get you started, simply run:

npm run dev

This command will:

  • 🏁 setup .env file
  • πŸ“¦ fetch and install all dependencies.
  • πŸ”’ generate TLS certificates and setup local dns for [api.|rtm.]realmq.local
  • πŸ€– launch the mqtt broker on rtm.realmq.local:1883
  • 🐳 launch the app on https://api.realmq.local
  • πŸ”₯ code reload - restart app upon code changes

Setting up TLS certificate

npm run dev generates self-signed certificates. In order to trust them on your machine, import the ROOT CA cert from certs/dev-ca-root.crt.pem.

  • Firefox: Go to "Preferences > Advanced > Certificates > View Certificates > Authorities > Import" and select the CA certificate.
  • Chrome: Go to "Settings > Manage Certificates > Authorities > Import" and select the CA certificate.

Documentation

License

Copyright (c) 2018-2023 RealMQ GmbH.
Licensed under the Open Software License version 3.0

FOSSA Status

realmq-platform's People

Contributors

alrik avatar baranga avatar dependabot[bot] avatar derlarsen avatar fossabot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

realmq-platform's Issues

Replace account functionallity with fixed admin token

The account functionality in the admin API is not appropriate for a standalone solution but belongs to some kind of cloud / as-a-service solution. Let's remove it and implement a configurable fixed admin token instead.

Password hashes must not be exposed.

POST /admin/v1/accounts
{
  "email": "[email protected]",
  "password": "password"
}

Actual Result:

{
    "id": "d17752d3-9e83-46bc-a66a-5c549a1a5ba0",
    "email": "[email protected]",
--> "passwordHash": ".....",
    "createdAt": "2018-06-08T13:12:17.803Z",
    "updatedAt": "2018-06-08T13:12:17.803Z"
}

Expeted Result:

{
    "id": "d17752d3-9e83-46bc-a66a-5c549a1a5ba0",
    "email": "[email protected]",
    "createdAt": "2018-06-08T13:12:17.803Z",
    "updatedAt": "2018-06-08T13:12:17.803Z"
}

Entity deletion cascades

Currently, when an entity is deleted from the database, related entities will remain. E.g. when a channel is removed all channel subscriptions will live on. This will leave us with inconsistent data and has to be solved. We have do define removal cascades for all relevant entities and make sure that the database stays consistent after entities are deleted.

Upgrade node to latest LTS version

We currently run node 8. It's time for a little upgrade.

  • Upgrade docker file to use node 16
  • Upgrade package.json
  • Upgrade package-lock.json
  • Upgrade CI

Dashboard

This is a meta issue for keeping track on the dashboard.

Goal

The purpose of the dashboard web app is to improve account & realm management.

Todos

  • Write dashboard specification
  • Implement the dashboard
  • Write dashboard documentation

πŸ‘‰ Thanks for votes and comments to help us prioritize

Add basic realm limits

Add basic realm limits:

  • Maximum number of online clients
  • Maximum message size (broker docs)

Not covered:

  • Maximum number of messages per session per second (broker only supports setting a global session limit but not different limit for each session; this needs to be implemented manually; upper hard cap could be achieved by using throttle modifier)
  • Maximum session lifetime (broker docs; this seems to be not supported by the current broker version we use)

Multi Channel

This is a meta ticket for discussing & keeping track on smtp / email integration.

Abstract

Multi channel functionality comes as a unified gateway interface for connecting the RealMQ platform with other event and messaging systems. This adds the ability to trigger SMS, email and push notifications upon channel events. A APN and GCM/FCM integration guarantee message delivery even if a subscriber is not connected.

Todos

  • Specify multi channel functionality
  • SMS Integration
  • Slack Integration
  • Email Integration
  • APN Integraion
  • GCM/FCM Integration

πŸ‘‰ Thanks for votes and comments to help us prioritize

Bootstrap the community

  • Setup realmq on Spectrum
  • Code of Conduct / Contributing guidelines
  • Use welcoming and inviting language
  • Ask users for feedback / ideas / integration requests / use cases etc.
  • Include users into product development

Channel Pipes

This is a meta ticket for discussing & keeping track on channel pipes.

Abstract

Channel Pipes are an advanced feature to control and manipulate data flows. Their base functionality is to route traffic from a list of source channels to a list of target channels.
A webhook on a pipe allows for filtering and routing the traffic on message level. Pipes are very handy for building any kind of automation on top of messaging (e.g. chatbots).

Todos

  • Specification of channel pipes
  • Implementation of channel pipes
  • Documentation of channel pipes

πŸ‘‰ Thanks for votes and comments to help us prioritize

Landing page

When hitting the root path we see a error right now:

{"code":"EndpointNotFound","message":"/ does not exist."}

That's valid but not very helpful. It would be really nice to have some kind of explaining resource under this path giving some explanation and pointing to further resources and documents.

CLI Tool

This is a meta issue for keeping track on the cli tool.

  • Write cli tool specification
  • Implement cli tool
  • Write cli tool documentation

πŸ‘‰ Thanks for votes and comments to help us prioritize

Webhooks

This is a meta ticket for keeping track on webhook implementation.

  • Write specification #68
  • Implement webhooks

πŸ‘‰ Thanks for votes and comments to help us prioritize

Problems with dev script on MacOS

There are some issues with the dev script on mac os.

  • /etc/hosts will be altered on every run (check grep)
  • colored output does not work
npm run dev        

> @realmq/[email protected] dev /Users/Robin/Dev/Ciro/realmq-platform
> .npm-scripts/start-dev.sh

npm WARN [email protected] requires a peer of acorn@^6.0.0 but none is installed. You must install peer dependencies yourself.

up to date in 4.895s
Starting realmq_broker_1       ... done
Starting realmq_database_1     ... done
Starting realmq_certificates_1 ... done
Starting realmq_platform_1     ... done
Starting realmq_dispatcher_1   ... done
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
    [-e pattern] [-f file] [--binary-files=value] [--color=when]
    [--context[=num]] [--directories=action] [--label] [--line-buffered]
    [--null] [pattern] [file ...]
\e[1;34m☞\e[0m Setting up \e[1mlocal DNS\e[0m.
Adding \e[1m127.0.0.1 realmq.local api.realmq.local rtm.realmq.local\e[0m to \e[4m/etc/hosts\e[0m.
\e[0;32mβœ“\e[0m Hosts have been added successfully
Attaching to realmq_platform_1

Public Cloud Service

This is a meta ticket for discussing & keeping track on public cloud service implementation.

  • Specify quota & limits
  • Discuss pricing & packages
  • Implement cloud service signup
  • Implement billing
  • Deploy cluster to api.realmq.com / rtm.realmq.com

πŸ‘‰ Thanks for votes and comments to help us prioritize

Getting started / Readme / Project Presentation

  • The realmq-platform repository needs to be baam boom, schacka lacka
  • Readme must be clear
    • Visual
    • Summary
    • Benefits (emotial)
    • Features (technical)
    • Roadmap (get users involved)
    • How to Contribute (get users involved)
    • Architectural Documentation (get users involved)
    • Getting started Guide (lead users to to website, eg. doc pages, to potentially convert them to customers)
  • Optimize cross project linkage:
    • route all traffic to realmq-platform in order to bundle the community (and stargazers)
    • All Projects need a unified look & feel with some grafic and realmq platform summary

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.