emqx / eip

EMQX Improvement Proposals

EMQ X Improvement Proposals (EIP)

This repository contains the EMQ X Improvement Proposals (EIPs), to documentation the ideas, designs, or implement details of new features. All the EIPs are in Markdown (*.md) format.

New EIPs should first go to the active directory by creating a pull request and ask for an approval. After the feature is implemented it will be put into the implemented directory.

Before submitting an EIP, please read the 0000-proposal-template, which is a template demonstrating the format of EIPs.

Creating UML diagrams

It is possible to add UML diagrams using PlantUML. In order to do this, create a directory called active/XXXX-assets (replace XXXX with the EIP number), and put the files there. All files should have uml extension.

Then run make to generate the images.

eip's Issues

0008-community-plugins.md needs discuss

@z8674558

Yes, I think it's important to support and promote community plug-ins.

EMQ encourages the community to contribute useful plug-ins. If we do it well enough, we can help the author promote it.

0008 is a little simple and inflexible

1. How to approve new plug-ins for the released EMQ X version
2. The security of the plug-in, whether the plug-in is officially recommended by EMQ, and whether it infringes the copyright or interests of some people
3. The version compatibility of the plug-in, users may ask: this plug-in is cool, but can I use it in my EMQ X?

Grafana sign-a-plugin
Grafama plugins

We can build an online plug-in market, which has been discussed before, but we have encountered some technical problems.

If we can solve the problem now, the plug-in market looks very cool

1. Can we provide a *.zip file (plugin installation package, like emqx_auth_sqlite_plugins.zip), which contains the main *.beam file. The user can set it to the lib/ directory, unzip it, and reload it?

Usage

wget https://market.emqx.io/plugins/emqx/emqx_auth_sqlite.zip?version=1.3.1
unzip emqx_auth_sqlite.zip
mv emqx_auth_sqlite/ /opt/emqx/lib/

emqx_ctl plugins load  emqx_auth_sqlite

0003-new-config-files.md needs update

New Config Syntax section should be referring to 0002.
YAML has been declined, so we should not mention that.

0002-new-config-syntax needs update

The original proposal was based on the assumption to drop cuttlefish completely.
It should be updated to reflect the actual in-progress design.

The current design: to first make it backward compatible as much as possible,
when we are ready to cease 4.x development, we can start rewriting conf files to take advantage of HOCON.

YAML is proposed in the original doc, it should be moved to the 'declined alternatives'

node evacuation

In order to perform smooth node maintenance which requires EMQ X node restart,
we need a way to evacuate all connections in the node, force clients to move to other nodes in the cluster.

When evacuation starts

1. The node should be configured to stop accepting new connections.
2. To support LB, we need to let LB know that new connections should not be forwarded to this node.
3. With a parameterised pace, disconnect clients (but not kick as it may lose persisted sessions).
   For MQTT 5 clients, close the connection with reason code 0x9C (Use another server)
4. With a parameterised pace, trigger persistent session take-over from other nodes.

There should be a user interface to abort evacuation.

update 0003 to reflect the latest design in code

local session resume

Today, when a client re-connects. A new connection process is spawned (= a new session).
the new session try to take over the old, the old will unsubscribe itself, and the new will re-subscribe (in the trie table).
This is done even if the old session is found in the local node.

This issue is opened to

1. Investigate if it is feasible to handover the socket to the old process.
2. If 1 is feasible, create a proposal to follow up the changes.

certificate revocation

we had a few customers asked for client certificate revocation support.

there are a few options for us to support it:

1. Make use of the Erlang's builtin crl check and cache functionality
   https://www.erlang.org/doc/man/ssl.html#type-crl_check
   https://www.erlang.org/doc/man/ssl.html#type-crl_cache_opts

2. Like 1, but implement the cache callbacks by ourselves.

3. Provide custom verify callback in SSL options, and implement ourown check and cache
   https://www.erlang.org/doc/man/ssl.html#type-custom_verify
   return {revoked, _} , in case the cert is revoked.

No matter how it is implemented, we should have a configuration and an HTTP API to interface the users.

• Bootstrap the cache from a file (path of which is configurable)
• Persist the list in database
• Add CLI/HTTP APIs to add to / delete from the revocation list (and cache)

Allow User Properties for MQTT V5 Protocol to be used as placeholders

Hola! 👋

Let me know if this is not the right place to post this feature request. I'm posting this request based on this discussion: emqx/emqx#11528

Basically, with the introduction of User Properties in the V5 version of the MQTT protocol, I think it expands the possibility to enhance authentication processes by allowing the specified properties to be used as placeholders in different authentication/authorization services.

for my particular use-case, I want do to something like this:

• A client sets a User-Property of (ApiKey, ABC123) in the CONNECT packet along with a username and a password.
• EMQx sees this and sends as a POST request a username, password and ApiKey: ABC123 to an external HTTP service.
• The HTTP Service receives a JSON object like this: {"username": ..., "password": ..., "apiKey": "ABC123" }.

Or expose them via headers too, so the HTTP Endpoint receives a custom header like X-API-KEY: ABC123.

Also, allowing user properties to be used as placeholders, will allow more granular control over the PUBLISH packet as well with ACLs.

Thanks!