emqx / emqx-rule-engine

EMQX Rule Engine

emqx-rule-engine

IoT Rule Engine for EMQ X Broker.

Concept

iot rule "Rule Name"
  when
     match TopicFilters and Conditions
  select
    para1 = val1
    para2 = val2
  then
    take action(#{para2 => val1, #para2 => val2})

Architecture

          |-----------------|
 Pub ---->| Message Routing |----> Sub
          |-----------------|
               |     /|\
              \|/     |
          |-----------------|
          |   Rule Engine   |
          |-----------------|
               |      |
        Backends Services Bridges

SQL for Rule query statement

select id, time, temperature as t from "topic/a" where t > 50;

License

Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

emqx-rule-engine's Issues

Rule engine republish messaging with default retain flags

The rule engine re-publish action sends a message without retain flags, this would cause webhook crashes due to function_clause.

Rule engine with auth_result = 'success' never trigger a resource

Environment

• OS: Ubuntu 16.04
• Erlang/OTP:
• EMQ: 3.2.3

Description

A rule engine with sql

SELECT * FROM "client.connected" WHERE auth_result = 'success'

is working using Test SQL from dashboard as expected, but never trigger a resource.

Removing auth_result = 'success' resource is triggered as expected.
auth_result is present in output from broker.

More actions?

Hello!

Are you planing to support more actions?
For example, "REWRITE TOPIC", "ACTION DENY", "ACTION DELAY", etc

In my case I need to allow subscription like "topic/{something_but_not_+}/#" and I have ACL allows "topic/+/#" but also I need lua hook to drop subscriptions starts with "topic/+".
And I could not just use eq in ACL, because there too much combinations of topics.
If I just put ACL rule deny {eq, "topic/+/#"} then I colud subscribe to "topic/+/+" and so on.

I would be nice to use Rule Engine even ACL syntax do not allow that (by sign wich allows any value but not plus).

Bad Arguments Error

While creating a new rule through the REST API, the body I am sending is :

{ "name": "test", "for": "data_to_webserver", "rawsql": "SELECT payload.Temperature as temp, clientid as device_id FROM e03719f2-f13b-416a-97b7-d0a93ae8ae3c WHERE temp < 10", "actions": [ { "name": "Google Cloud Functions Email Alert", "params": { "$resource": "resource:1b3f75d5" } } ], "description": "test" }

MQTT5 specific attributes

Hi!

The Rule Engine is amazing and powerfull part of EMQX. But it would be nice to be able to select MQTT5 specific attributes by it`s name or all of them by *.
Are you planning to do feature like that?

Thank you!

External DB for Rules

Instead of keeping the rules on the local DB, I'd like to use an external connection. Is there any way to replace it with Redis for example?

Config with Rules

Hello!

Are you planning to implement configuring Rule Engine by config file?
It would be great to have that ability making broker`s restart less pain.

Thank you!

Emqx rule crashes

2020-06-02 13:07:19.873 [critical] Can not re-build rule <<"rule:0af78057">>: {resource_not_initialized,<<"resource:1d373635">>}. The rule is disabled.Fix the issue and enable it manually.
Stacktrace: [{emqx_rule_engine,with_resource_params,1,[{file,"/emqx_rel/_build/emqx/lib/emqx_rule_engine/src/emqx_rule_engine.erl"},{line,356}]},{emqx_rule_engine,'-refresh_actions/2-fun-0-',2,[{file,"/emqx_rel/_build/emqx/lib/emqx_rule_engine/src/emqx_rule_engine.erl"},{line,528}]},{lists,foreach,2,[{file,"lists.erl"},{line,1338}]},{emqx_rule_engine,'-refresh_rules/0-lc$^0/1-0-',1,[{file,"/emqx_rel/_build/emqx/lib/emqx_rule_engine/src/emqx_rule_engine.erl"},{line,306}]},{emqx_rule_engine,refresh_rules,0,[{file,"/emqx_rel/_build/emqx/lib/emqx_rule_engine/src/emqx_rule_engine.erl"},{line,314}]},{emqx_rule_engine_app,start,2,[{file,"/emqx_rel/_build/emqx/lib/emqx_rule_engine/src/emqx_rule_engine_app.erl"},{line,33}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,277}]}]

Can some one explain the issue?

How to create a rule

Hi, I have installed the raspberry pi version of emqx and is it possible to create a rule there, and also i want to know if we can save the rules in mongodb instead of sql

Action failed sometimes

2020-06-12 08:59:15.535 [error] <<"T970G1X6ECLI">>@172.20.0.1:58302 Take action <<"data_to_kafka_1591952033848069670">> failed, continue next action, reason: {error,{badmatch,[]}}, Stack: [{wolff_producers,lookup_producer,2,[{file,"/emqx_rel/_checkouts/wolff/src/wolff_producers.erl"},{line,156}]},{wolff_producers,do_pick_producer,4,[{file,"/emqx_rel/_checkouts/wolff/src/wolff_producers.erl"},{line,124}]},{wolff,send_sync,3,[{file,"/emqx_rel/_checkouts/wolff/src/wolff.erl"},{line,117}]},{emqx_rule_runtime,take_action,4,[{file,"/emqx_rel/_checkouts/emqx_rule_engine/src/emqx_rule_runtime.erl"},{line,230}]},{lists,map,2,[{file,"lists.erl"},{line,1239}]},{emqx_rule_runtime,do_apply_rule,2,[{file,"/emqx_rel/_checkouts/emqx_rule_engine/src/emqx_rule_runtime.erl"},{line,104}]},{emqx_rule_runtime,apply_rules,2,[{file,"/emqx_rel/_checkouts/emqx_rule_engine/src/emqx_rule_runtime.erl"},{line,46}]},{emqx_rule_events,on_message_publish,2,[{file,"/emqx_rel/_checkouts/emqx_rule_engine/src/emqx_rule_events.erl"},{line,81}]}]

I used to the rule engine to transfer messages to kafka, there always have some actions failed.

Most of the messages are transfered successfully, but some failed for no reason, which is not stable for production environment.

Rules not persistent

After creating rules in emqx broker, when the pod restarted the rules were deleted. Have to create rules again for the Kubernetes deployment pod.

$events/message_published

Is it possible to make an optimized $events/message_published available via config with a topic pattern filter/username filter/clientid filter?

Its a great alternative for the fan-in pattern where I'm essentially ingesting a ton of messages and extracting the FROM:CLIENTID, FROM:USERNAME via topic pattern.

On disconnect unsubscribe is not triggered

Environment

OS: Ubuntu 16.04
EMQ: 3.2.3

Description

The intention is capture when specific device is no longer subscribed to a topic.
But if device is loosing internet connection for example only event.disconnected is triggered but not event.unsubscribe.

The reason for using unsubscribe is because we don't want to trigger engine rule for all connected devices and sql 'LIKE' operator doesn't work for filter by field (clientId for example).
So we tried to use unsubscribe filtering by topic and found this issue (if this is actually an issue).

You may not consider this as a bug because it may be functional but it doesn't seems like correct behavior.

btw Rule Engine is great feature, make life much easier.

Message batch processing available?

When create a rule with event like SELECT * FROM "t/#", message is sent to action handler one by one. Is it possible to put them to a message pool, e.g. SELECT * FROM "t/#" of last 1 second , then event is triggered each second, and all messages published in the last second are all put to the message pool as an array, and send to actions? Another example is SELECT BATCH 100 * FROM "t/#", then when matched messages is added up to 100, send to actions. Save a batch to db is more efficient then one by one.