Coder Social home page Coder Social logo

emqx / emqx-auth-pgsql Goto Github PK

View Code? Open in Web Editor NEW
36.0 19.0 29.0 441 KB

EMQX PostgreSQL Authentication Plugin

Home Page: https://www.emqx.io

License: Apache License 2.0

Erlang 96.56% Makefile 1.97% Dockerfile 1.47%
emqx emqx-plugin authentication postgresql

emqx-auth-pgsql's Introduction

emqx_auth_pgsql

Authentication/ACL with PostgreSQL Database.

Build Plugin

make && make tests

Configuration

File: etc/emqx_auth_pgsql.conf

## PostgreSQL server address.
##
## Value: Port | IP:Port
##
## Examples: 5432, 127.0.0.1:5432, localhost:5432
auth.pgsql.server = 127.0.0.1:5432

## PostgreSQL pool size.
##
## Value: Number
auth.pgsql.pool = 8

## PostgreSQL username.
##
## Value: String
auth.pgsql.username = root

## PostgreSQL password.
##
## Value: String
## auth.pgsql.password =

## PostgreSQL database.
##
## Value: String
auth.pgsql.database = mqtt

## PostgreSQL database encoding.
##
## Value: String
auth.pgsql.encoding = utf8

## Whether to enable SSL connection.
##
## Value: true | false
auth.pgsql.ssl = false

## SSL keyfile.
##
## Value: File
## auth.pgsql.ssl_opts.keyfile =

## SSL certfile.
##
## Value: File
## auth.pgsql.ssl_opts.certfile =

## SSL cacertfile.
##
## Value: File
## auth.pgsql.ssl_opts.cacertfile =

## Authentication query.
##
## Value: SQL
##
## Variables:
##  - %u: username
##  - %c: clientid
##
auth.pgsql.auth_query = select password from mqtt_user where username = '%u' limit 1

## Password hash.
##
## Value: plain | md5 | sha | sha256 | bcrypt
auth.pgsql.password_hash = sha256

## sha256 with salt prefix
## auth.pgsql.password_hash = salt,sha256

## sha256 with salt suffix
## auth.pgsql.password_hash = sha256,salt

## bcrypt with salt prefix
## auth.pgsql.password_hash = salt,bcrypt

## pbkdf2 with macfun iterations dklen
## macfun: md4, md5, ripemd160, sha, sha224, sha256, sha384, sha512
## auth.pgsql.password_hash = pbkdf2,sha256,1000,20

## Superuser query.
##
## Value: SQL
##
## Variables:
##  - %u: username
##  - %c: clientid
auth.pgsql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1

## ACL query. Comment this query, the ACL will be disabled.
##
## Value: SQL
##
## Variables:
##  - %a: ipaddress
##  - %u: username
##  - %c: clientid
auth.pgsql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'

Load Plugin

./bin/emqx_ctl plugins load emqx_auth_pgsql

Auth Table

Notice: This is a demo table. You could authenticate with any user table.

CREATE TABLE mqtt_user (
  id SERIAL primary key,
  is_superuser boolean,
  username character varying(100),
  password character varying(100),
  salt character varying(40)
)

ACL Table

CREATE TABLE mqtt_acl (
  id SERIAL primary key,
  allow integer,
  ipaddr character varying(60),
  username character varying(100),
  clientid character varying(100),
  access  integer,
  topic character varying(100)
)

INSERT INTO mqtt_acl (id, allow, ipaddr, username, clientid, access, topic)
VALUES
	(1,1,NULL,'$all',NULL,2,'#'),
	(2,0,NULL,'$all',NULL,1,'$SYS/#'),
	(3,0,NULL,'$all',NULL,1,'eq #'),
	(5,1,'127.0.0.1',NULL,NULL,2,'$SYS/#'),
	(6,1,'127.0.0.1',NULL,NULL,2,'#'),
	(7,1,NULL,'dashboard',NULL,1,'$SYS/#');

allow: Client's permission to access a topic. '0' means that the client does not have permission to access the topic, '1' means that the client have permission to access the topic.

ipaddr: Client IP address. For all ip addresses it can be '$all' or 'NULL'.

username: Client username. For all users it can be '$all' or 'NULL'.

clientid: Client id. For all client ids it can be '$all' or 'NULL'.

access: Operations that the client can perform. '1' means that the client can subscribe to a topic, '2' means that the client can publish to a topic, '3' means that the client can subscribe and can publish to a topic.

topic: Topic name. Topic wildcards are supported.

Notice that only one value allowed for ipaddr, username and clientid fields.

License

Apache License Version 2.0

Author

EMQX Team.

emqx-auth-pgsql's People

Contributors

emqplus avatar gilbertwong96 avatar hjianbo avatar huangdan avatar linjunjj avatar maltej avatar ngjaying avatar nini-os avatar rory-z avatar setamodius avatar spring2maz avatar terry-xiaoyu avatar tigercl avatar wwhai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

emqx-auth-pgsql's Issues

Configuration to use with pgcrypto extension?

Hello and thanks for your work.

Firtsly, I should say that plugin works perfectly saving and configuring broker and Postgresql with plain password. But now, I'm trying to configure database and broker to use encrypted passwords without luck.

To store passwords, I'm using pgcrypt posgresql extension as follow:
INSERT INTO mqtt_user(..., name, password,...) VALUES (..., 'test', crypt('test',gen_salt('bf')))

In the plugin .conf I set:
auth.pgsql.password_hash = bcrypt

but every time I try to connect to broker, login fails. The log shows:
2017-09-19 02:54:08.700 [error] <0.1525.0>@emqttd_protocol:process:217 Client([email protected]:64933): Username 'test' login failed for {function_clause,[{emqttd_auth_mod,passwd_hash,[bcrypt,<<"test">>],[{file,"src/emqttd_auth_mod.erl"},{line,55}]},{emq_auth_pgsql,check_pass,3,[{file,"src/emq_auth_pgsql.erl"},{line,57}]},{emq_auth_pgsql,check,3,[{file,"src/emq_auth_pgsql.erl"},{line,46}]},{emqttd_access_control,auth,3,[{file,"src/emqttd_access_control.erl"},{line,60}]},{emqttd_protocol,authenticate,2,[{file,"src/emqttd_protocol.erl"},{line,516}]},{emqttd_protocol,process,2,[{file,"src/emqttd_protocol.erl"},{line,197}]},{emqttd_client,received,2,[{file,"src/emqttd_client.erl"},{line,322}]},{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1046}]}]}
2017-09-19 02:54:08.709 [error] <0.1526.0>@emqttd_protocol:process:217 Client([email protected]:64934): Username 'test' login failed for {function_clause,[{emqttd_auth_mod,passwd_hash,[bcrypt,<<"test">>],[{file,"src/emqttd_auth_mod.erl"},{line,55}]},{emq_auth_pgsql,check_pass,3,[{file,"src/emq_auth_pgsql.erl"},{line,57}]},{emq_auth_pgsql,check,3,[{file,"src/emq_auth_pgsql.erl"},{line,46}]},{emqttd_access_control,auth,3,[{file,"src/emqttd_access_control.erl"},{line,60}]},{emqttd_protocol,authenticate,2,[{file,"src/emqttd_protocol.erl"},{line,516}]},{emqttd_protocol,process,2,[{file,"src/emqttd_protocol.erl"},{line,197}]},{emqttd_client,received,2,[{file,"src/emqttd_client.erl"},{line,322}]},{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,1046}]}]}

Can you guide me to configure correctly??

Maybe, my problem is related with #68. In that case, and if you consider, you could delete mine and I'll follow this one.

Thanks in advance.
Bye.

Access `topic` in SQL query

Hi,

it would be great if I could access the topic during the SQL auth query.

As I can see here: https://github.com/emqtt/emqttd_plugin_pgsql/blob/c81f663312ae371808fe10a5997ea78c3033eae7/src/emqttd_acl_pgsql.erl#L55

I can use {"%u", Username}, {"%c", ClientId}, {"%a", inet_parse:ntoa(IpAddr)} placeholders in the SQL query.

Would it be possible to add a new placeholder, e,g "%t, Topic so I can verify that the client is allowed to subscribe to a particular topic.

Also I would like to distinguish which topics can be used for publish and which for subscribe

Loading plugin failed

I get node is not running error when I try to load this plugin...I know the node is running..I started as a console as well as a service still get node is not running error when I try to load the plugin.I am on windows and using build emqttd-windows-0.12.2-beta-20151016 maybe I am missing something basic. any help would be appreciated. thanks

Support random order loading of emqx (auth_pgsql plugin) and Postgres

Currently, the plugin fails to load if Postgres is not initially available. More importantly, it never attempts to reconnect in this scenario. In automated cloud deployments or docker-compose like scenarios, this is problematic since ordering cannot be fixed. Is it possible to let the plugin load and continue reconnection attempts even if Postgres is not initially available?

can't load the emq_auth_pgsql when the db user is NOT "root"

OS: CentOS 6.8 x86_64
EMQTTD: 2.2

my emq_auth_pgsql.conf list below:
## Postgre Server: 5432, 127.0.0.1:5432, localhost:5432
auth.pgsql.server = 127.0.0.1:5432
auth.pgsql.pool = 8
auth.pgsql.username = mqtt
auth.pgsql.password = mqtt
auth.pgsql.database = mqtt
auth.pgsql.encoding = utf8
auth.pgsql.ssl = false

and i try to load the plugin:
$ emqttd_ctl plugins load emq_auth_pgsql

load plugin error: {emq_auth_pgsql,
{bad_return,
{{emq_auth_pgsql_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emq_auth_pgsql,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{error,fatal,<<"28000">>,
invalid_authorization_specification,
<<"role "root" does not exist">>,
[{file,<<"miscinit.c">>},
{line,<<"494">>},
{routine,
<<"InitializeSessionUserId">>}]}}}}}}}}},
[{emq_auth_pgsql_app,start,2,
[{file,"src/emq_auth_pgsql_app.erl"},{line,33}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,273}]}]}}}}}

when i check the crash.log, i found this message:

2017-07-19 15:20:59 =CRASH REPORT====
crasher:
initial call: ecpool_worker:init/1
pid: <0.1794.0>
registered_name: []
exception exit: {{error,fatal,<<"28000">>,invalid_authorization_specification,<<"role "root" does not exist">>,[{file,<<"miscinit.c">>},{line,<<"494">>},{routine,<<"InitializeSessionUserId">>}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,344}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}
ancestors: [<0.1793.0>,<0.1791.0>,emq_auth_pgsql_sup,<0.1789.0>]
messages: [{'EXIT',<0.1795.0>,normal}]
links: [<0.1793.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 376
stack_size: 27
reductions: 181
neighbours:
2017-07-19 15:20:59 =SUPERVISOR REPORT====
Supervisor: {<0.1793.0>,ecpool_worker_sup}
Context: start_error
Reason: {error,fatal,<<"28000">>,invalid_authorization_specification,<<"role "root" does not exist">>,[{file,<<"miscinit.c">>},{line,<<"494">>},{routine,<<"InitializeSessionUserId">>}]}
Offender: [{pid,undefined},{id,{worker,1}},{mfargs,{ecpool_worker,start_link,[emq_auth_pgsql,1,emq_auth_pgsql_cli,[{pool_size,8},{auto_reconnect,1},{host,"127.0.0.1"},{port,5432},{username,"root"},{password,[]},{database,"mqtt"},{encoding,utf8},{ssl,false},{ssl_opts,[]}]]}},{restart_type,transient},{shutdown,5000},{child_type,worker}]

2017-07-19 15:20:59 =SUPERVISOR REPORT====
Supervisor: {<0.1791.0>,ecpool_pool_sup}
Context: start_error
Reason: {shutdown,{failed_to_start_child,{worker,1},{error,fatal,<<"28000">>,invalid_authorization_specification,<<"role "root" does not exist">>,[{file,<<"miscinit.c">>},{line,<<"494">>},{routine,<<"InitializeSessionUserId">>}]}}}
Offender: [{pid,undefined},{id,worker_sup},{mfargs,{ecpool_worker_sup,start_link,[emq_auth_pgsql,emq_auth_pgsql_cli,[{pool_size,8},{auto_reconnect,1},{host,"127.0.0.1"},{port,5432},{username,"root"},{password,[]},{database,"mqtt"},{encoding,utf8},{ssl,false},{ssl_opts,[]}]]}},{restart_type,transient},{shutdown,infinity},{child_type,supervisor}]

2017-07-19 15:20:59 =SUPERVISOR REPORT====
Supervisor: {local,emq_auth_pgsql_sup}
Context: start_error
Reason: {shutdown,{failed_to_start_child,worker_sup,{shutdown,{failed_to_start_child,{worker,1},{error,fatal,<<"28000">>,invalid_authorization_specification,<<"role "root" does not exist">>,[{file,<<"miscinit.c">>},{line,<<"494">>},{routine,<<"InitializeSessionUserId">>}]}}}}}
Offender: [{pid,undefined},{id,emq_auth_pgsql},{mfargs,{ecpool,start_pool,[emq_auth_pgsql,emq_auth_pgsql_cli,[{pool_size,8},{auto_reconnect,1},{host,"127.0.0.1"},{port,5432},{username,"root"},{password,[]},{database,"mqtt"},{encoding,utf8},{ssl,false},{ssl_opts,[]}]]}},{restart_type,permanent},{shutdown,5000},{child_type,supervisor}]

2017-07-19 15:20:59 =CRASH REPORT====
crasher:
initial call: application_master:init/4
pid: <0.1788.0>
registered_name: []
exception exit: {{bad_return,{{emq_auth_pgsql_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{shutdown,{failed_to_start_child,emq_auth_pgsql,{shutdown,{failed_to_start_child,worker_sup,{shutdown,{failed_to_start_child,{worker,1},{error,fatal,<<"28000">>,invalid_authorization_specification,<<"role "root" does not exist">>,[{file,<<"miscinit.c">>},{line,<<"494">>},{routine,<<"InitializeSessionUserId">>}]}}}}}}}}},[{emq_auth_pgsql_app,start,2,[{file,"src/emq_auth_pgsql_app.erl"},{line,33}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,273}]}]}}}},[{application_master,init,4,[{file,"application_master.erl"},{line,134}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}
ancestors: [<0.1787.0>]
messages: [{'EXIT',<0.1789.0>,normal}]
links: [<0.1787.0>,<0.935.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 610
stack_size: 27
reductions: 126
neighbours:

i create the user named 'root' in the pg db, and reload the plug, it WORKS!!!
createuser -sP root
emqttd_ctl plugins load emq_auth_pgsql

Start apps: [emq_auth_pgsql]
Plugin emq_auth_pgsql loaded successfully.

it does NOT use the conf entry 'auth.pgsql.username'?

error using emqttd_plugin_pgsql (from emqttd #279)

hi,
i got this error when try to implement acl using emqttd_plugin_pgsql

Reason:     {function_clause,
                    [{emqttd_acl_pgsql,compile,
                         [{<<"1">>,<<"127.0.0.1">>,<<"chatteroux">>,
                           <<"chatteroux">>,<<"3">>,<<"#">>},
                          []],
                         [{file,"src/emqttd_acl_pgsql.erl"},{line,86}]},
                     {emqttd_acl_pgsql,check_acl,2,
                         [{file,"src/emqttd_acl_pgsql.erl"},{line,53}]},

emqttd_plugin_pgsql version=1.1.2 showing error. Connection Refused: bad user name or password.

The same configuration working on another machine.

Error log :

20:21:23.485 <0.6451.0> [error] Client(mosqsub/11188-it0@-----:34226): Username 'test_consumer' login failed for {function_clause,[{lists,zip,[[],[<<"test_consumer">>]],[{file,"lists.erl"},{line,386}]},{lists,zip,2,[{file,"lists.erl"},{line,386}]},{epgsql,equery,3,[{file,"src/epgsql.erl"},{line,150}]},{emqttd_auth_pgsql,check,3,[{file,"src/emqttd_auth_pgsql.erl"},{line,47}]},{emqttd_access_control,auth,3,[{file,"src/emqttd_access_control.erl"},{line,64}]},{emqttd_protocol,process,2,[{file,"src/emqttd_protocol.erl"},{line,143}]},{emqttd_client,received,2,[{file,"src/emqttd_client.erl"},{line,265}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,615}]}]}

Exception with pgBouncer

Setup

EMQX: v4.0.0
Environment: k8s (installed with Helm)

Problem

emqx_auth_pgsql crashes with postgres server that is accessed via pgbouncer (in Transaction Mode). It used to work fine when using it directly with postgres server.

The crash error is as follows :

([email protected])1> 2020-07-31 06:36:32.039 [error] <<"8_69_esp32_d9ab10">>@10.244.6.56:56432 [Postgres] query '"auth_query"' failed: { │
│                                          invalid_sql_statement_name,                                                                                                              │
│                                          <<"prepared statement \"auth_query\" does not exist">>,                                                                                  │
│                                          [{file,<<"prepare.c">>},                                                                                                                 │
│                                           {line,<<"512">>},                                                                                                                       │
│                                           {routine,                                                                                                                               │
│                                            <<"FetchPreparedStatement">>},                                                                                                         │
│                                           {severity,<<"ERROR">>}]}  

Should changing the mode of pgBouncer to Session or something else help with this issue?

Load plugins fail.I am sure my password in conf for postgresSql is correct,but the log say the password is invalid.

load plugin error: {emqx_auth_pgsql,
{bad_return,
{{emqx_auth_pgsql_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emqx_auth_pgsql,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
{badmatch,{error,invalid_password}}}}}}}}}},
[{emqx_auth_pgsql_app,start,2,
[{file,
"c:/emqx/ce/emqx-rel/_build/emqx/lib/emqx_auth_pgsql/src/emqx_auth_pgsql_app.erl"},
{line,35}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,277}]}]}}}}}

Issues with loading postgres auth plug in

Environment

OS: 3.13.0-74-generic GNU/Linux
Erlang/OTP:
sysdescr : Erlang MQTT Broker
version : 2.1.0
EMQ:
Description

I am not able to load either mysql or postrgress authentication plug ins, Here is the error trace when I tried to load pgsal auth plug in (similar error with mysql as well)

Verified that Postrgress URL , db username and password are correctly stated in the plug in configuration file.

root@ip-10-0-1-152:/home/ubuntu/emqttd# ./bin/emqttd_ctl plugins load emq_auth_pgsql
load plugin error: {emq_auth_pgsql,
{bad_return,
{{emq_auth_pgsql_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emq_auth_pgsql,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
econnrefused}}}}}}}},
[{emq_auth_pgsql_app,start,2,
[{file,"src/emq_auth_pgsql_app.erl"},{line,33}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,273}]}]}}}}}

Any help is appreciated

emqx-auth-pgsql instruction error

access: Operations that the client can perform. '1' means that the client can publish to a topic, '2' means that the client can subscribe to a topic, '3' means that the client can subscribe and can publish to a topic.

the right instruction is:
'1' means that the client can subscribe to a topic, '2' means that the client can publish to a topic,

Auto reconnect plugins

Hi @Erylee

how to plugins can auto reconnect to pgsql?
plugins has loaded, when postgresql down and postgresql up again. but plugins emqttd_plugin_pgsql false.

I must restart service emqttd for loaded plugins again.

Authentication fails while using Bcypt

User authentication fails while using Bcrypt password hash on pgsql plugin. Plain password works fine. I have tried 10 and 12 cycles, but authentication still fails.
I am using emqtt stable 2.2 on ubuntu 16.04.

Load plugins fail

I got this error, When load plugins :

load plugin error:

{emqttd_plugin_pgsql,
                    {bad_return,
                     {{emqttd_plugin_pgsql_app,start,[normal,[]]},
                      {'EXIT',
                       {{badmatch,
                         {error,
                          {shutdown,
                           {failed_to_start_child,
                            {emqttd_pgsql_pool_sup,pgauth},
                            {exists,
                             [{gproc_pool,call,1,
                               [{file,"src/gproc_pool.erl"},{line,599}]},
                              {emqttd_pgsql_pool_sup,init,1,
                               [{file,"src/emqttd_pgsql_pool_sup.erl"},
                                {line,45}]},
                              {supervisor,init,1,
                               [{file,"supervisor.erl"},{line,272}]},
                              {gen_server,init_it,6,
                               [{file,"gen_server.erl"},{line,328}]},
                              {proc_lib,init_p_do_apply,3,
                               [{file,"proc_lib.erl"},{line,239}]}]}}}}},
                        [{emqttd_plugin_pgsql_app,start,2,
                          [{file,"src/emqttd_plugin_pgsql_app.erl"},
                           {line,40}]},
                         {application_master,start_it_old,4,
                          [{file,"application_master.erl"},{line,273}]}]}}}}}

Broker need restart after making change to emq_auth_psql.conf

Here is the case:
Running Broker for the first time,
Changing emq_auth_psql.conf to suit my database credentials
load module failed because the broker didn't read the new emq_auth_psql.conf it still use the default credentials 'root' with an empty password
Solution: restart broker and it will take the new emq_auth_psql.conf

Someone has mentioned this before #61
I posted it here so new user like me know about it
You should mention this in the online doc

authenticate with bcrypt

I have rails project using bcrypt for encode password, But how can I using bcrypt with this project?

Trouble autenticate

Hi,

I am not able to authenticate using this plugins.

I have the following tables inside the database:

select * from mqtt_acl;                                      
 id | allow |  ipaddr   | username  | clientid | access | topic  
----+-------+-----------+-----------+----------+--------+--------
  1 |     1 |           | $all      |          |      3 | #
  2 |     0 |           | $all      |          |      1 | $SYS/#
  3 |     1 |           | $all      |          |      3 | eq #
  5 |     1 | 127.0.0.1 |           |          |      2 | $SYS/#
  6 |     1 | 127.0.0.1 |           |          |      2 | #
  7 |     1 |           | dashboard |          |      1 | $SYS/#
  8 |     1 |           | simo      |          |      3 | /ciao
(7 rows)

select * from mqtt_user;
 id | is_superuser | username | password | salt 
----+--------------+----------+----------+------
(0 rows)

Now, if I try to subscribe using mosquitto to the topic /ciao I get:

simo@simo:~$ mosquitto_sub -t /ciao -u simo -P simo
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t /ciao -u lois
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t /ciao
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t /test
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t test
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t test -u simo -P simo
Connection Refused: bad user name or password.
^C
simo@simo:~$ mosquitto_sub -t test -u simo 
Connection Refused: bad user name or password.
^C

However, in my understanding, all the subscribtion should match, at least the row id=1 of mqtt_acl.

What I am missing=

emqx_auth_pgsql插件启动失败,为什么?

错误提示:
Load plugin emqx_auth_pgsql error: {emqx_auth_pgsql,
{bad_return,
{{emqx_auth_pgsql_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,
emqx_auth_pgsql,
{shutdown,
{failed_to_start_child,
worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
invalid_authorization_specification}}}}}}}},
[{emqx_auth_pgsql_app,start,2,
[{file,"emqx_auth_pgsql_app.erl"},
{line,37}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},
{line,277}]}]}}}}}.
为什么emqx_auth_pgsql插件启动不了?

Acl table problem

mqtt_acl table
allow: 1, ipaddr: null, username: master, clientid: null, access: 1, topic: test
allow: 0, ipaddr: null, username: $all, clientid: null, access: 2, topic: test

auth.pgsql.acl_query
select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'

I created the table and the query as above. But I want user 'master' to be able to read the test topic but not write it. I looked at the documents on the internet but could not find any solution.
Where am I making a mistake?

Acl Deny Problem

Hi,

I have created mqtt_acl table as documented. I inserted a row for topic '/World' using following query;

INSERT INTO mqtt_acl (id, allow, ipaddr, username, clientid, access, topic)
VALUES (1,1,NULL,'416',NULL,2,'/World');

And then I subscribed and published to this topic. No message received. Secondly I changed username with NULL and clientid with value, not a chance.

I couldn't work with this plugin properly. What am I missing?

Any advice or guidance would be appreciated. Thanks for your help.

LOGS:
12:56:34.981 <0.356.0> [error] Client(C_1455731414614@): Cannot SUBSCRIBE [{<<"/World">>,0}] for ACL Deny
12:56:36.915 <0.356.0> [error] Client(C_1455731414614@:53015): Cannot publish to /World for ACL Deny

How to use SSL option

Is there an example for the following settings ? Is the cacertfile used for server validation?
##auth.pgsql.ssl_opts.keyfile =
##auth.pgsql.ssl_opts.certfile =
##auth.pgsql.ssl_opts.cacertfile =

How to set the sslmode for connection to postgres.
sslmode=require/verifyCA/verify-full etc.

Unable create users for authentication using Postgres in emqx

Hello,

I'm using postgres as authenicator and authorization in emqx (V5.3). When I tried to add a user by using below curl command, it was giving me bad request. For your reference, I'm attaching the request and response

Request:

curl --location 'http://localhost:18083/api/v5/authentication/password_based:postgresql/users' \
--header 'accept: application/json' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic <token>' \
--data '{
  "password": "12345678",
  "user_id": "user4"
}'

Response:
code: 400

{
    "code": "BAD_REQUEST",
    "message": "User error: unsupported_operation"
}

FYI,

  1. I have tried by inserting the values into db into their respective table. It is working as intended.
  2. I have used above curl command by changing the authenicator id to in-built-database. It also worked for me.

But with postgresql I'm getting the above error

If anyone is aware of this issue, please help me out

Thanks

error using same query with latest update plugin

Hi, i got this error when update to the latest emqttd_plugin_pgsql. This plugins run well before i update emqttd + this plugins to the latest.

mosquitto_pub -h 192.168.0.5 -p 1883 -q 1 -t 'topics/ok' -u test -P test -m ok -d
Client mosqpub/20754-Athena sending CONNECT
Client mosqpub/20754-Athena received CONNACK
Client mosqpub/20754-Athena sending PUBLISH (d0, q1, r0, m1, 'topics/ok', ... (2 bytes))
Error: The connection was lost.

emqttd_sasl.log

=SUPERVISOR REPORT==== 23-Nov-2015::01:32:53 ===
     Supervisor: 'esockd_connection_sup - <0.329.0>'
     Context:    connection_crashed
     Reason:     {badarg,
                     [{erlang,binary_to_list,[null],[]},
                      {emqttd_acl_pgsql,b2l,1,
                          [{file,"src/emqttd_acl_pgsql.erl"},{line,128}]},
                      {emqttd_acl_pgsql,who,3,
                          [{file,"src/emqttd_acl_pgsql.erl"},{line,98}]},
                      {emqttd_acl_pgsql,compile,2,
                          [{file,"src/emqttd_acl_pgsql.erl"},{line,89}]},
                      {emqttd_acl_pgsql,check_acl,2,
                          [{file,"src/emqttd_acl_pgsql.erl"},{line,53}]},
                      {emqttd_access_control,check_acl,4,
                          [{file,"src/emqttd_access_control.erl"},{line,102}]},
                      {emqttd_protocol,check_acl,3,
                          [{file,"src/emqttd_protocol.erl"},{line,402}]},
                      {emqttd_protocol,process,2,
                          [{file,"src/emqttd_protocol.erl"},{line,185}]}]}
     Offender:   [{pid,<0.7391.0>},
                  {name,connection},
                  {mfargs,
                      {emqttd_client,start_link,
                          [[{packet,
                                [{max_clientid_len,1024},
                                 {max_packet_size,65536}]},
                            {client,[{idle_timeout,20}]},
                            {session,
                                [{max_inflight,100},
                                 {unack_retry_interval,60},
                                 {await_rel_timeout,20},
                                 {max_awaiting_rel,0},
                                 {collect_interval,0},
                                 {expired_after,48}]},
                            {queue,
                                [{max_length,100},
                                 {low_watermark,0.2},
                                 {high_watermark,0.6},
                                 {queue_qos0,true}]}]]}}]

this is my config before update :

[
  {epgsql, [
      {pools, [
          {pgauth, [
              {size, 2},
              {host, "192.168.0.10"},
              {port, 5432},
              {username,  "emqttd"},
              {password,  "emqttd"},
              {database,  "emqttd"},
              {encoding,  utf8}
          ]}
      ]}
  ]},

  {emqttd_plugin_pgsql, [
    {authquery, "select password_hash, '54qXAd2A8' as salt from users where user_name = '%u' limit 1"},
    {password_hash, {salt, md5}},
    {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl
                 where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"},
    {acl_nomatch, deny}
  ]}
].

this is my config after update :

[
  {epgsql, [
      {pools, [
          {pgauth, [
              {size, 2},
              {host, "192.168.0.10"},
              {port, 5432},
              {username, "emqttd"},
              {password, "emqttd"},
              {database, "emqttd"},
              {encoding,  utf8},
              {reconnect, 5}
          ]}
      ]}
  ]},

  {emqttd_plugin_pgsql, [
    {authquery, "select password_hash, '54qXAd2A8' as salt from users where user_name = '%u' limit 1"},
    {password_hash, {salt, md5}},
    {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl
                 where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"},
    {acl_nomatch, deny}
  ]}
].

prepared statement problems

plugin seems to fail some when receiving frequent connections.
The postgres server returns these messages:

ERROR: bind message supplies 3 parameters, but prepared statement "" requires 1
STATEMENT: select password from mqtt_users where username=$1 and enabled=1 limit 1
ERROR: bind message supplies 3 parameters, but prepared statement "" requires 1
STATEMENT: select password from mqtt_users where username=$1 and enabled=1 limit 1
ERROR: unnamed prepared statement does not exist
ERROR: unnamed prepared statement does not exist
ERROR: bind message supplies 3 parameters, but prepared statement "" requires 1
STATEMENT: select password from mqtt_users where username=$1 and enabled=1 limit 1
ERROR: bind message supplies 3 parameters, but prepared statement "" requires 1
STATEMENT: select password from mqtt_users where username=$1 and enabled=1 limit 1
ERROR: unnamed prepared statement does not exist
ERROR: bind message supplies 1 parameters, but prepared statement "" requires 3
STATEMENT: select allow, ipaddr, username, clientid, access, topic from mqtt_acl where enabled=1 and (ipaddr = $1 or username = $2 or username = '$all' or clientid = $3) order by id asc
ERROR: unnamed prepared statement does not exist
ERROR: unnamed prepared statement does not exist

Changing the pool size does not solve the problem.

Exceptions in emq_auth_pgsql plugin

Hi,

while testing our EMQ deployment in production i discovered that emq_auth_pgsql plugin throws errors when connecting to the broker "from time to time". That is, sometimes the sql query cannot be executed correctly i guess.

Sometimes i get
2018-03-27 13:00:24.849 [error] <0.1909.0>@emqttd_protocol:process:234 Client([email protected]:63100): Username '0Tz9s' login failed for {error,error,<<"26000">>,invalid_sql_statement_name,<<"unnamed prepared statement does not exist">>,[{file,<<"postgres.c">>},{line,<<"1487">>},{routine,<<"exec_bind_message">>},{severity,<<"ERROR">>}]}

Other times the client can connect without any problem.

Sometimes i get:
2018-03-27 13:01:19.101 [error] <0.1959.0>@emqttd_protocol:process:287 Client([email protected]:63171): Cannot SUBSCRIBE [{<<"1.1/R/5FxrX">>,[{qos,0}]}] for ACL Deny

other times it works. Probably a problem regarding epgsql? Probably related to epgsql/epgsql#150

MQTT was disconnected due to badmatch error

I use mqtt acl table as follows

id allow ipaddr username clientid access topic
1 1 NULL test NULL 3 #

The following configuration has also been modified

allow_anonymous = false
acl_nomatch = deny
sed -i '/{allow, all}./d' /opt/emqx/etc/acl.conf  # delete {allow, all}.

and use emqx-auth-pgsql to certification.

There is no problem with the superuser but the following problem occurs with normal users such as test

Python MQTT as follows

def on_connect(client, userdata, flags, rc):
    logger.info("Connected with result code " + str(rc))
    if rc == 5:
        logger.critical("auth failed")
    client.subscribe(f"device1/#",2)
client = mqtt.Client(client_id = client_id)
client.on_connect = on_connect
client.username_pw_set(test,test)
client.connect(ip, port, 60)
client.loop_forever()

When I execute my python code, I keep output like this

INFO:main:Connected with result code 0
INFO:main:Connected with result code 0
INFO:main:Connected with result code 0
....

I subscribed to the system topic $SYS/brokers/+/clients/# and found that the python client had been connecting and then disconnected

{'clean_start': True, 'clientid': 'client_id', 'connack': 0, 'ipaddress': 'xxxx', 'keepalive': 60, 'proto_name': 'MQTT', 'proto_ver': 4, 'ts': 1587053466406, 'username': 'test'}
{'clientid': 'agent_eVEoS49gxJtspEKE2vkJtW', 'username': 'test', 'reason': 'badmatch', 'ts': 1587053466407}
.....

Where did I misconfigure this badmatch disconnect problem?

I used an emqx:v3.2.4 docker image

[documentation request] Autoreconnect to Postgres

Currently after the initial load, the plugin attempts reconnection to postgre server. But the technical aspects of this reconnection are not documented. Sample queries below.

  1. Is there max limit in time or count to this retry attempt?
  2. What type of retry is this: a)IMMEDIATE b)FIXED_INTERVAL c)RANDOM_INTERVAL d)LINEAR_BACKOFF e)EXPONENTIAL_BACKOFF f)EXPONENTIAL_BACKOFF_WITH_JITTER g)...

Documentation for generating salted password lacking

The documentation describes the process of how to generate a salted password. However, the description on point 2 really makes it very difficult to follow. It is described that "same salting rules and hash method as MySQL authentication" should be used. Presumably, this is the method referenced. This method however utilizes SHA1, whereas emqx_auth_pgsql also supports other cryptographic algorithms. In addition, there is no description about how the salt enters the equation (presumably in place of the randomized 20-bit sequence).

This really needs to be improved. I still cannot wrap my head around how to generate hashed passwords and salts in order to make the plugin able to authenticate.

IMHO, the plugin should use the methods available for cryptography in PostgreSQL, i.e. the pgcrypto crypto method, specifically designed for managing cryptography for passwords.

Cannot publish for ACL deny

Using 2.3.5 I am getting the following a lot:

Client([email protected]:1234): Cannot publish to 1234/abc for ACL Deny

Even though I have a row with clientid $all, access 2 and topic #. I am using emq-docker with the following env set:

EMQ_AUTH__PGSQL__ACL_QUERY
select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'

Any ideas?

How to use ACL without authentication based only on client id

I am using EMQ with the emq_sn plugin. I have activated the emq-auth-pgsql plugin to achieve ACL based on client id. However, as reported in issue #18, the broker denies access for clients that are not authenticated (i.e. with username and password).

To my best knowledge, MQTT-SN does not support authentication (at least it is not specified). This makes the emq-auth-pgsql plugin unusable with MQTT-SN.

is there a chance of decoupling auth and acl or configuring the plugin such that it allows for unauthenticated access?

PBKDF2 Implementation

Hi, I am trying to change our broker to emqttd. I had two issues to make it work with our system and I need some guidance to solve these problems.

With plain password settings, broker is working as expected. I can authenticate with username and password. However I couldn't make configuration for pbkdf2. It failed to match username and password.

this is a sample password data: PBKDF2**$sha256$32000$EELHOiLFWI9boYh6uHzbrCtMXW+2KJR2$**6XkSiMwMy0Tu7OT6eNg/1YPUWL1pPF2t95B0qjpSihclzPb+iI2i+GxZ3zSvhW3kgc2cYjXA==

I am sending username and plain password to connect, but it couldn't match with this.

How I am gonna design table structure for pbkdf2 password lookup? Should I separate salt and hash to 2 columns or I can make it work with some config options that I didn't know yet?

Thanks for your help.

Have a nice day.

epgsql no such file or directory

load plugin error: {epgsql,{"no such file or directory","epgsql.app"}}

using instruction in the latest readme to build the plugins and got that error when load emqttd_plugin_pgsql.

I think its still need epgsql in plugins directory.

git submodule add https://github.com/epgsql/epgsql.git plugins/epgsql

emqtt postgres plug-in load error

when i try to load plugin "./bin/emqttd_ctl plugins load emq_auth_pgsql " , it gives error.
load plugin error: {emq_auth_pgsql, {bad_return, {{emq_auth_pgsql_app,start,[normal,[]]}, {'EXIT', {{badmatch, {error, {shutdown, {failed_to_start_child,emq_auth_pgsql, {shutdown, {failed_to_start_child,worker_sup, {shutdown, {failed_to_start_child, {worker,1}, invalid_password}}}}}}}}, [{emq_auth_pgsql_app,start,2, [{file,"src/emq_auth_pgsql_app.erl"},{line,33}]}, {application_master,start_it_old,4, [{file,"application_master.erl"},{line,273}]}]}}}}}

(sorry for the link, could not format file properly) : https://pastebin.com/1VaRVHZP

Password salt support

When trying to configure the plugin to use:
auth.pgsql.password_hash = salt sha256
the broker refuses to start with the following error:
"salt sha256" is not a valid enum value, acceptable values are: plain, md5, sha, sha256, pbkdf2

Reviewing the code in
https://github.com/emqtt/emq-auth-pgsql/blob/master/priv/emq_auth_pgsql.schema
it seems that the option of salting passwords it is not supported, although docs and example configuration files include it. In addition, it is not clear to me how this would be handled in https://github.com/emqtt/emq-auth-pgsql/blob/master/src/emq_auth_pgsql.erl (the code in the mysql plugin is a little bit different).

Other configuration options shuch as "plain" or "sha256" are working ok. Is password salt a work in progress or am i missing something?

Environment:

  • Version: 2.0.7
  • Platform: Ubuntu 16.04

Best regards

Plugin can't load : invalid_authorization_specification

I filled my db parameters in /etc/plugin.config, but I get this error when I try to load plugin

{emqttd_plugin_pgsql,
{bad_return,
{{emqttd_plugin_pgsql_app,start,[normal,[]]},
{'EXIT',
{{badmatch,
{error,
{shutdown,
{failed_to_start_child,emqttd_plugin_pgsql,
{shutdown,
{failed_to_start_child,worker_sup,
{shutdown,
{failed_to_start_child,
{worker,1},
invalid_authorization_specification}}}}}}}},
[{emqttd_plugin_pgsql_app,start,2,
[{file,"src/emqttd_plugin_pgsql_app.erl"},
{line,34}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,273}]}]}}}}}

here is the plugin details in emqtt_ctl plugins list command:
Plugin(emqttd_plugin_pgsql, version=1.1.2, description=emqttd PostgreSQL Plugin, active=false)

How to configure auth.pgsql.password_hash for django password

I want emqx to authenticate my Django user. Django uses PBKDF2 with sha356 + 150000 iterations + salt. Django has a make_password method to hash plain password in a secure way.

>>> from django.contrib.auth.hashers import make_password
>>> make_password("hello world")
'pbkdf2_sha256$150000$YSu4ev2bmi9t$YieK5QrYzdlktW9E1EymOjY3T0RUbF8EaPoR57otsq4='

it is in the following format:

<algorithm>$<iterations>$<salt>$<hash>

Check this for more info: https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#how-django-stores-passwords.

I can break the string into subsequent components and store them in separate columns.

How should I configure auth.pgsql.password_hash in this case??

I am thinking of something the following:

auth.pgsql.password_hash = salt,pbkdf2,sha256,150000,32
auth.pgsql.auth_query = select password from mqtt_user where username = '%u' limit 1

Will it work? Is the configuration correct? How will emqx know what the salt is? Or as of now using salt with pbkdf2 is not supported? Or should I store the salt with password hash in one column only with some separator(or without separator)? Also, should the hash and salt both be base64 encoded?

SSL options to connect to pgsql

I think that it can be useful to have the possibility to specify the ssl options while configuring the connection to the pgsql database.

Now it is possible to enable ssl connections but it does not seem possible to enable CA verification and full verification of the server certificate (see libpq ssl support).

The epgsql library seems to support this kind of configuration through the ssl_opts in the connect_options of the connect method (see connect example).

I think that this enhancement would require some modifications of the configuration file processing and in the code of the method conn_opts in emq_auth_pgsql_cli.erl.

Best regards.

k8s cluster restart plugin

Hi
i have a Kubernetes cluster with 2 emqx nodes.
if a pod go down and the deployment get it up again, is there a way to start up the plugin too?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.