Coder Social home page Coder Social logo

joyieldinc / predixy Goto Github PK

View Code? Open in Web Editor NEW
1.4K 76.0 342.0 1.43 MB

A high performance and fully featured proxy for redis, support redis sentinel and redis cluster

License: BSD 3-Clause "New" or "Revised" License

Makefile 0.53% C++ 92.49% C 0.67% Python 6.31%
redis redis-cluster redis-sentinel twemproxy codis corvus

predixy's Introduction

Predixy 中文版

Predixy is a high performance and fully featured proxy for redis sentinel and redis cluster

Features

  • High performance and lightweight.
  • Multi-threads support.
  • Works on Linux, OSX, BSD, Windows(Cygwin).
  • Supports Redis Sentinel, single/multi redis group[s].
  • Supports Redis Cluster.
  • Supports redis block command, eg:blpop, brpop, brpoplpush.
  • Supports scan command, even multi redis instances.
  • Multi-keys command support: mset/msetnx/mget/del/unlink/touch/exists.
  • Multi-databases support, means redis command select is avaliable.
  • Supports redis transaction, limit in Redis Sentinel single redis group.
  • Supports redis Scripts, script load, eval, evalsha.
  • Supports redis Pub/Sub.
  • Multi-DataCenters support, read from slaves.
  • Extend AUTH, readonly/readwrite/admin permission, keyspace limit.
  • Log level sample, async log record.
  • Log file auto rotate by time and/or file size.
  • Stats info, CPU/Memory/Requests/Responses and so on.
  • Latency monitor.

Build

Predixy can be compiled and used on Linux, OSX, BSD, Windows(Cygwin). Requires C++11 compiler.

It is as simple as:

$ make

To build in debug mode:

$ make debug

Some other build options:

  • CXX=c++compiler, default is g++, you can specify other, eg:CXX=clang++
  • EV=epoll|poll|kqueue, default it is auto detect according by platform.
  • MT=false, disable multi-threads support.
  • TS=true, enable predixy function call time stats, debug only for developer.

For examples:

$ make CXX=clang++
$ make EV=poll
$ make MT=false
$ make debug MT=false TS=true

Install

Just copy src/predixy to the install path

$ cp src/predixy /path/to/bin

Configuration

See below files:

  • predixy.conf, basic config, will refrence below config files.
  • cluster.conf, Redis Cluster backend config.
  • sentinel.conf, Redis Sentinel backend config.
  • auth.conf, authority control config.
  • dc.conf, multi-datacenters config.
  • latency.conf, latency monitor config.

Running

$ src/predixy conf/predixy.conf

With default predixy.conf, Predixy will listen at 0.0.0.0:7617 and proxy to Redis Cluster 127.0.0.1:6379. In general, 127.0.0.1:6379 is not running in Redis Cluster mode. So you will look mass log output, but you can still test it with redis-cli.

$ redis-cli -p 7617 info

More command line arguments:

$ src/predixy -h

Stats

Like redis, predixy use INFO command to give stats.

Show predixy running info and latency monitors

redis> INFO

Show latency monitors by latency name

redis> INFO Latency <latency-name>

A latency monitor example:

LatencyMonitorName:all
            latency(us)   sum(us)           counts
<=          100              3769836            91339 91.34%
<=          200               777185             5900 97.24%
<=          300               287565             1181 98.42%
<=          400               185891              537 98.96%
<=          500               132773              299 99.26%
<=          600                85050              156 99.41%
<=          700                85455              133 99.54%
<=          800                40088               54 99.60%
<=         1000                67788               77 99.68%
>          1000               601012              325 100.00%
T            60              6032643           100001
The last line is total summary, 60 is average latency(us)

Show latency monitors by server address and latency name

redis> INFO ServerLatency <server-address> [latency-name]

Reset all stats and latency monitors, require admin permission.

redis> CONFIG ResetStat

Benchmark

predixy is fast, how fast? more than twemproxy, codis, redis-cerberus

See wiki benchmark

License

Copyright (C) 2017 Joyield, Inc. <joyield.com#gmail.com>

All rights reserved.

License under BSD 3-clause "New" or "Revised" License

WeChat:cppfan wechat

predixy's People

Contributors

crierr avatar fortrue avatar jeaklee avatar joyield avatar leenr avatar mosquito avatar shuaiming avatar yoonian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

predixy's Issues

about thread-safe

Can you push some document about predixy's design?
I read your code, but still don't know your design, I worry not thread-safe when push request to backend from multi client and push multi response to client from different backend redis . and either merge multi reply, then send to client.

有一些整体设计的文档么?
我简单看了下代码,还没搞懂你的设计,我不知道是否存在线程安全问题:
1、当从不同客户端读取到请求然后转发给某一个redis 后端时,如果此redis后端链接和clien他不在同一个Handler线程。
2、当从多个redis后端收到reply,这些reply属于同一个client时,怎么将reply(线程安全的)告诉client 。
3、 来自client的某个request可能涉及到多个redis后端进行处理,那么收到后者的reply之后是否存在merge处理?
4、当一个client存在多个pending的请求:A、B、C。 如果他们分别被不同的redis 后端处理,那么当B对应redis server先处理完毕,它的reply先返回,是否考虑了顺序问题(也就是等B之前的请求,也就是A完成了才能将B告诉client)?

redis-cluster 集群节点下线后,predixy 无法检测, 造成集群不可用

2018-09-28 11:53:19.256012 N Handler.cpp:265 server 172.31.171.161:6382 mark failure
2018-09-28 11:53:19.256014 N Handler.cpp:276 h 3 close s 172.31.171.161:6382 33 and c None -1 with status 4 EventError
2018-09-28 11:53:19.256276 W ClusterServerPool.cpp:109 redis cluster nodes get node invalid 5d767a972f3490cad28cbd9c6c873f4745f5b75f slave,fail,noaddr 1cd473ba77b7eabe78129f99b4f3cf391916c9ef
2018-09-28 11:53:20.257144 N ConnectConnectionPool.cpp:190 h 0 check server reopen connection 172.31.171.161:6382 33
2018-09-28 11:53:20.257192 E Handler.cpp:435 h 0 s 172.31.171.161:6382 33 will be close with status 4 EventError
2018-09-28 11:53:20.257196 N Handler.cpp:265 server 172.31.171.161:6382 mark failure
2018-09-28 11:53:20.257198 N Handler.cpp:276 h 0 close s 172.31.171.161:6382 33 and c None -1 with status 4 EventError
2018-09-28 11:53:20.257509 W ClusterServerPool.cpp:109 redis cluster nodes get node invalid 5d767a972f3490cad28cbd9c6c873f4745f5b75f slave,fail,noaddr 1cd473ba77b7eabe78129f99b4f3cf391916c9ef
2018-09-28 11:53:21.258685 N ConnectConnectionPool.cpp:190 h 0 check server reopen connection 172.31.171.161:6382 33
2018-09-28 11:53:21.258821 N Handler.cpp:1371 h 0 s 172.31.171.161:6382 33 mark server alive
2018-09-28 11:53:37.180128 N ConnectConnectionPool.cpp:50 h 3 reopen server connection 172.31.171.161:6382 34
2018-09-28 11:53:38.181339 N Handler.cpp:454 h 2 create connection pool for server 172.31.171.161:6382
2018-09-28 11:53:38.181392 N ConnectConnectionPool.cpp:42 h 2 create server connection 172.31.171.161:6382 35
2018-09-28 11:56:06.369964 N ConnectConnectionPool.cpp:50 h 1 reopen server connection 172.31.171.161:6382 36

erlang redis 驱动auth失败

Command = "AUTH password\r\n",
gen_tcp:send(Socket, Command),

已验证过连redis-server上是可以的

连predixy报错如下:
{error,{unexpected_data,{ok,"-ERR unknown command 'auth'\r\n"}}}

是不是predixy对auth这块的解析和redis-server有区别?
还是auth的协议有变化?

无故断掉

部署在服务器上第二天发现自己断开了,怎么解呢

请问下predixy的集群如何做啊

predixy集群的话,是通过LB做负载均衡做呢,还是业务方代码里面配多个predixy的地址。 想问下推荐哪种方法比较好?

Bug in handling offline shard

Hello! I guess I've faced a bug while testing the Predixy with redis-4.0.6.

I

Steps to reproduce:

  1. Set up 3 - nodes Redis cluster, no replicas, masters only
  2. Set up Predixy on each nodes with Redis cluster, to make a cross-connections.
    Direct each Predixy to three Redis nodes.
  3. Turn off the server with one of the Redis nodes by power button (turn off VM) .
  4. Predixy wouldn't mark turned off Redis node as fail. No log entry like this appears:
    "2018-02-21 22:18:48.555883 N Handler.cpp:231 server 192.168.10.121:6383 mark failure"

Also if some element that should be located on turned off node will be requested, Predixy will just hang, not capable to process the MOVED command received from cluster.

II

Also currently if one of Redis nodes was just stopped as a service without turning off the server (let say, 192.168.10.121:6383 ),
Predixy will mark the node failure and will return an error on request of element located on stopped node:

192.168.10.120:7000> get sdssd444e93
(error) ERR no server connection available

I think these MOVED replies pointing to unavailable nodes should return just "(nil)", not custom error as now.

192.168.10.122:6385> get sdssd444e93
(error) MOVED 10763 192.168.10.121:6383

如何支持无sentinel和cluster

我有多台redis server,如何支持无sentinel和cluster,直接透传到后面的所有机器上。找了好久没找到配置

不支持flushdb、dbsize等命令

redis客户端单连集群中的某个节点,支持flushdb命令,但不影响其他节点。若要每个都清空,得一个一个清空,甚是麻烦。能否支持flushdb命令,全部清除?
像这些集群中单个节点支持的命令,比如dbsize、keys……,后续能否得到支持?

运行一段时间后会僵死

使用的是 1.0.4a 版,配置了redis cluster 和 dc。部分参数:

ClusterServerPool {
        MasterReadPriority 50
        DynamicSlaveReadPriority 50
        RefreshInterval 1s
        ServerTimeout 1s
        ServerFailureLimit 10
        ServerRetryTimeout 1s
        KeepAlive 300s
        Servers {
        ……
        }
}
Name Predixy
Bind 0.0.0.0:7617
WorkerThreads 8
MaxMemory 10G
ClientTimeout 300
Log /home/logs/predixy.log
LogRotate 1d
LogVerbSample 0
LogDebugSample 0
LogInfoSample 0
LogNoticeSample 1
LogWarnSample 1
LogErrorSample 1
Include auth.conf
Include cluster.conf
LocalDC 10_1_5
Include dc.conf
Include latency.conf

表现为 tcp 端口可连上,但输入任何命令都无响应,同时从日志看到连接数不断上升。使用 kill 无法使其退出,日志显示连接都断开,只能用 kill -9 强退。注意到发生这种情况似乎都是由一个 Error 导致的,都是2018-04-28 02:34:30.842877 E Backtrace.h:18 predixy backtrace(11)

2018-04-28 01:44:40.184035 N Handler.cpp:318 h 6 accept c 127.0.0.1:56795 325
2018-04-28 01:44:40.184151 N Handler.cpp:212 h 0 remove c 127.0.0.1:56795 325 with status 2 End
2018-04-28 01:44:45.202704 N Handler.cpp:318 h 7 accept c 127.0.0.1:57161 325
2018-04-28 01:44:45.202747 N Handler.cpp:212 h 0 remove c 127.0.0.1:57161 325 with status 2 End
2018-04-28 01:44:50.225166 N Handler.cpp:318 h 5 accept c 127.0.0.1:57521 325
2018-04-28 01:44:50.225214 N Handler.cpp:212 h 0 remove c 127.0.0.1:57521 325 with status 2 End
2018-04-28 01:44:55.243163 N Handler.cpp:318 h 6 accept c 127.0.0.1:57897 325
2018-04-28 01:44:55.243207 N Handler.cpp:212 h 0 remove c 127.0.0.1:57897 325 with status 2 End
2018-04-28 01:45:00.261560 N Handler.cpp:318 h 5 accept c 127.0.0.1:58345 325
2018-04-28 01:45:00.261599 N Handler.cpp:212 h 0 remove c 127.0.0.1:58345 325 with status 2 End
**2018-04-28 01:45:04.177890 E Backtrace.h:18 predixy backtrace(11)**
2018-04-28 01:45:05.278381 N Handler.cpp:318 h 6 accept c 127.0.0.1:58682 325
2018-04-28 01:45:05.278431 N Handler.cpp:212 h 0 remove c 127.0.0.1:58682 325 with status 2 End
2018-04-28 01:45:06.482859 N Handler.cpp:318 h 6 accept c 127.0.0.1:58723 325
2018-04-28 01:45:10.295592 N Handler.cpp:318 h 6 accept c 127.0.0.1:58861 332
2018-04-28 01:45:10.295641 N Handler.cpp:212 h 1 remove c 127.0.0.1:58861 332 with status 2 End
2018-04-28 01:45:13.656034 N Handler.cpp:318 h 6 accept c 127.0.0.1:59082 331
2018-04-28 01:45:15.324703 N Handler.cpp:318 h 4 accept c 127.0.0.1:59198 332
2018-04-28 01:45:15.324739 N Handler.cpp:212 h 2 remove c 127.0.0.1:59198 332 with status 2 End
2018-04-28 01:45:20.352982 N Handler.cpp:318 h 6 accept c 127.0.0.1:59444 332
2018-04-28 01:45:20.353043 N Handler.cpp:212 h 2 remove c 127.0.0.1:59444 332 with status 2 End
2018-04-28 01:45:25.369364 N Handler.cpp:318 h 6 accept c 127.0.0.1:59775 332
2018-04-28 01:45:25.369405 N Handler.cpp:212 h 2 remove c 127.0.0.1:59775 332 with status 2 End
2018-04-28 01:45:27.287941 N Handler.cpp:318 h 7 accept c 127.0.0.1:59889 332
2018-04-28 01:45:30.391796 N Handler.cpp:318 h 6 accept c 127.0.0.1:60109 336
2018-04-28 01:45:30.391837 N Handler.cpp:212 h 3 remove c 127.0.0.1:60109 336 with status 2 End
2018-04-28 01:45:35.111581 N Handler.cpp:318 h 6 accept c 10.81.3.16:49063 336
2018-04-28 01:45:35.111781 N Handler.cpp:212 h 3 remove c 10.81.3.16:49063 336 with status 1 None
2018-04-28 01:45:35.408776 N Handler.cpp:318 h 6 accept c 127.0.0.1:60388 336
2018-04-28 01:45:35.408814 N Handler.cpp:212 h 3 remove c 127.0.0.1:60388 336 with status 2 End
2018-04-28 01:45:40.429624 N Handler.cpp:318 h 6 accept c 127.0.0.1:60758 336
2018-04-28 01:45:40.429669 N Handler.cpp:212 h 3 remove c 127.0.0.1:60758 336 with status 2 End
2018-04-28 01:45:40.668865 N Handler.cpp:318 h 7 accept c 127.0.0.1:60778 336
2018-04-28 01:45:45.458702 N Handler.cpp:318 h 6 accept c 127.0.0.1:32884 337
2018-04-28 01:45:45.458740 N Handler.cpp:212 h 4 remove c 127.0.0.1:32884 337 with status 2 End
2018-04-28 01:45:48.287920 N Handler.cpp:318 h 0 accept c 127.0.0.1:33057 337
2018-04-28 01:45:50.479755 N Handler.cpp:318 h 4 accept c 127.0.0.1:33210 339
2018-04-28 01:45:55.496468 N Handler.cpp:318 h 3 accept c 127.0.0.1:33572 338
2018-04-28 01:46:00.517845 N Handler.cpp:318 h 3 accept c 127.0.0.1:33904 340
2018-04-28 01:46:05.534125 N Handler.cpp:318 h 6 accept c 127.0.0.1:34354 341
2018-04-28 01:46:10.562635 N Handler.cpp:318 h 6 accept c 127.0.0.1:34752 342
2018-04-28 01:46:15.580118 N Handler.cpp:318 h 7 accept c 127.0.0.1:35122 343
2018-04-28 01:46:20.599403 N Handler.cpp:318 h 7 accept c 127.0.0.1:35538 345
2018-04-28 01:46:25.622856 N Handler.cpp:318 h 6 accept c 127.0.0.1:35929 344
2018-04-28 01:46:30.652098 N Handler.cpp:318 h 2 accept c 127.0.0.1:36436 346
2018-04-28 01:46:35.155094 N Handler.cpp:318 h 3 accept c 10.81.3.16:51579 347
2018-04-28 01:46:35.673639 N Handler.cpp:318 h 7 accept c 127.0.0.1:36830 348
2018-04-28 01:46:40.691313 N Handler.cpp:318 h 6 accept c 127.0.0.1:37156 349
2018-04-28 01:46:45.798461 N Handler.cpp:318 h 7 accept c 127.0.0.1:37459 350
2018-04-28 01:46:50.828426 N Handler.cpp:318 h 7 accept c 127.0.0.1:37753 351
2018-04-28 01:46:55.858724 N Handler.cpp:318 h 2 accept c 127.0.0.1:37985 352
2018-04-28 01:47:00.879482 N Handler.cpp:318 h 6 accept c 127.0.0.1:38286 353
2018-04-28 01:47:05.908095 N Handler.cpp:318 h 4 accept c 127.0.0.1:38686 354
2018-04-28 01:47:10.925493 N Handler.cpp:318 h 4 accept c 127.0.0.1:39116 355
2018-04-28 01:47:15.942123 N Handler.cpp:318 h 2 accept c 127.0.0.1:39408 356
2018-04-28 01:47:20.971427 N Handler.cpp:318 h 4 accept c 127.0.0.1:39735 357
2018-04-28 01:47:26.000153 N Handler.cpp:318 h 6 accept c 127.0.0.1:40055 358
2018-04-28 01:47:31.017114 N Handler.cpp:318 h 4 accept c 127.0.0.1:40371 359
2018-04-28 01:47:35.166371 N Handler.cpp:318 h 6 accept c 10.81.3.16:53991 360
2018-04-28 01:47:36.033593 N Handler.cpp:318 h 7 accept c 127.0.0.1:40692 361
2018-04-28 01:47:41.062560 N Handler.cpp:318 h 7 accept c 127.0.0.1:41063 362
2018-04-28 01:47:46.079986 N Handler.cpp:318 h 1 accept c 127.0.0.1:41423 363
2018-04-28 01:47:51.108988 N Handler.cpp:318 h 6 accept c 127.0.0.1:41813 364
2018-04-28 01:47:56.126150 N Handler.cpp:318 h 6 accept c 127.0.0.1:42217 365
2018-04-28 01:48:01.146164 N Handler.cpp:318 h 4 accept c 127.0.0.1:42542 366
2018-04-28 01:48:06.175298 N Handler.cpp:318 h 7 accept c 127.0.0.1:42922 367
2018-04-28 01:48:11.198903 N Handler.cpp:318 h 7 accept c 127.0.0.1:43288 368
2018-04-28 01:48:16.221315 N Handler.cpp:318 h 4 accept c 127.0.0.1:43618 369
2018-04-28 01:48:21.239193 N Handler.cpp:318 h 7 accept c 127.0.0.1:44001 370
2018-04-28 01:48:26.268138 N Handler.cpp:318 h 7 accept c 127.0.0.1:44335 371
2018-04-28 01:48:31.296765 N Handler.cpp:318 h 7 accept c 127.0.0.1:44650 372
……
……
2018-04-28 02:12:45.716744 N Handler.cpp:318 h 7 accept c 127.0.0.1:55478 1807
2018-04-28 02:12:50.749294 N Handler.cpp:318 h 7 accept c 127.0.0.1:55487 1808
2018-04-28 02:12:55.778369 N Handler.cpp:318 h 4 accept c 127.0.0.1:55496 1809
2018-04-28 02:13:00.805953 N Handler.cpp:318 h 6 accept c 127.0.0.1:55505 1810
2018-04-28 02:13:05.833758 N Handler.cpp:318 h 3 accept c 127.0.0.1:55514 1811
2018-04-28 02:13:10.868617 N Handler.cpp:318 h 4 accept c 127.0.0.1:55523 1812
2018-04-28 02:13:15.896918 N Handler.cpp:318 h 4 accept c 127.0.0.1:55532 1813
2018-04-28 02:13:20.925598 N Handler.cpp:318 h 4 accept c 127.0.0.1:55541 1815
2018-04-28 02:13:25.960274 N Handler.cpp:318 h 6 accept c 127.0.0.1:55550 1814
2018-04-28 02:13:30.988601 N Handler.cpp:318 h 6 accept c 127.0.0.1:55559 1816
2018-04-28 02:13:35.146914 N Handler.cpp:318 h 7 accept c 10.81.3.16:33842 1817
2018-04-28 02:13:36.028638 N Handler.cpp:318 h 6 accept c 127.0.0.1:55568 1818
2018-04-28 02:13:41.057391 N Handler.cpp:318 h 6 accept c 127.0.0.1:55578 1819
2018-04-28 02:13:46.095323 N Handler.cpp:318 h 6 accept c 127.0.0.1:55587 1820
2018-04-28 02:13:48.413412 N Handler.cpp:76 handler 4 stopped
2018-04-28 02:13:48.465139 N Handler.cpp:76 handler 7 stopped
2018-04-28 02:13:48.496449 N Handler.cpp:76 handler 0 stopped
2018-04-28 02:13:48.498232 N Handler.cpp:76 handler 6 stopped
2018-04-28 02:13:48.498557 N Handler.cpp:76 handler 3 stopped
2018-04-28 02:13:48.498774 N Handler.cpp:76 handler 1 stopped

进程跑一天左右会异常僵死和无异常

版本:predixy predixy-1.0.5-pref
配置参数:
Name PredixyTrade
Bind 0.0.0.0:22152
WorkerThreads 4
MaxMemory 0
ClientTimeout 60
Log /data/srv/store/logs/predixy.log
LogRotate 1d
LogVerbSample 0
LogDebugSample 0
LogInfoSample 10000
LogNoticeSample 1
LogWarnSample 1
LogErrorSample 1
Include auth.conf
Include try.conf
Include latency.conf

auth.conf:
Authority {
Auth {
Mode write
}
Auth "#a complex password#" {
Mode admin
}
}

try.conf:
SentinelServerPool {
Databases 16
Hash crc16
Distribution modula
MasterReadPriority 0
StaticSlaveReadPriority 100
DynamicSlaveReadPriority 100
KeepAlive 120
Sentinels {
+ 0.0.0.1:17021
}
Group configs {
+ 0.0.0.2:22151
+ 0.0.0.3:22151
}
}

刚使用的第一天凌晨0点左右有报错宕机
qq20180529-101621

picture

第二日尝试编写shell脚本让其进程while tre自动运行,凌晨3点宕机无错误日志,重启进程后正常
qq111

More detailed cluster.conf parameters description

Hello, can anyone add to Wiki or project description more details about how these parameters affect the operation of predixy?

  • MasterReadPriority
  • StaticSlaveReadPriority
  • DynamicSlaveReadPriority
  • RefreshInterval
  • ServerFailureLimit
  • ServerRetryTimeout

Also I'm wonder how proxy performs checks of cluster nodes and is it keeps keep-alive connection to them all the time.

Brew

Make it available on HomeBrew, DockerHub.

Tks

predixy crashed with SIGABRT

-- Logs begin at Sun 2018-05-13 02:21:16 +07, end at Tue 2018-05-15 12:42:52 +07. --
May 14 15:05:57 redis04 start-stop-daemon[156313]: *** Error in `/opt/redis/bin/predixy': free(): corrupted unsorted chunks: 0x00007fa85407d500 ***
May 14 15:05:57 redis04 start-stop-daemon[156313]: ======= Backtrace: =========
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fa8777227e5]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fa87772b37a]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fa87772f53c]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Request5clearEv+0x9e7)[0x48a007]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN16AcceptConnection5writeEP7HandlerP5IOVeci+0xe99)[0x453fd9]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN16AcceptConnection10writeEventEP7Handler+0x28)[0x4545f8]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Handler25postAcceptConnectionEventEv+0x80)[0x4aa4a0]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Handler3runEv+0x56)[0x4ad956]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(execute_native_thread_routine+0x20)[0x4e9560]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fa877f9b6ba]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa8777b241d]
May 14 15:05:57 redis04 start-stop-daemon[156313]: ======= Memory map: ========
May 14 15:05:57 redis04 start-stop-daemon[156313]: 00400000-00543000 r-xp 00000000 08:03 36176100                           /opt/redis/bin/predixy
May 14 15:05:57 redis04 start-stop-daemon[156313]: 00743000-00749000 r--p 00143000 08:03 36176100                           /opt/redis/bin/predixy
May 14 15:05:57 redis04 start-stop-daemon[156313]: 00749000-0074b000 rw-p 00149000 08:03 36176100                           /opt/redis/bin/predixy
May 14 15:05:57 redis04 start-stop-daemon[156313]: 0074b000-00760000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 00875000-008da000 rw-p 00000000 00:00 0                                  [heap]
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa83c000000-7fa83c021000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa83c021000-7fa840000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa844000000-7fa844098000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa844098000-7fa848000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa848000000-7fa8480b1000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8480b1000-7fa84c000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa84c000000-7fa84c0b6000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa84c0b6000-7fa850000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa850000000-7fa8500a0000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8500a0000-7fa854000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa854000000-7fa8540a4000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8540a4000-7fa858000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa858000000-7fa8580a9000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8580a9000-7fa85c000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85d7fb000-7fa85d7fc000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85d7fc000-7fa85dffc000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85dffc000-7fa85dffd000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85dffd000-7fa85e7fd000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85e7fd000-7fa85e7fe000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85e7fe000-7fa85effe000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85effe000-7fa85efff000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85efff000-7fa85f7ff000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85f7ff000-7fa85f800000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa85f800000-7fa860000000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa860000000-7fa860097000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa860097000-7fa864000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa864000000-7fa86409c000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa86409c000-7fa868000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa868000000-7fa868096000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa868096000-7fa86c000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa86c000000-7fa86c094000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa86c094000-7fa870000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa870000000-7fa870023000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa870023000-7fa874000000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8746a5000-7fa8746a6000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8746a6000-7fa874ea6000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa874ea6000-7fa874ea7000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa874ea7000-7fa8756a7000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8756a7000-7fa8756a8000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8756a8000-7fa875ea8000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa875ea8000-7fa875ea9000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa875ea9000-7fa8766a9000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8766a9000-7fa8766aa000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8766aa000-7fa876eaa000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa876eaa000-7fa876eab000 ---p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa876eab000-7fa8776ab000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8776ab000-7fa87786b000 r-xp 00000000 08:03 33951993                   /lib/x86_64-linux-gnu/libc-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa87786b000-7fa877a6b000 ---p 001c0000 08:03 33951993                   /lib/x86_64-linux-gnu/libc-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877a6b000-7fa877a6f000 r--p 001c0000 08:03 33951993                   /lib/x86_64-linux-gnu/libc-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877a6f000-7fa877a71000 rw-p 001c4000 08:03 33951993                   /lib/x86_64-linux-gnu/libc-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877a71000-7fa877a75000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877a75000-7fa877a8b000 r-xp 00000000 08:03 33948183                   /lib/x86_64-linux-gnu/libgcc_s.so.1
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877a8b000-7fa877c8a000 ---p 00016000 08:03 33948183                   /lib/x86_64-linux-gnu/libgcc_s.so.1
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877c8a000-7fa877c8b000 rw-p 00015000 08:03 33948183                   /lib/x86_64-linux-gnu/libgcc_s.so.1
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877c8b000-7fa877d93000 r-xp 00000000 08:03 33951989                   /lib/x86_64-linux-gnu/libm-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877d93000-7fa877f92000 ---p 00108000 08:03 33951989                   /lib/x86_64-linux-gnu/libm-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877f92000-7fa877f93000 r--p 00107000 08:03 33951989                   /lib/x86_64-linux-gnu/libm-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877f93000-7fa877f94000 rw-p 00108000 08:03 33951989                   /lib/x86_64-linux-gnu/libm-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877f94000-7fa877fac000 r-xp 00000000 08:03 33951992                   /lib/x86_64-linux-gnu/libpthread-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa877fac000-7fa8781ab000 ---p 00018000 08:03 33951992                   /lib/x86_64-linux-gnu/libpthread-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8781ab000-7fa8781ac000 r--p 00017000 08:03 33951992                   /lib/x86_64-linux-gnu/libpthread-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8781ac000-7fa8781ad000 rw-p 00018000 08:03 33951992                   /lib/x86_64-linux-gnu/libpthread-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8781ad000-7fa8781b1000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8781b1000-7fa8781d7000 r-xp 00000000 08:03 33951991                   /lib/x86_64-linux-gnu/ld-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8783a3000-7fa8783ca000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8783d5000-7fa8783d6000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8783d6000-7fa8783d7000 r--p 00025000 08:03 33951991                   /lib/x86_64-linux-gnu/ld-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8783d7000-7fa8783d8000 rw-p 00026000 08:03 33951991                   /lib/x86_64-linux-gnu/ld-2.23.so
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7fa8783d8000-7fa8783d9000 rw-p 00000000 00:00 0
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7ffde7f2e000-7ffde7f4f000 rw-p 00000000 00:00 0                          [stack]
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7ffde7fc0000-7ffde7fc3000 r--p 00000000 00:00 0                          [vvar]
May 14 15:05:57 redis04 start-stop-daemon[156313]: 7ffde7fc3000-7ffde7fc5000 r-xp 00000000 00:00 0                          [vdso]
May 14 15:05:57 redis04 start-stop-daemon[156313]: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_Z9traceInfoi+0x12f)[0x4b8a6f]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy[0x4b785d]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7fa8776e04b0]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fa8776e0428]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fa8776e202a]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(+0x777ea)[0x7fa8777227ea]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fa87772b37a]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fa87772f53c]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Request5clearEv+0x9e7)[0x48a007]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN16AcceptConnection5writeEP7HandlerP5IOVeci+0xe99)[0x453fd9]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN16AcceptConnection10writeEventEP7Handler+0x28)[0x4545f8]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Handler25postAcceptConnectionEventEv+0x80)[0x4aa4a0]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(_ZN7Handler3runEv+0x56)[0x4ad956]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /opt/redis/bin/predixy(execute_native_thread_routine+0x20)[0x4e9560]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fa877f9b6ba]
May 14 15:05:57 redis04 start-stop-daemon[156313]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa8777b241d]
May 14 15:05:59 redis04 systemd[1]: predixy_cluster.service: Main process exited, code=killed, status=6/ABRT
May 14 15:05:59 redis04 systemd[1]: predixy_cluster.service: Unit entered failed state.
May 14 15:05:59 redis04 systemd[1]: predixy_cluster.service: Failed with result 'signal'.
May 14 15:05:59 redis04 systemd[1]: predixy_cluster.service: Service hold-off time over, scheduling restart.
May 14 15:05:59 redis04 systemd[1]: Stopped Redis Cluster Proxy Service.

We're using predixy 1.0.5-pre on Ubuntu 16.04, which is built from latest master.

# uname -a
Linux redis04 4.4.0-121-generic #145-Ubuntu SMP Fri Apr 13 13:47:23 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

configuration attached.
predixy.conf.txt

I think we're experiencing basically same issue as #29.

We're performing load testing on redis cluster, A predixy instance is processing 10K requests per sec. predixy doesn't crash so often, but when it crashes, it is critical.

127.0.0.1:7000> info
# Proxy
Version:1.0.5-pre
Name:
Bind:0.0.0.0:7000
RedisMode:proxy
SingleThread:false
WorkerThreads:10
Uptime:1526354727
UptimeSince:2018-05-15 10:25:27

# SystemResource
UsedMemory:5256256
MaxMemory:0
MaxRSS:8794112
UsedCpuSys:1625.952
UsedCpuUser:682.592

# Stats
Accept:3987
ClientConnections:600
TotalRequests:85803809
TotalResponses:85803814
TotalRecvClientBytes:6244194469
TotalSendServerBytes:6244644215
TotalRecvServerBytes:6760996314
TotalSendClientBytes:6724113683

May require additional node.js client query restructuring for queries against predixy

I'll add more info & perhaps reproducible test scenario later when I have some time, as we're using a customized fork of predixy for our specific deployment. But just wanted to note this for potential node.js users of redis who may use predixy, to be aware of a possible issue.

I noticed that when switching from regular redis (non-cluster, standalone server) to predixy, using https://github.com/NodeRedis/node_redis, I came across issue when you nest queries that also involve iteration. Like a dependent hgetall that follows an smembers query, where you iterate over the smembers returned for hgetall. However, if we follow smembers with a single hgetall only, then things are ok, it's only an issue if we iterate over a set of smembers for hgetall, whatever N might be (I haven't debugged to find the breaking N).

The particular issue observed for the problem scenario is that smembers may return null (i.e. empty) when we don't expect it to (say when running smember query by itself), or the whole query chain gets stuck and times out eventually. The same query pattern has no issue under regular standalone redis.

An example of the type of query causing the problem can be found in the original sample code from: https://chrysohous.wordpress.com/2012/09/06/redis-and-nodejs-problem-with-asynchronism, including the initial solution. I haven't tried but the updated solution in that post (resulting from a comment from me) might solve this problem. I worked around the issue with a different solution for my use case, avoiding the nested/chained query.

On a related note, had no problems with python redis client migrating from standalone redis to predixy. It seems to be something with the node.js client driver, or the asynchronous nature of node.js in querying redis/predixy, since python client is synchronous I believe.

build failed on g++ 4.8.4

I tried to build predixy on Ubuntu 14.04, g++ 4.8.4 and it failed.

$ make -j 8
make[1]: Entering directory `/home/user/predixy/src'
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Crc16.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c HashFunc.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Timer.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Logger.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c LogFileSink.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Alloc.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Socket.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c ListenSocket.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c AcceptSocket.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c ConnectSocket.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c EpollMultiplexor.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Subscribe.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Connection.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c AcceptConnection.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c ConnectConnection.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Buffer.cpp
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Command.cpp

In file included from Command.h:11:0,
                 from Command.cpp:11:
Command.h: In instantiation of ‘Command::InitFail::InitFail(A&& ...) [with A = {const char (&)[12], int, const char (&)[32], const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&}]’:
Command.cpp:199:9:   required from here
Exception.h:58:41: error: cannot pass objects of non-trivially-copyable type ‘const class std::basic_string<char>’ through ‘...’
     T(A&&... args):ExceptionBase(args...) {}            \
                                         ^
Command.h:19:5: note: in expansion of macro ‘DefException’
     DefException(InitFail);
     ^
make[1]: *** [Command.o] Error 1
make[1]: Leaving directory `/home/fo3-mgr/predixy/src'
make: *** [default] Error 2

$ g++ --version
g++ (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

FYI, build on g++ 5.4.0 (Ubuntu 16.04) was successful.

make failed 失败

# make
make[1]: Entering directory `/usr/local/soft/predixy/src'
g++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -o predixy Crc16.o HashFunc.o Timer.o Logger.o LogFileSink.o Alloc.o Socket.o ListenSocket.o AcceptSocket.o ConnectSocket.o EpollMultiplexor.o Subscribe.o Connection.o AcceptConnection.o ConnectConnection.o Buffer.o Command.o Distribution.o Reply.o ConfParser.o Conf.o Auth.o DC.o LatencyMonitor.o RequestParser.o Request.o ResponseParser.o Response.o Server.o ServerGroup.o ServerPool.o ClusterNodesParser.o ClusterServerPool.o SentinelServerPool.o ConnectConnectionPool.o Handler.o Proxy.o main.o -static-libstdc++ -rdynamic -lpthread
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make[1]: *** [predixy] Error 1
make[1]: Leaving directory `/usr/local/soft/predixy/src'
make: *** [default] Error 2

gcc环境

# gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/usr/bin/ld: cannot find -lstdc++是我的gcc版本不对吗?

进程异常退出

一共遇到2次,配置和 #32 里的一样
错误日志:

2018-05-01 03:38:35.091997 N Handler.cpp:212 h 0 remove c 127.0.0.1:59610 230 with status 2 End
2018-05-01 03:38:35.190520 N Handler.cpp:318 h 2 accept c 10.81.3.16:58238 230
2018-05-01 03:38:35.190725 N Handler.cpp:212 h 0 remove c 10.81.3.16:58238 230 with status 1 None
2018-05-01 03:38:40.125014 N Handler.cpp:318 h 7 accept c 127.0.0.1:60172 230
2018-05-01 03:38:40.125056 N Handler.cpp:212 h 0 remove c 127.0.0.1:60172 230 with status 2 End
2018-05-01 03:38:40.126854 E Backtrace.h:18 predixy backtrace(6)
/usr/local/predixy/bin/predixy(_Z9traceInfoi+0x12f)[0x4b609f]
/usr/local/predixy/bin/predixy[0x4b4f7d]
/lib64/libc.so.6[0x3a5fe32660]
/lib64/libc.so.6(gsignal+0x35)[0x3a5fe325e5]
/lib64/libc.so.6(abort+0x175)[0x3a5fe33dc5]
/lib64/libc.so.6[0x3a5fe704f7]
/lib64/libc.so.6[0x3a5fe75f3e]
/lib64/libc.so.6[0x3a5fe78dd0]
/usr/local/predixy/bin/predixy(_ZN9RefCntObjI16AcceptConnectionSt6atomicIiEE5unrefEv+0x68)[0x45ace8]
/usr/local/predixy/bin/predixy(_ZN7Handler25postAcceptConnectionEventEv+0x26c)[0x4a7a2c]
/usr/local/predixy/bin/predixy(_ZN7Handler3runEv+0x56)[0x4aacf6]
/usr/local/predixy/bin/predixy[0x4ed250]
/lib64/libpthread.so.0[0x3a60207aa1]
/lib64/libc.so.6(clone+0x6d)[0x3a5fee8aad]

数据获取异常 (error) NOAUTH Authentication required.

------- auth 配置----
Authority {
Auth {
Mode write
}
Auth 12312300 {
Mode admin
}
}

------ 错误提示-------
127.0.0.1:7617> set name daven
OK
127.0.0.1:7617> get name
(error) NOAUTH Authentication required.
127.0.0.1:7617> info

Proxy

Version:1.0.5-pre
Name:Session
Bind:0.0.0.0:7617
RedisMode:proxy
SingleThread:false
WorkerThreads:4
Uptime:1535441754
UptimeSince:2018-08-28 15:35:54

SystemResource

UsedMemory:139744
MaxMemory:0
MaxRSS:1933312
UsedCpuSys:0.029
UsedCpuUser:0.021

Stats

Accept:2
ClientConnections:1
TotalRequests:116
TotalResponses:116
TotalRecvClientBytes:136
TotalSendServerBytes:2560
TotalRecvServerBytes:41024
TotalSendClientBytes:1849

Servers

Server:127.0.0.1:6001
Role:master
Group:176dfd3a99641812449251898536fa2962e11eac
DC:
CurrentIsFail:0
Connections:3
Connect:3
Requests:16
Responses:16
SendBytes:376
RecvBytes:7566

Server:127.0.0.1:6002
Role:master
Group:77c97fc32c865d937b34ba7fab391779f9fbac59
DC:
CurrentIsFail:0
Connections:4
Connect:4
Requests:20
Responses:20
SendBytes:470
RecvBytes:8336

Server:127.0.0.1:6003
Role:master
Group:afb732990985b792baa754ec89da6d6fd1c2524b
DC:
CurrentIsFail:0
Connections:2
Connect:2
Requests:9
Responses:9
SendBytes:204
RecvBytes:3789

Server:127.0.0.1:6004
Role:slave
Group:176dfd3a99641812449251898536fa2962e11eac
DC:
CurrentIsFail:0
Connections:4
Connect:4
Requests:27
Responses:27
SendBytes:660
RecvBytes:14355

Server:127.0.0.1:6005
Role:slave
Group:afb732990985b792baa754ec89da6d6fd1c2524b
DC:
CurrentIsFail:0
Connections:3
Connect:3
Requests:14
Responses:14
SendBytes:320
RecvBytes:6060

Server:127.0.0.1:6006
Role:slave
Group:77c97fc32c865d937b34ba7fab391779f9fbac59
DC:
CurrentIsFail:0
Connections:3
Connect:9
Requests:27
Responses:27
SendBytes:530
RecvBytes:918

LatencyMonitor

LatencyMonitorName:all
<= 100 18 2 33.33%
<= 200 118 1 50.00%
<= 300 452 2 83.33%
<= 400 303 1 100.00%
T 148 891 6

LatencyMonitorName:get
<= 300 452 2 100.00%
T 226 452 2

LatencyMonitorName:set
<= 400 303 1 100.00%
T 303 303 1

LatencyMonitorName:blist

关于配置多机房Redis-cluster

你好,配置中好像没有提到多机房的节点配置,dc.conf只是用于标识IP所在机房;
1,在cluster.conf中自行配置多个节点后,运行predixy只能识别最后一组节点,当一组Redis-cluster挂掉之后并不会自动切换到另一组Redis-cluster上。
2,Redis-cluster中某个节点挂掉之后就无法写入和读取了,服务不可用了。但是是单独登陆Redis-cluster的宿主机,用redis-trib.rb检查是正常的,节点也是正常使用的。
期待您的回复!

日志中报错, 报错信息如下

2018-10-17 11:23:32.301017 N Handler.cpp:371 h 2 accept c 10.162.204.28:5065 91 assign to h 3
2018-10-17 11:23:32.301080 N Handler.cpp:212 h 3 remove c 10.170.178.216:46445 95 with status 2 End
2018-10-17 11:23:32.301623 N Handler.cpp:212 h 2 remove c 10.172.166.210:26558 151 with status 2 End
2018-10-17 11:23:32.302346 N Handler.cpp:371 h 2 accept c 172.31.160.139:40152 95 assign to h 3
2018-10-17 11:23:32.302497 N Handler.cpp:371 h 2 accept c 10.44.39.150:47320 147 assign to h 0
2018-10-17 11:23:32.302890 N Handler.cpp:212 h 1 remove c 10.170.185.19:44515 96 with status 2 End
2018-10-17 11:23:32.304583 N Handler.cpp:212 h 0 remove c 10.44.27.215:7532 59 with status 2 End
2018-10-17 11:23:32.304891 N Handler.cpp:371 h 2 accept c 10.170.183.179:21861 59 assign to h 1
2018-10-17 11:23:32.304976 N Handler.cpp:212 h 1 remove c 172.31.160.139:39552 28 with status 2 End
2018-10-17 11:23:32.305018 N Handler.cpp:371 h 1 accept c 10.44.27.215:7573 28 assign to h 1
2018-10-17 11:23:32.305380 N Handler.cpp:212 h 3 remove c 10.162.204.28:5065 91 with status 2 End
2018-10-17 11:23:32.306373 N Handler.cpp:212 h 3 remove c 10.171.34.148:64208 62 with status 2 End
2018-10-17 11:23:32.306895 N Handler.cpp:371 h 2 accept c 10.171.34.148:64212 62 assign to h 3
2018-10-17 11:23:32.307157 N Handler.cpp:371 h 2 accept c 10.170.185.19:44519 91 assign to h 3
2018-10-17 11:23:32.307433 N Handler.cpp:212 h 0 remove c 10.44.39.150:47320 147 with status 2 End
2018-10-17 11:23:32.307574 N Handler.cpp:371 h 2 accept c 10.170.180.167:54323 96 assign to h 0
2018-10-17 11:23:32.308235 N Handler.cpp:371 h 2 accept c 172.31.160.151:53498 147 assign to h 0
2018-10-17 11:23:32.308796 N Handler.cpp:212 h 1 remove c 172.31.160.139:39658 78 with status 2 End
2018-10-17 11:23:32.309289 N Handler.cpp:212 h 0 remove c 10.162.204.28:5062 19 with status 2 End
2018-10-17 11:23:32.309323 N Handler.cpp:212 h 1 remove c 10.170.183.179:21861 59 with status 2 End
2018-10-17 11:23:32.309893 N Handler.cpp:371 h 2 accept c 172.31.160.145:51116 19 assign to h 1
2018-10-17 11:23:32.310246 N Handler.cpp:371 h 2 accept c 10.171.34.148:64214 59 assign to h 1
2018-10-17 11:23:32.310597 N Handler.cpp:371 h 2 accept c 172.31.160.139:40182 78 assign to h 0
2018-10-17 11:23:32.310772 N Handler.cpp:212 h 1 remove c 10.44.27.215:7573 28 with status 2 End
2018-10-17 11:23:41.988641 N Handler.cpp:212 h 3 remove c 10.163.3.95:16233 134 with status 4 EventError
2018-10-17 11:23:42.081579 N Handler.cpp:212 h 0 remove c 10.44.39.150:47150 97 with status 4 EventError
2018-10-17 11:23:42.128975 N Handler.cpp:212 h 3 remove c 10.165.118.52:59121 11 with status 4 EventError
2018-10-17 11:23:42.155938 N Handler.cpp:212 h 0 remove c 10.44.27.215:7525 93 with status 4 EventError
2018-10-17 11:23:42.160593 N Handler.cpp:212 h 1 remove c 10.171.34.148:64136 34 with status 4 EventError
2018-10-17 11:23:42.186558 N Handler.cpp:212 h 2 remove c 172.31.160.145:50594 35 with status 4 EventError
2018-10-17 11:23:42.191317 N Handler.cpp:212 h 0 remove c 10.171.34.148:64165 84 with status 4 EventError
2018-10-17 11:23:42.196557 N Handler.cpp:212 h 2 remove c 10.171.11.134:26778 15 with status 4 EventError
2018-10-17 11:23:42.216597 N Handler.cpp:212 h 1 remove c 10.171.34.148:64131 113 with status 4 EventError
2018-10-17 11:23:42.243555 N Handler.cpp:212 h 3 remove c 172.31.160.139:39788 108 with status 4 EventError
2018-10-17 11:23:42.253320 N Handler.cpp:212 h 2 remove c 172.31.160.139:39814 104 with status 4 EventError
2018-10-17 11:23:42.254884 N Handler.cpp:212 h 3 remove c 10.163.3.95:16249 70 with status 4 EventError
2018-10-17 11:23:42.262113 N Handler.cpp:212 h 1 remove c 172.31.160.145:50918 98 with status 4 EventError

Handler.cpp:228 server :0 mark failure

hi all. i have a problem with predixy
logfile

2017-10-30 13:27:05.836496 N Handler.cpp:239 h 0 close s 10.33.51.206:0 23 and c None -1 with status 4 EventError
2017-10-30 13:27:15.836580 N ConnectConnectionPool.cpp:182 h 0 check server reopen connection 10.33.51.206:0 23
2017-10-30 13:27:15.836766 E Handler.cpp:395 h 0 s 10.33.51.206:0 23 will be close with status 4 EventError

config predixy.conf

Name loan
Bind 0.0.0.0:7617
WorkerThreads 2
MaxMemory 512M
ClientTimeout 300
BufSize 8192
Log /var/log/predixy/predixy.log
LogRotate 1d
LogVerbSample 0
LogDebugSample 0
LogInfoSample 0
LogNoticeSample 1
LogWarnSample 1
LogErrorSample 1
Include sentinel.conf
Include latency.conf

config sentinel.conf

SentinelServerPool {
Databases 16
Hash crc16
HashTag "{}"
Distribution modula
MasterReadPriority 100
StaticSlaveReadPriority 0
DynamicSlaveReadPriority 0
RefreshInterval 30
ServerFailureLimit 60
ServerRetryTimeout 10
Sentinels {
+ 10.33.51.205:26379
+ 10.33.51.206:26379
}
Group ppa-s1 {
+ 10.33.51.205:6379
+ 10.33.51.206:6379
}
}

不支持cluster nodes命令

在使用过程中,发现predixy不支持cluster nodes命令,希望能够增加该支持,可以返回代理自身的一个模拟cluster nodes结果,提高兼容性

predix 连接集群 redis cluster 4错误

日志:
2018-05-30 18:02:02.271284 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.223:6380 6
2018-05-30 18:02:02.273297 W Handler.cpp:1387 h 0 s 200.200.200.223:6380 6 readonly fail
2018-05-30 18:02:02.273347 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:02.273354 E Handler.cpp:434 h 0 s 200.200.200.223:6380 6 will be close with status 101 Custom
2018-05-30 18:02:02.273361 N Handler.cpp:276 h 0 close s 200.200.200.223:6380 6 and c None -1 with status 101 Custom
2018-05-30 18:02:03.292893 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.222:6379 6
2018-05-30 18:02:03.293781 W Handler.cpp:1387 h 0 s 200.200.200.222:6379 6 readonly fail
2018-05-30 18:02:03.293811 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:03.293815 E Handler.cpp:434 h 0 s 200.200.200.222:6379 6 will be close with status 101 Custom
2018-05-30 18:02:03.293818 N Handler.cpp:276 h 0 close s 200.200.200.222:6379 6 and c None -1 with status 101 Custom
2018-05-30 18:02:04.312134 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.223:6379 6
2018-05-30 18:02:04.312979 W Handler.cpp:1387 h 0 s 200.200.200.223:6379 6 readonly fail
2018-05-30 18:02:04.313005 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:04.313010 E Handler.cpp:434 h 0 s 200.200.200.223:6379 6 will be close with status 101 Custom
2018-05-30 18:02:04.313013 N Handler.cpp:276 h 0 close s 200.200.200.223:6379 6 and c None -1 with status 101 Custom
2018-05-30 18:02:05.331245 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.222:6379 6
2018-05-30 18:02:14.443381 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.221:6379 7
2018-05-30 18:02:14.443808 N Handler.cpp:276 h 0 close s 200.200.200.222:6379 6 and c None -1 with status 102 Custom
2018-05-30 18:02:14.444811 W Handler.cpp:1387 h 0 s 200.200.200.221:6379 7 readonly fail
2018-05-30 18:02:14.444885 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:14.444889 E Handler.cpp:434 h 0 s 200.200.200.221:6379 7 will be close with status 101 Custom
2018-05-30 18:02:14.444892 N Handler.cpp:276 h 0 close s 200.200.200.221:6379 7 and c None -1 with status 101 Custom
2018-05-30 18:02:15.424937 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.223:6379 6
2018-05-30 18:02:15.428349 W Handler.cpp:1387 h 0 s 200.200.200.223:6379 6 readonly fail
2018-05-30 18:02:15.428389 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:15.428393 E Handler.cpp:434 h 0 s 200.200.200.223:6379 6 will be close with status 101 Custom
2018-05-30 18:02:15.428396 N Handler.cpp:276 h 0 close s 200.200.200.223:6379 6 and c None -1 with status 101 Custom
2018-05-30 18:02:16.444573 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.221:6380 6
2018-05-30 18:02:16.445017 W Handler.cpp:1387 h 0 s 200.200.200.221:6380 6 readonly fail
2018-05-30 18:02:16.445035 E ClusterServerPool.cpp:188 redis cluster nodes parse error
2018-05-30 18:02:16.445039 E Handler.cpp:434 h 0 s 200.200.200.221:6380 6 will be close with status 101 Custom
2018-05-30 18:02:16.445042 N Handler.cpp:276 h 0 close s 200.200.200.221:6380 6 and c None -1 with status 101 Custom
2018-05-30 18:02:17.469791 N ConnectConnectionPool.cpp:50 h 0 reopen server connection 200.200.200.221:6379 6
2018-05-30 18:02:17.469904 N Handler.cpp:76 handler 0 stopped

配置:

grep -v -E "^#|^$" predixy.conf

Name PredixyExample
Bind 200.200.200.221:7617
WorkerThreads 1
MaxMemory 128M
ClientTimeout 300
BufSize 4096
Log ./predixy.log
LogVerbSample 0
LogDebugSample 0
LogInfoSample 10000
LogNoticeSample 1
LogWarnSample 1
LogErrorSample 1
Include auth.conf
Include cluster.conf

grep -v -E "^#|^$" cluster.conf

ClusterServerPool {
MasterReadPriority 60
StaticSlaveReadPriority 50
DynamicSlaveReadPriority 50
RefreshInterval 1
ServerTimeout 1
ServerFailureLimit 10
ServerRetryTimeout 1
KeepAlive 120
Servers {
+ 200.200.200.221:6379 #master
+ 200.200.200.222:6379
+ 200.200.200.223:6379
+ 200.200.200.221:6380 #slave
+ 200.200.200.222:6380
+ 200.200.200.223:6380
}
}

centos6.5下编译各种问题

你好,由于centos6的GCC版本问题,升级成4.8+编译也不成功,但是同样的包拿到centos7上编译没有问题。
是否可以提供二进制的包,或者centos6.5上的解决方法。

[root@localhost ~]# gcc --version
gcc (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ls
anaconda-ks.cfg devtools-2.repo install.log install.log.syslog predixy-master
[root@localhost ~]# cd predixy-master/
[root@localhost predixy-master]# ls
conf _config.yml doc LICENSE Makefile README_CN.md README.md src test
[root@localhost predixy-master]# make
make[1]: Entering directory /root/predixy-master/src' /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Crc16.cpp /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c HashFunc.cpp /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Timer.cpp /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Logger.cpp /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c LogFileSink.cpp /opt/rh/devtoolset-2/root/usr/bin/c++ -std=c++11 -Wall -w -g -O3 -D_PREDIXY_BACKTRACE_ -D_EPOLL_ -c Alloc.cpp Alloc.cpp: In function ‘int& _ZTWN5AllocIT_XT0_EE4SizeE()’: Alloc.cpp:10:36: internal compiler error: in var_defined_without_dynamic_init, at cp/decl2.c:2811 AtomicLong AllocBase::UsedMemory(0); ^ Please submit a full bug report, with preprocessed source if appropriate. See <http://bugzilla.redhat.com/bugzilla> for instructions. Preprocessed source stored into /tmp/ccnvCPIf.out file, please attach this to your bugreport. make[1]: *** [Alloc.o] 错误 1 make[1]: Leaving directory /root/predixy-master/src'
make: *** [default] 错误 2

Question about latency monitor

What exact latency shows the Predixy latency?
What measure unit it shows: microseconds or milliseconds?
I mean a TimeSpan parameters from the latency.conf.

LatencyMonitor

LatencyMonitorName:all
<= 100 270607384 5489974 93.80%
<= 200 36624590 283961 98.65%
<= 300 8696824 37530 99.29%
<= 400 2514521 7476 99.42%

无法用del命令删除key

删除key ,这个要指定key的值,操作起来不方便
172.16.11.43:7617> set test 123
OK
172.16.11.43:7617> get test
"123"
172.16.11.43:7617> del test
(error) ERR unknown command '$4'

编译没问题,启动报错

2017-09-02 09:56:24.964168 N Handler.cpp:420 h 0 create connection pool for server 10.249.200.12:6379@16379
2017-09-02 09:56:24.966543 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6379@16379:Servname not supported for ai_socktype
2017-09-02 09:56:25.975254 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6383@16383:Servname not supported for ai_socktype
2017-09-02 09:56:26.983231 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6383@16383:Servname not supported for ai_socktype
2017-09-02 09:56:27.991183 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6381@16381:Servname not supported for ai_socktype
2017-09-02 09:56:29.001205 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6382@16382:Servname not supported for ai_socktype
2017-09-02 09:56:30.010620 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6379@16379:Servname not supported for ai_socktype
2017-09-02 09:56:31.020511 E Handler.cpp:88 h 0 refresh server pool exception:Socket.cpp:120 invalid addr 10.249.200.12:6381@16381:Servname not supported for ai_socktype

启动后就报错,系统是centos7

predixy-1.0.4 crash

2018-04-18 20:31:59.024826 N Handler.cpp:212 h 3 remove c 192.168.1.3:35988 1415 with status 2 End
*** glibc detected *** ./predixy: free(): corrupted unsorted chunks: 0x00007f94207c1d10 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75e66)[0x7f943866fe66]
/lib64/libc.so.6(+0x789b3)[0x7f94386729b3]
./predixy(_ZN7Request5clearEv+0x748)[0x4b2a28]
./predixy(_ZN16AcceptConnection5writeEP7HandlerP5IOVeci+0x156e)[0x479efe]
./predixy(_ZN16AcceptConnection10writeEventEP7Handler+0x28)[0x47a708]
./predixy(_ZN7Handler25postAcceptConnectionEventEv+0x7b)[0x4cf68b]
./predixy(_ZN7Handler3runEv+0x1d0)[0x4d1c90]
./predixy[0x530db0]
/lib64/libpthread.so.0(+0x79d1)[0x7f9438e2f9d1]
/lib64/libc.so.6(clone+0x6d)[0x7f94386e28fd]
======= Memory map: ========
00400000-005be000 r-xp 00000000 08:04 2152684892 /home/cycker/predixy-1.0.2/predixy
007bd000-007c6000 rw-p 001bd000 08:04 2152684892 /home/cycker/predixy-1.0.2/predixy
007c6000-007c9000 rw-p 00000000 00:00 0
0196e000-019c1000 rw-p 00000000 00:00 0 [heap]
7f9418000000-7f941b5f0000 rw-p 00000000 00:00 0
7f941b5f0000-7f941c000000 ---p 00000000 00:00 0
7f9420000000-7f9423a40000 rw-p 00000000 00:00 0
7f9423a40000-7f9424000000 ---p 00000000 00:00 0
7f9428000000-7f942b437000 rw-p 00000000 00:00 0
7f942b437000-7f942c000000 ---p 00000000 00:00 0
7f942c000000-7f942e9b0000 rw-p 00000000 00:00 0
7f942e9b0000-7f9430000000 ---p 00000000 00:00 0
7f9430000000-7f9430023000 rw-p 00000000 00:00 0
7f9430023000-7f9434000000 ---p 00000000 00:00 0
7f9435df5000-7f9435df6000 ---p 00000000 00:00 0
7f9435df6000-7f94365f6000 rw-p 00000000 00:00 0 [stack:32928]
7f94365f6000-7f94365f7000 ---p 00000000 00:00 0
7f94365f7000-7f9436df7000 rw-p 00000000 00:00 0 [stack:32927]
7f9436df7000-7f9436df8000 ---p 00000000 00:00 0
7f9436df8000-7f94375f8000 rw-p 00000000 00:00 0 [stack:32926]
7f94375f8000-7f94375f9000 ---p 00000000 00:00 0
7f94375f9000-7f9437df9000 rw-p 00000000 00:00 0 [stack:32925]
7f9437df9000-7f9437dfa000 ---p 00000000 00:00 0
7f9437dfa000-7f94385fa000 rw-p 00000000 00:00 0 [stack:32924]
7f94385fa000-7f9438784000 r-xp 00000000 08:04 2152333693 /lib64/libc-2.12.so
7f9438784000-7f9438984000 ---p 0018a000 08:04 2152333693 /lib64/libc-2.12.so
7f9438984000-7f9438988000 r--p 0018a000 08:04 2152333693 /lib64/libc-2.12.so
7f9438988000-7f9438989000 rw-p 0018e000 08:04 2152333693 /lib64/libc-2.12.so
7f9438989000-7f943898e000 rw-p 00000000 00:00 0
7f943898e000-7f94389a4000 r-xp 00000000 08:04 4302589354 /lib64/libgcc_s-4.4.6-20110824.so.1
7f94389a4000-7f9438ba3000 ---p 00016000 08:04 4302589354 /lib64/libgcc_s-4.4.6-20110824.so.1
7f9438ba3000-7f9438ba4000 rw-p 00015000 08:04 4302589354 /lib64/libgcc_s-4.4.6-20110824.so.1
7f9438ba4000-7f9438c27000 r-xp 00000000 08:04 2152333694 /lib64/libm-2.12.so
7f9438c27000-7f9438e26000 ---p 00083000 08:04 2152333694 /lib64/libm-2.12.so
7f9438e26000-7f9438e27000 r--p 00082000 08:04 2152333694 /lib64/libm-2.12.so
7f9438e27000-7f9438e28000 rw-p 00083000 08:04 2152333694 /lib64/libm-2.12.so
7f9438e28000-7f9438e3f000 r-xp 00000000 08:04 2152333646 /lib64/libpthread-2.12.so
7f9438e3f000-7f943903f000 ---p 00017000 08:04 2152333646 /lib64/libpthread-2.12.so
7f943903f000-7f9439040000 r--p 00017000 08:04 2152333646 /lib64/libpthread-2.12.so
7f9439040000-7f9439041000 rw-p 00018000 08:04 2152333646 /lib64/libpthread-2.12.so
7f9439041000-7f9439045000 rw-p 00000000 00:00 0
7f9439045000-7f9439065000 r-xp 00000000 08:04 2152333638 /lib64/ld-2.12.so
7f943922d000-7f9439254000 rw-p 00000000 00:00 0
7f9439261000-7f9439264000 rw-p 00000000 00:00 0
7f9439264000-7f9439265000 r--p 0001f000 08:04 2152333638 /lib64/ld-2.12.so
7f9439265000-7f9439266000 rw-p 00020000 08:04 2152333638 /lib64/ld-2.12.so
7f9439266000-7f9439267000 rw-p 00000000 00:00 0
7ffc4ccc7000-7ffc4cce8000 rw-p 00000000 00:00 0 [stack]
7ffc4cd95000-7ffc4cd96000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
2018-04-18 20:31:59.025535 E Backtrace.h:18 predixy backtrace(6)
./predixy(_Z9traceInfoi+0x114)[0x4dbe84]
./predixy[0x4dad6d]
/lib64/libc.so.6(+0x326a0)[0x7f943862c6a0]
/lib64/libc.so.6(gsignal+0x35)[0x7f943862c625]
/lib64/libc.so.6(abort+0x175)[0x7f943862de05]
/lib64/libc.so.6(+0x70537)[0x7f943866a537]
/lib64/libc.so.6(+0x75e66)[0x7f943866fe66]
/lib64/libc.so.6(+0x789b3)[0x7f94386729b3]
./predixy(_ZN7Request5clearEv+0x748)[0x4b2a28]
./predixy(_ZN16AcceptConnection5writeEP7HandlerP5IOVeci+0x156e)[0x479efe]
./predixy(_ZN16AcceptConnection10writeEventEP7Handler+0x28)[0x47a708]
./predixy(_ZN7Handler25postAcceptConnectionEventEv+0x7b)[0x4cf68b]
./predixy(_ZN7Handler3runEv+0x1d0)[0x4d1c90]
./predixy[0x530db0]
/lib64/libpthread.so.0(+0x79d1)[0x7f9438e2f9d1]
/lib64/libc.so.6(clone+0x6d)[0x7f94386e28fd]

`EXEC without MULTI`

When supportTransaction() is false, WATCH and MULTI commands are forbidden.

EXEC should also be forbidden. Otherwise, MULTI command gets dropped (by predixy) but EXEC gets passed along, causing redis to return EXEC without MULTI error.

关于每个线程一个epoll去监听listenfd

您好,我发现代码里面是每个线程一个epoll去监听客户端的连接,这样当新的链接到来时,每个线程都会得到可读事件然后调用accept,在我理解 这里应该只有第一个线程会接收到连接,其他线程都会accept失败(监听socket设置了非阻塞),这样会不会有性能损失?

不能使用 redis sentinel

环境: CentOS7
Redis: 3.0.7
predixy: github master分支
Redis为1主3从

SentinelServerPool {
    Hash crc16
    Distribution modula
    #MasterReadPriority 30
    #StaticSlaveReadPriority 50
    #DynamicSlaveReadPriority 50
    RefreshInterval 5
    ServerFailureLimit 10
    ServerRetryTimeout 5
    Sentinels {
        + 10.90.19.206:26379
        + 10.90.19.28:26379
        + 10.90.19.6:26379
        + 10.90.19.158:26379
    }
    Group db01 {
        + 10.90.19.206:6382
        + 10.90.19.28:6382
        + 10.90.19.6:6382
        + 10.90.19.158:6382
    }
}

进入predixy后get操作报:(error) ERR no server avaliable

日志为:

2017-11-06 20:27:41.217238 N Proxy.cpp:112 predixy listen in 0.0.0.0:6378
2017-11-06 20:27:41.217377 N Proxy.cpp:143 predixy running with Name:phototask-redis-db Workers:4
2017-11-06 20:27:41.217745 N Handler.cpp:414 h 1 create connection pool for server 10.90.19.206:26379
2017-11-06 20:27:41.217771 N ConnectConnectionPool.cpp:40 h 1 create server connection 10.90.19.206:26379 8
2017-11-06 20:27:46.227055 N Handler.cpp:414 h 2 create connection pool for server 10.90.19.6:26379
2017-11-06 20:27:46.227108 N ConnectConnectionPool.cpp:40 h 2 create server connection 10.90.19.6:26379 9
2017-11-06 20:27:46.227191 N Handler.cpp:414 h 2 create connection pool for server 10.90.19.158:26379
2017-11-06 20:27:46.227216 N ConnectConnectionPool.cpp:40 h 2 create server connection 10.90.19.158:26379 10
2017-11-06 20:27:46.227252 N Handler.cpp:414 h 2 create connection pool for server 10.90.19.28:26379
2017-11-06 20:27:46.227277 N ConnectConnectionPool.cpp:40 h 2 create server connection 10.90.19.28:26379 11
2017-11-06 20:27:51.236447 N Handler.cpp:414 h 0 create connection pool for server 10.90.19.6:26379
2017-11-06 20:27:51.236492 N ConnectConnectionPool.cpp:40 h 0 create server connection 10.90.19.6:26379 12
2017-11-06 20:27:51.236557 N Handler.cpp:414 h 0 create connection pool for server 10.90.19.28:26379
2017-11-06 20:27:51.236579 N ConnectConnectionPool.cpp:40 h 0 create server connection 10.90.19.28:26379 13
2017-11-06 20:27:51.236618 N Handler.cpp:414 h 0 create connection pool for server 10.90.19.158:26379
2017-11-06 20:27:51.236636 N ConnectConnectionPool.cpp:40 h 0 create server connection 10.90.19.158:26379 14
2017-11-06 20:27:56.245632 N Handler.cpp:414 h 3 create connection pool for server 10.90.19.206:26379
2017-11-06 20:27:56.245682 N ConnectConnectionPool.cpp:40 h 3 create server connection 10.90.19.206:26379 15
2017-11-06 20:27:56.245765 N Handler.cpp:414 h 3 create connection pool for server 10.90.19.28:26379
2017-11-06 20:27:56.245785 N ConnectConnectionPool.cpp:40 h 3 create server connection 10.90.19.28:26379 16
2017-11-06 20:27:56.245813 N Handler.cpp:414 h 3 create connection pool for server 10.90.19.6:26379
2017-11-06 20:27:56.245834 N ConnectConnectionPool.cpp:40 h 3 create server connection 10.90.19.6:26379 17
2017-11-06 20:28:09.296668 N Handler.cpp:281 h 3 accept c 127.0.0.1:41780 19
2017-11-06 20:28:11.273221 N Handler.cpp:414 h 1 create connection pool for server 10.90.19.6:26379
2017-11-06 20:28:11.273274 N ConnectConnectionPool.cpp:40 h 1 create server connection 10.90.19.6:26379 18
2017-11-06 20:28:11.273335 N Handler.cpp:414 h 1 create connection pool for server 10.90.19.158:26379
2017-11-06 20:28:11.273354 N ConnectConnectionPool.cpp:40 h 1 create server connection 10.90.19.158:26379 20
2017-11-06 20:28:11.273392 N Handler.cpp:414 h 1 create connection pool for server 10.90.19.28:26379
2017-11-06 20:28:11.273411 N ConnectConnectionPool.cpp:40 h 1 create server connection 10.90.19.28:26379 21
2017-11-06 20:28:15.251126 N Handler.cpp:179 h 0 remove c 127.0.0.1:41780 19 with status 2 End
2017-11-06 20:28:15.964430 N Handler.cpp:281 h 2 accept c 127.0.0.1:41788 19

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.