Coder Social home page Coder Social logo

redis-cluster-docker-swarm's Introduction

Redis Cluster Cache for Docker Swarm

CircleCI

Quick and dirty Redis cluster taking advantage of Redis Sentinel for automatic failover. Persistence is turned off by default.

Usage

  1. Setup docker swarm
  2. Create a overlay network:
docker network create --attachable --driver overlay redis
  1. Modify scripts/docker-compose.yml to how you want to deploy the stack.
  2. Run scripts/bootstrap.sh.
bash scripts/bootstrap.sh latest
  1. Connect to with redis-cli
docker run --rm --network redis -ti redis:4.0.11-alpine redis-cli -h redis

To access the redis cluster outside of docker, port 6379 needs to be expose. This can be done by adding ports to the docker-compose file:

...
  redis:
    image: thomasjpfan/redis-look
    ports:
      - "6379:6379"
...

Details

A docker service called redis-zero is created to serve as the initial master for the redis sentinels to setup. The redis-look instances watches the redis sentinels for a master, and connects to redis-zero once a master has been decided. Once the dust has settled, remove the redis-zero instance and wait for failover to take over so a new redis-master will take over. Use redis-utils to reset sentinels so that its metadata is accurate with the correct state.

The use of redis-zero as a bootstrapping step allows for the docker-compose.yml to provide only the long running services:

version: '3.1'

services:

  redis-sentinel:
    image: thomasjpfan/redis-sentinel
    environment:
      - REDIS_IP=redis-zero
      - REDIS_MASTER_NAME=redismaster
    deploy:
      replicas: 3
    networks:
      - redis

  redis:
    image: thomasjpfan/redis-look
    environment:
      - REDIS_SENTINEL_IP=redis-sentinel
      - REDIS_MASTER_NAME=redismaster
      - REDIS_SENTINEL_PORT=26379
    deploy:
      replicas: 3
    networks:
      - redis

networks:
  redis:
    external: true

Scaling

From now on just scale redis to expand the number of slaves or scale redis-sentinel to increase the number of sentinels.

redis-cluster-docker-swarm's People

Contributors

thomasjpfan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redis-cluster-docker-swarm's Issues

Initial failover no more happening with docker 18.06.1-ce and 18.06.0-ce

Hi,

after upgrading to docker 18.06.1-ce, the initial failover when removing redis-zero is no more happening:

Sentinel logs :

>docker logs -f cache_redis-sentinel.kv6tvkjf1pe4wti7iq1h909jo.2l557usypfqotzon8afqx1tfa
17:X 14 Sep 08:38:51.430 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
17:X 14 Sep 08:38:51.430 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=17, just started
17:X 14 Sep 08:38:51.430 # Configuration loaded
17:X 14 Sep 08:38:51.432 * Running mode=sentinel, port=26379.
17:X 14 Sep 08:38:51.432 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
17:X 14 Sep 08:38:51.442 # Sentinel ID is 499f4b1712ba02ac782d364eea425c755eafbe9c
17:X 14 Sep 08:38:51.443 # +monitor master redismaster 10.0.0.38 6379 quorum 2
17:X 14 Sep 08:38:53.778 * +sentinel sentinel 516b21ecc1df8f716ca97f45a00d91808edd05f8 10.0.0.43 26379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:38:56.016 * +sentinel sentinel 45ab2152f72756d4808d2719b7efcca382dc549f 10.0.0.41 26379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:01.527 * +slave slave 10.0.0.3:6379 10.0.0.3 6379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:01.535 * +slave slave 10.0.0.4:6379 10.0.0.4 6379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:01.537 * +slave slave 10.0.0.2:6379 10.0.0.2 6379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:02.545 # +sdown slave 10.0.0.4:6379 10.0.0.4 6379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:02.545 # +sdown slave 10.0.0.2:6379 10.0.0.2 6379 @ redismaster 10.0.0.38 6379
17:X 14 Sep 08:39:02.545 # +sdown slave 10.0.0.3:6379 10.0.0.3 6379 @ redismaster 10.0.0.38 6379

Redis logs :

>docker logs -f cache_redis.kv6tvkjf1pe4wti7iq1h909jo.97f3k98b4hiijljxx6gqfne7y
15:C 14 Sep 08:38:51.556 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
15:C 14 Sep 08:38:51.556 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=15, just started
15:C 14 Sep 08:38:51.556 # Configuration loaded
15:S 14 Sep 08:38:51.558 * Running mode=standalone, port=6379.
15:S 14 Sep 08:38:51.558 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15:S 14 Sep 08:38:51.558 # Server initialized
15:S 14 Sep 08:38:51.558 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15:S 14 Sep 08:38:51.558 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15:S 14 Sep 08:38:51.558 * Ready to accept connections
15:S 14 Sep 08:38:51.558 * Connecting to MASTER 10.0.0.38:6379
15:S 14 Sep 08:38:51.558 * MASTER <-> SLAVE sync started
15:S 14 Sep 08:38:51.558 * Non blocking connect for SYNC fired the event.
15:S 14 Sep 08:38:51.559 * Master replied to PING, replication can continue...
15:S 14 Sep 08:38:51.559 * Partial resynchronization not possible (no cached master)
15:S 14 Sep 08:38:51.560 * Full resync from master: 5b0f1449d4de973cc59ee8fe4b299ca678fb947b:0
15:S 14 Sep 08:38:51.619 * MASTER <-> SLAVE sync: receiving 175 bytes from master
15:S 14 Sep 08:38:51.619 * MASTER <-> SLAVE sync: Flushing old data
15:S 14 Sep 08:38:51.619 * MASTER <-> SLAVE sync: Loading DB in memory
15:S 14 Sep 08:38:51.619 * MASTER <-> SLAVE sync: Finished with success

### REMOVING redis-zero HERE

15:S 14 Sep 08:39:40.293 # Connection with master lost.
15:S 14 Sep 08:39:40.298 * Caching the disconnected master state.
15:S 14 Sep 08:39:40.693 * Connecting to MASTER 10.0.0.38:6379
15:S 14 Sep 08:39:40.693 * MASTER <-> SLAVE sync started
15:S 14 Sep 08:39:51.020 # Error condition on socket for SYNC: Host is unreachable
15:S 14 Sep 08:39:51.727 * Connecting to MASTER 10.0.0.38:6379
15:S 14 Sep 08:39:51.728 * MASTER <-> SLAVE sync started
15:S 14 Sep 08:39:54.092 # Error condition on socket for SYNC: Host is unreachable
15:S 14 Sep 08:39:54.737 * Connecting to MASTER 10.0.0.38:6379
15:S 14 Sep 08:39:54.738 * MASTER <-> SLAVE sync started
15:S 14 Sep 08:39:57.164 # Error condition on socket for SYNC: Host is unreachable

A docker bug or maybe a BC in last docker releases....
Do you have more info about this ?

When downgrading to docker-ce=18.03.1~ce-0~debian everything is fine

Thanks for this project ! ๐Ÿ˜ƒ

Connecting with redis-cli

I am trying to connect to the cluster with redis-cli, but am having some issues. Do you have an example of how to do this?

I am logged in to the machine running the docker master node.

Error: docker: Error response from daemon: pull access denied for redis-utils, repository does not exist or may require 'docker login'.

The image looks promising, but I'm facing this issue, even after docker login.

> bash scripts/bootstrap.sh latest
Starting redis-zero
hhvwrbkf8nym98fw11bhs5wlt
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged
Starting services
Creating service cache_redis-sentinel
Creating service cache_redis
Unable to find image 'redis-utils:latest' locally
docker: Error response from daemon: pull access denied for redis-utils, repository does not exist or may require 'docker login'.
See 'docker run --help'.
Sentinels not set up yet - sleeping
Unable to find image 'redis-utils:latest' locally
docker: Error response from daemon: pull access denied for redis-utils, repository does not exist or may require 'docker login'.
See 'docker run --help'.
Sentinels not set up yet - sleeping
Unable to find image 'redis-utils:latest' locally
docker: Error response from daemon: pull access denied for redis-utils, repository does not exist or may require 'docker login'.
See 'docker run --help'.
Sentinels not set up yet - sleeping
^C

READONLY You can't write against a read only slave.

I am success to start the stack on a swarm cluster.

i am success to connect with redis-cli and externaly with "redis desktop manager".

But when i try to execute a simple command "set foo bar", i receive this error : "-READONLY You can't write against a read only slave."

Thanks for your help...

Allow for users to set password

Hey,

I was wondering if there is an option to let users set passwords for the cluster, or is it something that is not implemented? If not, how would you suggest going about it?

Add LICENSE

I'd like to incorporate this into one of my projects but need to make sure it's licensed to allow this. Would you mind adding a LICENSE (e.g. MIT) to this repo?

build.sh question

Hello,
Are the images in your repo different from the ones build by build.sh?
For example, I want to make some custom images, and I'm using build.sh to get the redis images and apply your changes as well as mine and produce a new image.
For some reason I'm seeing strange behavior where the other nodes aren't coming up.
Even if I back out my changes and just build using your build.sh and then of course modify the yml file to include, for example: redis-utils instead of thomasjpfan/redis-utils.

Thanks,

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.