lhns / docker-swarm-keepalived

Operator for keepalived on docker swarm

docker-swarm-keepalived

Operator for keepalived on docker swarm.

Uses osixia/docker-keepalived.

Usage

• Enable the "ip_vs" kernel module if not enabled

lsmod | grep -P '^ip_vs\s' || (echo "modprobe ip_vs" >> /etc/modules && modprobe ip_vs)

• Set a different priority for each node

docker node update node1 --label-add KEEPALIVED_PRIORITY=100
docker node update node2 --label-add KEEPALIVED_PRIORITY=101
docker node update node3 --label-add KEEPALIVED_PRIORITY=102

• Deploy the stack

Docker images

https://github.com/lhns/docker-swarm-keepalived/pkgs/container/keepalived-swarm

Docker Stack

version: '3.8'

services:
  keepalived:
    image: ghcr.io/lhns/keepalived-swarm
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - host
    deploy:
      mode: global
      placement:
        constraints: [node.role == manager]
    environment:
      KEEPALIVED_VIRTUAL_IPS: "192.168.1.231, 192.168.1.232"

networks:
  host:
    external: true
    name: host

Helpful Links

• https://github.com/acassen/keepalived
• https://github.com/osixia/docker-keepalived
• https://geek-cookbook.funkypenguin.co.nz/ha-docker-swarm/keepalived/

License

This project uses the Apache 2.0 License. See the file called LICENSE.

docker-swarm-keepalived's Issues

Fail to start "is not a regular non-executable file"

Hi, when i use in a swarm with 3 manager nodes, one of then fails with this:

*** CONTAINER_LOG_LEVEL = 3 (info)
*** Search service in CONTAINER_SERVICE_DIR = /container/service :
*** link /container/service/keepalived/startup.sh to /container/run/startup/keepalived
*** link /container/service/keepalived/process.sh to /container/run/process/keepalived/run
*** link /container/service/keepalived/finish.sh to /container/run/process/keepalived/finish
*** Set environment for startup files
*** Environment files will be proccessed in this order : 
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.yaml
To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
/container/tool/run:294: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  env_vars = yaml.load(f)
*** Running /container/run/startup/keepalived...
*** Set environment for container process
*** Environment files will be proccessed in this order : 
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.yaml
To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
*** Running /container/run/process/keepalived/run...
Waiting config file /usr/local/etc/keepalived/keepalived.confok
Thu Nov 24 07:54:41 2022: Starting Keepalived v2.0.20 (01/22,2020)
Thu Nov 24 07:54:41 2022: Running on Linux 5.10.0-16-amd64 #1 SMP Debian 5.10.127-1 (2022-06-30) (built for Linux 4.19.36)
Thu Nov 24 07:54:41 2022: Command line: '/usr/local/sbin/keepalived' '-f' '/usr/local/etc/keepalived/keepalived.conf'
Thu Nov 24 07:54:41 2022:               '--dont-fork' '--log-console' '--log-detail' '--dump-conf'
Thu Nov 24 07:54:41 2022: Configuration file '/usr/local/etc/keepalived/keepalived.conf' is not a regular non-executable file
Thu Nov 24 07:54:41 2022: Stopped Keepalived v2.0.20 (01/22,2020)
*** /container/run/process/keepalived/run exited with status 6
*** Running /container/run/process/keepalived/finish...
*** Killing all processes...

how can i fix this?

Container failed

docker: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by docker)

docker: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by docker)

docker: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by docker)

docker: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by docker)

Any idea?
Thanks

Worker nodes

It appears that if I have keepalived on the worker nodes priority and unicast does not get set properly because we using the docker node ls command to populate those. But if we are on a worker node we don't have access to that information. Can we confirm this only works if we are using this repo on manager nodes?

Compile error after updating to 0.2.4

I received the below errors after updating to the recent 0.2.4 update. Works when I revert to 0.2.3.

Thanks for taking a look at this -- Let me know if I can provide any further logs or information to assist in troubleshooting!

[root@docker-node1 keepalived]# docker stack deploy keepalived --compose-file docker-compose.yml 
Updating service keepalived_keepalived (id: e8ca0ogd0uvcychqoc4iv86jz)
[root@docker-node1 keepalived]# docker service logs --follow keepalived_keepalived 
keepalived_keepalived.0.pgf26ltyzfux@docker-node1    | jq: error: Top-level program not given (try ".")
keepalived_keepalived.0.ucsk8nqq5x8u@docker-node1    | jq: error: Top-level program not given (try ".")
keepalived_keepalived.0.pgf26ltyzfux@docker-node1    | jq: 1 compile error
keepalived_keepalived.0.ucsk8nqq5x8u@docker-node1  | jq: 1 compile error
keepalived_keepalived.0.jrog4ek6ufz8@docker-node1    | jq: error: Top-level program not given (try ".")

Here's my config file:

version: '3.8'

services:
  keepalived:
    image: lolhens/keepalived-swarm:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker:ro
    networks:
      - host
    deploy:
      mode: global
      placement:
        constraints: [node.role == manager]
    environment:
      KEEPALIVED_VIRTUAL_IPS: "[IP_ADDRESS]"

networks:
  host:
    external: true
    name: host

Blocks Docker's Overlay Network

I have noticed when running this service on any node in a docker swarm, the docker overlay network begins to behave very strangely. For instance, containers are no longer able to talk to each other across nodes or locally. Problem goes away when docker-swarm-keepalived service is stopped.

I am not really sure what to provide to help describe the problem as clusters and swarms can quickly become pretty complex.