Coder Social home page Coder Social logo

ahmetb / wagl Goto Github PK

View Code? Open in Web Editor NEW
284.0 14.0 21.0 9.91 MB

:bee: DNS Service Discovery for Docker Swarm. Works out of the box. (NOW OBSOLETE, USE SWARM MODE)

Home Page: http://ahmetb.github.io/wagl

License: Apache License 2.0

Go 100.00%
dns-service-discovery docker-swarm

wagl's Introduction

⚠️ As of Docker v1.12 wagl is now obsolete, as Swarm is integrated into the Docker Engine and has built in service discovery and routing features. You should not be using wagl anymore. ⚠️

wagl: DNS Service Discovery for Docker Swarm v1.0

wagl runs inside your Docker Swarm cluster and provides DNS-based service discovery (using DNS A and SRV records) and simple load balancing by rotating the list of IP addresses in DNS records.

For instance, if you run your API container with command:

docker run -d -l dns.service=api -p 80:80 nginx

other containers in the cluster will be able to reach this container using URL http://api.swarm. It is a minimalist solution, yet handles most of the basic DNS service discovery functionality well ––but we're open to pull requests.

wagl runs inside a container in the Swarm cluster (preferably on manager nodes) and is easy to deploy.

  1. wagl Command-Line Interface
  2. Deploying wagl
  3. Service Naming for DNS
  4. DNS Forwarding for External Domains
  5. Best Practices

Demo

Watch the demo at: https://www.youtube.com/watch?v=H7dr6lZqw6I

5-Minute Tutorial

Let's create a Docker Swarm cluster with docker-machine and deploy a wagl container to serve as a DNS server to this cluster:

Step 0: Download docker client and docker-swarm on your machine.

Step 1: Obtain a Swarm discovery token:

$ docker run --rm swarm create
9746027c20071fdabf9347203fc380fa 

Step 2: Create a single master and 3-node Swarm cluster with docker-machine

TOKEN=9746027c20071fdabf9347203fc380fa # <-- paste your token
docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://$TOKEN swarm-m && \
  for i in {0..2}; do docker-machine create -d virtualbox --swarm --swarm-discovery token://$TOKEN swarm-$i; done

Step 3: Deploy the wagl DNS container to the Swarm master node:

docker-machine ssh swarm-m

and then run:

docker run -d --restart=always  \
    --link=swarm-agent-master:swarm \
    -v /var/lib/boot2docker/ca.pem:/certs/ca.pem \
    -v /var/lib/boot2docker/server.pem:/certs/cert.pem \
    -v /var/lib/boot2docker/server-key.pem:/certs/key.pem \
    -p 53:53/udp \
    --name=dns \
    ahmet/wagl \
      wagl --swarm tcp://swarm:3376 \
      --swarm-cert-path /certs

The following command deploys a wagl container (named dns) pointing it to a “Swarm manager” running on the same node on :3376 and starts listening for DNS queries on port 53.

After the container is working (verify with docker ps), exit the SSH prompt.

Step 4: Schedule some web server containers on your cluster.

Pay attention to how we use Docker labels (-l argument) to name our services:

$ eval $(docker-machine env --swarm swarm-m)
$ docker run -d -l dns.service=blog -p 80:80 nginx
$ docker run -d -l dns.service=api -l dns.domain=billing -p 80:80 nginx
$ docker run -d -l dns.service=api -l dns.domain=billing -p 80:80 nginx

Step 5: Verify the DNS works! Start a container in the cluster with --dns argument as IP address where the dns container running (in this case, master node) and make a request for http://blog.swarm:

$ master=$(docker-machine ip swarm-m)
$ docker run -it --dns $master busybox
/ # wget -qO- http://blog.swarm
Connecting to blog.swarm (192.168.99.101:80)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>...

Let's quit this container and launch a Debian container in the cluster to make DNS lookups to these api containers for A/SRV records:

$ docker run -it --dns $master debian
/# apt-get -q update && apt-get -qqy install dnsutils
...

/# dig +short A api.billing.swarm
192.168.99.103
192.168.99.102

/# dig +short SRV _api._tcp.billing.swarm
1 1 80 192.168.99.102.
1 1 80 192.168.99.103.

As you can notice the IP addresses are returned in random order for very naive load-balancing via the DNS records.

This is wagl in a nutshell. Play and experiment with it!

Not Implemented Properties

Some features are not implemented for the sake of minimalism. Please be aware of these before using.

  • DNSSEC
  • IPv6 records (such as type AAAA)
  • Not-so-needed record types (NS, SOA, MX etc)
  • HTTP REST API to query records
  • Proper and configurable DNS message exchange timeouts
  • DNS over TCP: currently we only do UDP, I have no idea what happens to large DNS queries or answers.
  • Recursion on external nameservers: We just randomly pick an external NS to forward the request and if that fails we don't try others, we just call it failed.
  • Staleness checks are fragile to system clock changes because Go language does not have monotonically increasing clock implementation.

For the not implemented features, we return NOTIMP status code in DNS answers and any server failure returns SERVFAIL status code.

Authors

License

This project is licensed under Apache License Version 2.0. Please refer to LICENSE.

Disclaimer

This project is affiliated neither with Microsoft Corporation nor Docker Inc.

Why the name?

It turns out the scientists obvserved that the honeybees coming back from a food source to the bee hive, they tended to waggle about excitedly in a figure 8 pattern which shares the location of the food source with other bees. This is called “The Waggle Dance”. It is actually pretty amazing, you should just watch the video.

wagl's People

Contributors

ahmetb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wagl's Issues

Can't run wagl in swarm

Hi.
I haved read your readme and watch your demo video. But when I use docker-machine create swarm and ssh to swarm-master and run

docker run -d --restart=always  \
    --link=swarm-agent-master:swarm \
    -v /var/lib/boot2docker/ca.pem:/certs/ca.pem \
    -v /var/lib/boot2docker/server.pem:/certs/cert.pem \
    -v /var/lib/boot2docker/server-key.pem:/certs/key.pem \
    -p 53:53/udp \
    --name=dns \
    ahmet/wagl \
      wagl --swarm tcp://swarm:3376 \
      --swarm-cert-path /certs

It returned

Digest: sha256:b467b5a1d62771e01856ac65ce0b79716fb938e24ba133943359a4d4bdb955dc
Status: Downloaded newer image for ahmet/wagl:latest
Error response from daemon: Conflicting options: --net=host can't be used with links. This would result in undefined behavior

Please help me.
Thankyou

How to use wagl with Docker Cloud

I had wagl working with my Swarm, but it doesn't work on http://cloud.docker.com (formerly Tutum).

wagl-5632abe2-1 | 2016-02-10T10:26:39.162468735Z 2016/02/10 10:26:39 Configuration:
wagl-5632abe2-1 | 2016-02-10T10:26:39.162522895Z  - Domain:    "swarm."
wagl-5632abe2-1 | 2016-02-10T10:26:39.162535266Z  - Listen:    ":53"
wagl-5632abe2-1 | 2016-02-10T10:26:39.162540147Z  - Swarm:     127.0.0.1:2376
wagl-5632abe2-1 | 2016-02-10T10:26:39.162546110Z    - TLS:      (verify: false)
wagl-5632abe2-1 | 2016-02-10T10:26:39.162550452Z  - External:  true (ns: [10.60.248.11:53,10.60.250.11:53])
wagl-5632abe2-1 | 2016-02-10T10:26:39.162554158Z  - Refresh:   Every 15s (timeout: 10s) (staleness: 1m0s)
wagl-5632abe2-1 | 2016-02-10T10:26:39.162557849Z -------------------
wagl-5632abe2-1 | 2016-02-10T10:26:39.162812974Z 2016/02/10 10:26:39 Starting to refresh DNS records every 15s...
wagl-5632abe2-1 | 2016-02-10T10:26:39.168477573Z 2016/02/10 10:26:39 DNS server started listening at :53
wagl-5632abe2-1 | 2016-02-10T10:26:54.162741571Z 2016/02/10 10:26:54 Refreshing DNS records...
wagl-5632abe2-1 | 2016-02-10T10:26:54.163117138Z 2016/02/10 10:26:54 Refresh error (#1): error fetching cluster state: Get http://127.0.0.1:2376/containers/json?all=false: dial tcp 127.0.0.1:2376: getsockopt: connection refused 
wagl-5632abe2-1 | 2016-02-10T10:27:09.162763768Z 2016/02/10 10:27:09 Refreshing DNS records...
wagl-5632abe2-1 | 2016-02-10T10:27:09.162908847Z 2016/02/10 10:27:09 Refresh error (#2): error fetching cluster state: Get http://127.0.0.1:2376/containers/json?all=false: dial tcp 127.0.0.1:2376: getsockopt: connection refused 
wagl-5632abe2-1 | 2016-02-10T10:27:24.162765861Z 2016/02/10 10:27:24 Refreshing DNS records...
wagl-5632abe2-1 | 2016-02-10T10:27:24.162807836Z 2016/02/10 10:27:24 Refresh error (#3): error fetching cluster state: Get http://127.0.0.1:2376/containers/json?all=false: dial tcp 127.0.0.1:2376: getsockopt: connection refused 

Unable to connect to DNS server

Hi there, sorry to spam you with issues, especially after I resolve the last one my self.

I have now tried this on 2 separate swarm installs and I see the same behaviour on both.

when I start up the service, I am unable to use it. I can't attach to the container, as it is continually restarting.

I have tried to run other containers and they can't resolve any names, and if I try and use nslookup from my host I get connection time out, no servers could be reached.

docker run -d --restart=always --link=SwarmManager:swarm -v /etc/docker/ca.pem:/certs/ca.pem -v /etc/docker/node1-cert.pem:/certs/cert.pem -v /etc/docker/node1-node1-priv-key.pem:/certs/key.pem -p 53:53/udp --name=dns ahmet/wagl wagl --swarm tcp://127.0.0.1:3376 --swarm-cert-path /certs 

mjones@mlclustervs110:~$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                             PORTS                           NAMES
f01ba92a5c33        ahmet/wagl          "wagl --swarm tcp://1"   3 hours ago         Restarting (1) About an hour ago   172.17.147.110:53->53/udp       mlclustervs110/dns
9acc37833d66        registry:2          "/bin/registry /etc/d"   6 days ago          Up 23 hours                        172.17.147.110:5000->5000/tcp   mlclustervs110/registry

mjones@mlclustervs110:~$ docker run -it --dns 172.17.147.110 -l dns.service=web ubuntu
root@273644bb7c8b:/# ping web
ping: unknown host web
root@273644bb7c8b:/# ping web.swarm
ping: unknown host web.swarm
root@273644bb7c8b:/#

mjones@mlclustervs110:~$ nslookup                      
> server 172.17.147.110                                
Default server: 172.17.147.110                         
Address: 172.17.147.110#53                             
> web                                                  
;; connection timed out; no servers could be reached   
>                        


$ netstat a�-atun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0     36 172.17.147.110:22        192.168.137.1:54322     ESTABLISHED
tcp6       0      0 :::3376                 :::*                    LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN
tcp6       0      0 :::2376                 :::*                    LISTEN
tcp6       0      0 172.17.147.110:2376      172.17.0.3:46684        ESTABLISHED
tcp6       0      0 172.17.147.110:2376      172.17.0.3:46682        ESTABLISHED
tcp6       0      0 172.17.147.110:2376      172.17.0.3:43562        ESTABLISHED
udp6       0      0 :::53                   :::*
$

Docker overlay networks support

Hi,

I really like the simplicity of Wagl, it's exactly what I needed for my Swarm clusters.

However, I would like to take advantage of the new Docker Network introduced in 1.9 and it seems like the provided documentation in the README doesn't explain how to use Wagl in that context.

Is it supported? If no, is that something you are working on? If it is supported, do you have some documentation on how to set it up?

Thanks a lot!

So much love to your code!

Ahmet,

I was confused about only one thing here:

If I use BIND and wagl together, can I publish my DNS records-- that is, make it so that they forward into the global DNS system?

Thank you Ahmet!

[proposal] use polling only at start time, and then use docker events

I saw in your talk @dockercon that you often cited "huge list" polling as a problem, and that it's not really event driven.

Could you make it so the polling is only done at startup, in order to get the containers before the wagl is listnening, and then only use docker events?

it would reduce the network footprint (?) and also reduce the verbosity and repetition of the logs.

Thoughts?

SRV records RFC compliance

Hi Ahmet,
Sorry for my awful english.

A target in SRV record must be a domain name (not ip address). Consequently target ends in a dot.

dig +short SRV _api._tcp.billing.swarm
1 1 80 192.168.99.102.
1 1 80 192.168.99.103.

Sure I can delete last char in template, but it poor hack.
https://www.ietf.org/rfc/rfc2782.txt

Errata about containers.

I have been a bit confused by this bit.

other containers in the cluster will be able to reach this container using URL http://api.swarm

It made me assume that I would be resolved to the actual containers network. And not just to the network of the swarm node / container host.

Would be great if we actually resolved all the way to the individual container, and thereby didn't need to worry about exposing anything (and Ports already being in use)

Unable to resolve hosts from wagl container

2016/01/25 08:01:08 Tasks with DNS records: 7
2016/01/25 08:01:08     +RR: A discovery.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _discovery._tcp.myswarm. 10.165.41.47:8500
2016/01/25 08:01:08     +RR: SRV _discovery._udp.myswarm. 10.165.41.47:8600
2016/01/25 08:01:08     +RR: A microlith.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _microlith._tcp.myswarm. 10.165.41.47:8001
2016/01/25 08:01:08     +RR: A logspout.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _logspout._tcp.myswarm. 10.165.41.47:8000
2016/01/25 08:01:08     +RR: A registry.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _registry._tcp.myswarm. 10.165.41.47:5000
2016/01/25 08:01:08     +RR: A registry-ui.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _registry-ui._tcp.myswarm. 10.165.41.47:8081
2016/01/25 08:01:08     +RR: A micro-hipchat.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _micro-hipchat._tcp.myswarm. 10.165.41.47:32768
2016/01/25 08:01:08     +RR: A dns.myswarm. 10.165.41.47
2016/01/25 08:01:08     +RR: SRV _dns._tcp.myswarm. 10.165.41.47:1234
2016/01/25 08:01:08 Successfully refreshed records.
root@26248824d73b:/go/src/github.com/ahmetalpbalkan/wagl# cat /etc/resolv.conf 
search myswarm
nameserver 10.2.2.11
nameserver 10.1.2.11

docker exec -it 26248824d73b bash
root@26248824d73b:/go/src/github.com/ahmetalpbalkan/wagl# ping dns.myswarm                                                                                                        
**ping: unknown host**

Here's my YAML file for it:

dns:
    image: ahmet/wagl
    restart: always
    environment:
        env: ${ENV}
        dc: ${DC}
        DOMAIN: ${DOMAIN}
        SWARM_HOST: ${SWARM_HOST}
    labels:
        - dns.service=dns
    volumes:
        - /var/lib/docker/volumes/ucp-swarm-node-certs/_data:/etc/docker
    ports:
        - "1234:1234"
    dns_search: '$DOMAIN'
    command: wagl --domain $DOMAIN --swarm-cert-path /etc/docker --swarm tcp://$SWARM_HOST:2376

(I'm only exposing port 1234 to fool "registrator" container into noticing the container, FYI - nothing runs on that port).

Here's the command, showing the command looks OK:

docker inspect  -f "{{.Name}} {{.Config.Cmd}}" dns
/dns {[wagl --domain myswarm --swarm-cert-path /etc/docker --swarm tcp://10.165.41.47:2376]}

Any idea why I can't ping from the wagl host itself?

I'm not sure why this quit working.

How do I get wagl to work with my swarm

I can see my TLS .pem files like this in /etc/docker:

docker run --rm --volumes-from swarm-data ahmet/wagl ls /etc/docker`
ca.pem
server-cert.pem
server-key.pem

(being mounted from the swarm data)

But this doesn't seem to work.

docker run -d --restart=always --name=dns \
 -p 53:53/udp \
    --link "$SWARM_HASH":swarm \
    ahmet/wagl wagl --volumes-from swarm-data \
    --swarm-cert-path /etc/docker \
    --swarm tcp://swarm:2376

For some reason wagl docker logs show the following, and it keeps restarting:

Incorrect Usage.

NAME:
   wagl - DNS service discovery for Docker Swarm clusters

[ snipped....]

wagl dig doesn't work on the public IP address, only an internal IP address

It looks like I have wagl working with this:

WAGL_CONTAINER=`docker run -d --restart=always --name=dns \
    -p 53:53/udp \
    --link "$SWARM_HASH":swarm \
    --volumes-from ssl-data \
    ahmet/wagl wagl \
    --domain my.domain \
    --swarm-cert-path /etc/docker \
    --swarm tcp://swarm:2376`

where ssl-data is my cert/PEM container.

docker logs dns shows:

2015/12/22 20:59:17 Tasks with DNS records: 2
2015/12/22 20:59:17 +RR: A micro-hipchat.my.domain. 1.2.3.4
2015/12/22 20:59:17 +RR: SRV _micro-hipchat._tcp.my.domain. 1.2.3.4:8001
2015/12/22 20:59:17 +RR: A microlith.my.domain. 1.2.3.4
2015/12/22 20:59:17 +RR: SRV _microlith._tcp.my.domain. 1.2.3.4:8000
2015/12/22 20:59:17 Successfully refreshed records.

so it's looks like wagl is working, but for some reason any dig to the DNS shows this:

docker exec -it microlith dig micro-hipchat.my.domain
;; reply from unexpected source: 172.17.0.1#53, expected 104.130.0.177#53
;; reply from unexpected source: 172.17.0.1#53, expected 104.130.0.177#53

If I dig @172.17.0.1, it works fine, but I don't understand this behavior. wagl is on the IP mentioned.

docker port dns 
53/udp -> 104.130.0.177:53

using wagl out of a swarm context

I'd like to try wagl out of a cluster by using

docker run --net host --restart=always -p 53:53/udp --name=dns ahmet/wagl wagl --swarm "127.0.0.1:2375"

The problem is that every entry resolves to 0.0.0.0:

 dig @localhost _web._tcp.swarm SRV             

; <<>> DiG 9.10.3 <<>> @localhost _web._tcp.swarm SRV
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19899
;; flags: qr aa rd; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;_web._tcp.swarm.       IN  SRV

;; ANSWER SECTION:
_web._tcp.swarm.    0   IN  SRV 1 1 32770 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32772 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32769 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32768 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32774 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32776 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32775 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32773 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32777 0.0.0.0.
_web._tcp.swarm.    0   IN  SRV 1 1 32771 0.0.0.0.

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Wed Dec 09 23:11:30 CET 2015
;; MSG SIZE  rcvd: 453

Any idea how I could circumvent that?
I know it's not really the goal of this project, but could there be a "mode" where wagl would actually use the container ip instead of the host ip ?

Wagl vs DNS vs Service Discovery

Hi Ahmet,

Currently I’m working on a POC of Docker, Swarm & service discovery.
The setup is as follows:
2 Docker hosts (host A & host B)
Consul server (service discovery)
Swarm manager
Let’s say that I have 2 services: service A & service B
Service B is deployed 10 times (with random ports): 5 times on host A and 5 times on host B
When service A communicates with service B, for example, it sends a request to serviceB.example.com (hard coded)
In order to get an IP and a port, service A should query the Consul server for an SRV record.
It will get 10 ip:port pairs for which the client should apply some load-balancing logic.
Is there a simpler way to handle this without me developing a client resolver (+LB) library?
Can Wagl possibly resolve this?

I’m looking forward to your answer.
Thanks,

Royee

Container command not found or does not exist

Hi there,

Just found your project and I am very keen to give it a try.

I am running the following command on my swarm manager. The container is created, but then I also get an error message. Here is my command and output:

mjones@mlclustervs110:~$ docker run -d --restart=always
--link=SwarmManager:swarm
-v /etc/docker/ca.pem:/certs/ca.pem
-v /etc/docker/node1-cert.pem:/certs/cert.pem
-v /etc/docker/node1-node1-priv-key.pem:/certs/key.pem
-p 53:53/udp
--name= dns
ahmet/wagl --swarm tcp://172.17.147.110:3376
--swarm-cert-path /certs

8c7ed3c98605412df02ff76483ef46014aa62460f07f3c7cbd785feca3f64f17

docker: Error response from daemon: Container command not found or does not exist..

If I run the container with -i I also see this:

exec: "--swarm": executable file not found in $PATH

Any ideas?

edit

I missed the wagl command out of the run command, my mistake.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.