Kubernetes: Config auto-reload? about deployment

[miekg]

[ Quoting <[email protected]> in "[coredns/deployment] Kubernetes: Co..." ]

Hi, just tested out coredns and feels great! I was kind of surprised that when I made changes to the Config, the changes didn't get applied automatically. You can send `SIGUSR1` to coredns to reload the config, but would be nice to reload automatically. Before finding any other way to implement this, just asking do you know is there any plan to implement this into the coredns Kubernetes integration?

You need to add the reload plugin for this to work, coredns.io/plugins/reload /Miek

…

-- Miek Gieben

[ernoaapa]

Oh sweet, thanks! Need to read more about coredns, looks awesome :)

[ernoaapa]

Actually, I have problems with the reload plugin on Kubernetes.
My Corefile is following:

.:53 {
    reload 10s
    errors
    health
    kubernetes cluster.local in-addr.arpa ip6.arpa {
      pods insecure
      upstream
      fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    proxy . /etc/resolv.conf
    cache 30
}
    }

If I make any change to the config, I get an error:

.:53
CoreDNS-1.1.1
linux/amd64, go1.10, 231c2c0e
2018/04/03 02:18:41 [INFO] CoreDNS-1.1.1
2018/04/03 02:18:41 [INFO] linux/amd64, go1.10, 231c2c0e
2018/04/03 02:18:41 [INFO] Running configuration MD5 = ac8e7f69920f519bce9d3f8ef43b9587
2018/04/03 02:23:34 [INFO] Reloading
2018/04/03 02:23:34 [ERROR] Corefile changed but reload failed: listen tcp :8080: bind: address already in use

[chrisohaver]

[ERROR] Corefile changed but reload failed: listen tcp :8080: bind: address already in use

This is an open bug affecting the reload plugin. There are a few open issues in coredns/coredns e.g. coredns/coredns#1604. The same behavior is also seen when sending a SIGUSR1 signal.

[chrisohaver]

I think if you disable all plugins that listen on ports (in your case health and prometheus), then graceful reloads would work. But disabling those is not typically practical.

[ernoaapa]

Ok, thanks for help.
It’s quite rare that I need to change the configs so deleting pods one by one is good workaround until issue is fixed so I’ll go with that.

[johnbelamaric]

I would like to have the reload plugin enabled by default in the K8s deployment manifest. @chrisohaver @fturib What fixes are still needed in CoreDNS to make that feasible?

[chrisohaver]

What fixes are still needed in CoreDNS to make that feasible?

AFAIK: PR coredns/coredns#1621
which aims to resolve coredns/coredns#1604 and coredns/coredns#1618

[fturib]

these issues are now resolved.
the fix is in coredns/MAIN and should be is planned to be available on next release 1.1.2

=> need then to update the corefile for K8s when 1.1.2 will be available (all fixes are in, but there is no due date defined) : https://github.com/coredns/coredns/milestone/16

[chrisohaver]

The reload plugin is now included in the Corefile created by the deployment tool.

[fturib]

@ernoaapa : following comment of @chrisohaver , I guess we can close this issue. Correct ?

[ernoaapa]

Sorry, yes this can be closed now