Buddy does not deregister service from consul. about containerpilot

[tgross]

Is this a bug?

It's a known limitation. It was in the README at one point (ref cadb5e6) but it looks like I missed it when I moved the Roadmap items into Github issues. Let's mark this issue as an enhancement.

[misterbisson]

This is an interesting question in the context of marking nodes for maintenance.

Should that process accept two signals: one will suspend health checks and mark the service as down, or another (new) signal can fully remove it from Consul. Alternatively, marking nodes for maintenance could simply remove the service entirely on the current signal.

I have mixed feelings about what's "correct" here.

[tgross]

Something to think about here is that we have a limited number of signals at our disposal before we start possibly interfering with normal operations. If we use a signal for this:

• SIGHUP: reload config
• SIGUSR1: mark nodes for maintenance
• SIGUSR2: mark nodes for removal

And that's about it. The remaining signals are going to be used by the OS to manage the container or are just not right semantically.

Another thing to note here is that if the container halts on its own (i.e. it crashes), we still end up with nodes left behind in Consul. But I don't know... maybe that's ok because it differentiates between nodes intentionally removed and nodes involuntarily removed?

[tgross]

I almost hate to suggest this, but one way out of the signal namespace problem is to have a sock in the container (a la docker.sock except inside the container) that accepts some simple protocol. I'd like to avoid having an HTTP interface just because then securing it becomes a pain in the ass; this way we can assume that the container can manage itself and that any scheduler (which needs to have docker exec privileges anyway) can manage it too.

[tgross]

@misterbisson and I had a phone call about this and we've realized that the method we're using in #15 is limiting us here. If we reap the service from the discovery service instead of simply sending a FailTTL signal, we accomplish the same thing from the perspective of doing temporary maintenance and can cover the issue of reaping as well.

I'm going to update #15 in lieu of that.

[tgross]

#15 has been merged and I've got a release awaiting #19.

[tgross]

Included in release: https://github.com/joyent/containerbuddy/releases/tag/0.0.2-alpha