Where should I run the etcd-manager-ctl commands? about etcd-manager

[dbarranco]

hey @eddeh83! 👋

Thank you very much for that answer, that clarified some things for me. However, I got more questions :)

• When you mention "git clone the etcd-manager repo down to the server". What do you refer by server? Your workstation? a Kubernetes master node? A development server you have?

I guess I will need to dive a bit in the code to properly understand how etcd-manager-ctl works. Cause it seems you can run the tool outside the Kubernetes cluster (dev server / personal workstation...) as the method to trigger the recover of a etcd backup seems to be related with the S3 bucket that etcd-manger reads. Maybe setting a file or modifying an existing one that will make etcd-manager trigger the recover process operation.

I think I'll craft a Dockerfile to use the etcd-manager-ctl tool meanwhile #227 gets merged. Will share it here in case it can help somebody.

Thanks again for your answer! 🤙

[edmund-nelson]

I'm not sure of where these commands should be run - but how I got it working was as follows (on Centos)

git clone the etcd-manager repo down to the server

cd /tmp
git clone https://github.com/kopeio/etcd-manager.git

Install some prereqs

yum install http://opensource.wandisco.com/centos/7/git/x86_64/wandisco-git-release-7-2.noarch.rpm
yum install pkgconfig-0.27.1-4.el7.x86_64
yum install gcc-c++-4.8.5-36.el7_6.2.x86_64
yum install zlib-devel
yum install python36-3.6.8-1.el7.x86_64
yum install patch

Download Bazel from https://github.com/bazelbuild/bazel/releases and get it on the server

Install Bazel

./bazel-0.24.0-installer-linux-x86_64.sh

Change to the cloned etcd-manager directory and use bazel to build the binaries and symlink

cd /tmp/etcd-manager
bazel build //cmd/etcd-manager //cmd/etcd-manager-ctl
ln -sf bazel-bin/cmd/etcd-manager-ctl/linux_amd64_stripped/etcd-manager-ctl
ln -sf bazel-bin/cmd/etcd-manager/linux_amd64_stripped/etcd-manager

Then all being well you can run etcd-manager-ctl from there. e.g.

./etcd-manager-ctl -alsologtostderr -v 10 -backup-store=s3://<bucket_name>/<cluster_name>/backups/etcd/main list-backups

Obviously it'd need to be set up at a proper path rather than in /tmp

I didn't need to set anything up keys wise on my cluster (Kops, 3 masters, various numbers of nodes). I suspect the cluster sorts that out for you at build.

Once I did the above - it worked - although I'm yet to successfully restore a backup - I raised issue 228 with that in mind.

[edmund-nelson]

Hi, sorry I didnt clarify, in this case the server is basically an ec2 unix box that we use to deploy and manage a k8s cluster from using kops, call it a management server if you will.
On this server I pulled down a docker image with the requisite tooling (kops, kubectl, ansible).
After deploying the cluster using kops version 12, etcd-manager was set up across the masters.

I think you would be able to have etcd-manager-ctl on one of the masters if you wanted, but would be better on a seperate management server as well for redundancy.

Etcd-manager just uses the s3 bucket used by the k8s cluster, putting its backups in a "backup" subfolder.

[dbarranco]

I think you would be able to have etcd-manager-ctl on one of the masters if you wanted, but would be better on a seperate management server as well for redundancy.
Etcd-manager just uses the s3 bucket used by the k8s cluster, putting its backups in a "backup" subfolder.

Yeah, don't worry about that. Past week I was dealing with the upgrade from the 1.11 to the 1.12 with etcd-manger + TLS comms between all the etcd peers. I was just trying to guess how the etcd-manager-ctl tool interacts with the cluster of etcd-manager's meanwhile I finish other tasks I have 😅

I have to give it a try to create an internal runbook to have it ready in case of disaster :)

Thannks!

[rdrgmnzs]

FYI etcd-manager-ctl is now also package in the docker image and released as a binary on github.