Add ability to specify cluster by cluster name rather than IP about tanka

[issue-label-bot]

Issue-Label Bot is automatically applying the label kind/feature to this issue, with a confidence of 0.98. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[sh0rez]

I see the point of having a generic way to specify a cluster by name, but I value @malcolmholmes concern higher: Because it makes the cluster ambiguous, it works against our efforts to prevent applying the correct config to the wrong cluster. Only an IP really uniquely identifies a cluster.

IMHO it is the job of a correct DNS setup to deal with differences across environments (that is /etc/resolv.conf, /etc/hosts or some other way of local DNS.

I personally like using k3d for my local Kubernetes needs (https://github.com/rancher/k3d), because it allows to run the whole cluster inside of a docker container and port-binds the apiServer to localhost:6443.

[malcolmholmes]

@rfratto the apiServer param will be a URL, therefore you can specify http://minikube just as well as specifying an IP Address.

[rfratto]

@malcolmholmes right, I guess that would fix the problem with tanka but I feel it just shifts the issue elsewhere: in the general case, on each machine I'd still have to manage the hostname resolution (in /etc/resolv.conf?) every time the IP updates.

That's fine, but I still think what I'm asking for here may be useful and easier to work with since tools like minikube are already updating ~/.kube/config with the latest IP address.

It looks like something along these lines has been requested for ksonnet as well.

[malcolmholmes]

So you're asking to use a kubeconfig cluster in spec.json instead of the URL?

The issue with this is that kubeconfig is user configurable - it can differ across users, and is therefore not guaranteed consistent. This could lead to code being incorrectly deployed to the wrong cluster if a user incorrectly configured their KUBECONFIG.

[rfratto]

Yeah, the fact that it is user configurable and can differ across users is actually the behavior I want - aside from the IP changing whenever I recreate the cluster, I'm also trying to re-use the same ksonnet on multiple machines running minikube, where the only thing that's identical between them is the cluster name in KUBECONFIG.

I view this as a secondary behavior for local development rather than the preferred way to specify environments. Feel free to close this if this just isn't something that should be supported, though; I can make tooling to handle this for my specific use case.

[Nashluffy]

Only an IP really uniquely identifies a cluster.

We use teleport that acts as a proxy to our kube clusters. What this means is that all clusters share the same apiServer (hostname and port) as far as Tanka is concerned. It also disproves that an IP uniquely identifies a cluster. Here is what our "clusters" list looks like

Nash $ k config view
apiVersion: v1
kind: Config
clusters:
...
- cluster:
    certificate-authority-data: ...
    server: https://proxy-server:9999
  name: proxy-server

and then the actual context for "example-cluster" looks like

- context:
    cluster: proxy-server
    user: example-cluster-user
  name: example-cluster

Teleport does some magic with the user that it uses inside the app (I assume), but that's kinda beside the point.

Our automation still connects directly over the private network, bypassing teleport. This means there can be multiple valid contexts for the same cluster.

If we had the ability to specify something like

    "contextNames": ["example-cluster", "example-cluster-automation"],

Tanka would work with Teleport. As of now it does not work, unfortunately. Perhaps contextNames or something similar could be added, with docs noting the use/warnings?