Is there anyway to enable --net=host on all docker containers in the v1.6 script about kubeadm-dind-cluster

[kokhang]

Hi Ivan, I work with @aramisjohnson. I got a question. Is there a way to use the docker.sock from the host in all the k8s nodes? That way, we can run whatever we need in a container with --net=host and not have the k8s nodes run with --net=host? Great project by the way!

[ivan4th]

Thanks @kokhang . You may try editing the script (dind-cluster-v1.6.sh, the place is here) and adding -v /var/run/docker.sock:/var/run/outer-docker.sock. Then you can use export DOCKER_HOST=unix:///var/run/outer-docker.sock inside DIND containers to connect to the outer docker (If you're doing this from a pod, you'll also need to mount /var/run/outer-docker.sock into the pod using hostPath).

[ivan4th]

Unfortunately I'm afraid it's not possible to set --net=host for docker containers that represent nodes. The problem is that every node must have it's own IP address, it's impossible to run several kubelets on single IP. Or did I misunderstand the question?.. Could you please describe what's the problem with rbd commands exactly?

[aramisjohnson]

hmm... the problem is I am create a rook pod/cluster inside of v1.6 cluster but when I attempt to launch a pod that mounts a created block or filesystem volume it times out waiting for mount operation to complete. When I use just straight kubeadm, this starting a pod that consumes a block volume succeeds (not Dind). The only thing different I see between what your scripts do and kubeadm init does, is weave. Although I do see weave as an option for CNI_PLUGIN in your script, but my vm just hung create the k8s cluster. Im going to restore my vm from a snapshot and try it again, to see if I can get weave to work with the mirantis script.

[ivan4th]

I didn't try Rook on DIND yet, it may be very well some deep Ceph-related issue. I'll definitely try it, although can't guarantee that I'll be able to do it quickly unfortunately. As of weave, it should work with 1.6, please try to do ./dind-cluster.sh clean before changing CNI_PLUGIN

[ivan4th]

But frankly I doubt Weave will help wigh rook, to fix this it's necessary to look inside rook's pod

[ivan4th]

Please try it and let me know whether it works, although I'm not quite sure how to add this functionality to the script myself because it's a bit exotic use of DIND.

[kokhang]

Thanks @ivan4th. This worked for me. I couldnt map the socke to /var/run/outer-docker.sock in the container node but i was able to map it to /opt/outer-docker.sock. I think this should unblock @aramisjohnson and not have to do net=host for rook.

[ivan4th]

Glad it helped. I actually thought about adding a mechanism to access host docker socket from within containers, it can be useful for e.g. implementing flexvolume driver based on docker volumes (such volumes can be accessible from all nodes in DIND, with possibility to add minikube-like automatic provisioning later). Today I stumbled upon a problem in k8s flexvolume code though, it fails to unmount volumes mounted using mount --bind, so don't know whether this will make sense unless I fix flexvolume code in k8s....

[lukaszo]

It looks like the issue is solved for now