devicemapper support. about kubeadm-dind-cluster

[jellonek]

Exactly that it's the case.
Instead of using device mapper please try overlay2 graph driver (needs change in /etc/sysconfig/docker-storage - this is the path on fedora, probably the same is on CentOS).

[warmchang]

Hi @jellonek , thanks for your advice. I use the overlay2 graphdriver. Unfortunate got the same issue:

[root@localhost kubeadm-dind-cluster]# ./dind-cluster-v1.5.sh up

• Making sure DIND image is up to date
  Trying to pull repository docker.io/mirantis/kubeadm-dind-cluster ...
  v1.5: Pulling from docker.io/mirantis/kubeadm-dind-cluster
  Digest: sha256:051af9b28a1cb767e91a678d89bbaa36007606b39d1242da68f5a069481d016e
• Starting DIND container: kube-master
• Running kubeadm: init --skip-preflight-checks
  Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
  docker failed to start. Diagnostics below:
  ● docker.service - Docker Application Container Engine
  Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
  └─10-dind.conf, 20-fs.conf
  Active: failed (Result: exit-code) since Tue 2017-03-28 11:56:08 UTC; 23ms ago
  Docs: https://docs.docker.com
  Process: 180 ExecStart=/usr/local/bin/rundocker $DOCKER_EXTRA_OPTS (code=exited, status=1/FAILURE)
  Main PID: 180 (code=exited, status=1/FAILURE)

Mar 28 11:56:07 kube-master systemd[1]: Starting Docker Application Container Engine...
Mar 28 11:56:07 kube-master rundocker[180]: Trying to load overlay module (this may fail)
Mar 28 11:56:07 kube-master rundocker[180]: time="2017-03-28T11:56:07.773915271Z" level=info msg="libcontainerd: new containerd process, pid: 205"
Mar 28 11:56:08 kube-master rundocker[180]: time="2017-03-28T11:56:08.804184359Z" level=fatal msg="Error starting daemon: error initializing graphdriver: driver not supported"
Mar 28 11:56:08 kube-master systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Mar 28 11:56:08 kube-master systemd[1]: Failed to start Docker Application Container Engine.
Mar 28 11:56:08 kube-master systemd[1]: docker.service: Unit entered failed state.
Mar 28 11:56:08 kube-master systemd[1]: docker.service: Failed with result 'exit-code'.
*** kubeadm failed
[root@localhost kubeadm-dind-cluster]#
[root@localhost kubeadm-dind-cluster]# journalctl -xe
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for virbr0.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Withdrawing address record for fe80::42a:9e0d:a1b0:2f05 on ens33.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 192.168.10.129 on ens33.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for ens33.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for lo.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Host name conflict, retrying with linux-114
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::3c79:e5ff:fe46:a96f on veth32f3ffb..
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::42:fbff:fe3a:b7f8 on br-530c75b72323..
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for 10.192.0.1 on br-530c75b72323.IPv4.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for 172.17.0.1 on docker0.IPv4.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.42.1 on virbr1.IPv4.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.122.1 on virbr0.IPv4.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::42a:9e0d:a1b0:2f05 on ens33..
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::4341:5582:cd38:9d80 on ens33..
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.10.129 on ens33.IPv4.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Server startup complete. Host name is linux-114.local. Local service cookie is 358647450.
Mar 28 19:56:12 localhost.localdomain avahi-daemon[708]: Registering HINFO record with values 'X86_64'/'LINUX'.
Mar 28 19:56:19 localhost.localdomain kernel: br-530c75b72323: port 1(veth32f3ffb) entered forwarding state
Mar 28 19:56:22 localhost.localdomain kernel: dind0: port 1(eth0) entered forwarding state
Mar 28 19:56:22 localhost.localdomain dockerd-current[1061]: time="2017-03-28T19:56:22.256622979+08:00" level=error msg="Error trying v2 registry: context canceled"
Mar 28 19:56:22 localhost.localdomain dockerd-current[1061]: time="2017-03-28T19:56:22.256680048+08:00" level=error msg="Not continuing with pull after error: context cancele
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for fe80::3c79:e5ff:fe46:a96f on veth32f3ffb.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for veth32f3ffb.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for fe80::42:fbff:fe3a:b7f8 on br-530c75b72323.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 10.192.0.1 on br-530c75b72323.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for br-530c75b72323.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 172.17.0.1 on docker0.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for docker0.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for virbr1-nic.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 192.168.42.1 on virbr1.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for virbr1.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for virbr0-nic.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 192.168.122.1 on virbr0.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for virbr0.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for fe80::42a:9e0d:a1b0:2f05 on ens33.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing address record for 192.168.10.129 on ens33.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for ens33.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Withdrawing workstation service for lo.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Host name conflict, retrying with linux-115
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::3c79:e5ff:fe46:a96f on veth32f3ffb..
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::42:fbff:fe3a:b7f8 on br-530c75b72323..
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for 10.192.0.1 on br-530c75b72323.IPv4.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for 172.17.0.1 on docker0.IPv4.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.42.1 on virbr1.IPv4.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.122.1 on virbr0.IPv4.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::42a:9e0d:a1b0:2f05 on ens33..
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for fe80::4341:5582:cd38:9d80 on ens33..
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering new address record for 192.168.10.129 on ens33.IPv4.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Server startup complete. Host name is linux-115.local. Local service cookie is 358647450.
Mar 28 19:56:32 localhost.localdomain avahi-daemon[708]: Registering HINFO record with values 'X86_64'/'LINUX'.
[root@localhost kubeadm-dind-cluster]#

And the docker info & docker ps as followed.
Actually I run the CentOS vm (vmware) on the windows host OS, does this caused the issue?

[root@localhost kubeadm-dind-cluster]# docker info
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.12.6
Storage Driver: overlay2
Backing Filesystem: xfs
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host null overlay
Swarm: inactive
Runtimes: runc docker-runc
Default Runtime: docker-runc
Security Options: seccomp selinux
Kernel Version: 3.10.0-514.10.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 4
Total Memory: 7.624 GiB
Name: localhost.localdomain
ID: GDDD:OSDK:6KW5:CAHK:4LP5:22OG:R56B:FSO7:D4MF:BWGE:6UUS:P4TT
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
127.0.0.0/8
Registries: docker.io (secure)
[root@localhost kubeadm-dind-cluster]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
45f52d28907a mirantis/kubeadm-dind-cluster:v1.5 "/sbin/dind_init syst" 5 minutes ago Up 5 minutes 127.0.0.1:8080->8080/tcp kube-master
[root@localhost kubeadm-dind-cluster]#

[warmchang]

Switch to fedora, deploy again, then everything is ok:

NAME STATUS AGE
kube-master Ready,master 2h
kube-node-1 Ready 2h
kube-node-2 Ready 2h

• Access dashboard at: http://localhost:8080/ui

[root@localhost kubeadm-dind-cluster]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system dummy-2088944543-20fdn 1/1 Running 1 3h
kube-system etcd-kube-master 1/1 Running 1 3h
kube-system kube-apiserver-kube-master 1/1 Running 0 3h
kube-system kube-controller-manager-kube-master 1/1 Running 0 3h
kube-system kube-discovery-2220406887-d6gsd 1/1 Running 1 3h
kube-system kube-dns-1873809965-0jnhq 3/3 Running 0 3h
kube-system kube-proxy-03s71 1/1 Running 1 3h
kube-system kube-proxy-1j1f8 1/1 Running 1 3h
kube-system kube-proxy-k5tc5 1/1 Running 1 3h
kube-system kube-scheduler-kube-master 1/1 Running 0 3h
kube-system kubernetes-dashboard-3203962772-228xm 1/1 Running 0 44m
[root@localhost kubeadm-dind-cluster]# kubectl get deploy --all-namespaces
NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-system kube-discovery 1 1 1 1 3h
kube-system kube-dns 1 1 1 1 3h
kube-system kubernetes-dashboard 1 1 1 1 3h
[root@localhost kubeadm-dind-cluster]# kubectl get node
NAME STATUS AGE
kube-master Ready,master 3h
kube-node-1 Ready 3h
kube-node-2 Ready 3h
[root@localhost kubeadm-dind-cluster]#

Thanks @jellonek for help.