CNI plugin not configured? about sig-windows-dev-tools

[johnSchnake]

@knabben thanks for the help. I do think that was the core of the issue. The problem seems to be that I, to try and figure out how to run it, did a cat README.md and so I saw the instructions closer to the bottom which said to just do a vagrant-up which doesn't do some of the provisioning steps.

I'll tweak the readme in a PR to try and avoid that, but this was 100% user error.

This morning I tried 2x with just a make all and once it hung (for who knows what reason, it restarted the windows machine then stalled after saying it was done rebooting but didnt do anything else). The 2nd time it worked like a charm (I was using antrea fyi)

[johnSchnake]

My cluster is still up, so if there are some debug commands I can run that would be useful, lay'em on me!

[knabben]

@johnSchnake you should be receiving 10.20.30.11 - https://github.com/kubernetes-sigs/sig-windows-dev-tools/blob/master/sync/shared/variables.yaml#L9, in the Vagrantfile.

Have you changed the settings or can you confirm what value the Vagrantfile is having?

[knabben]

when running the provision you will see something like:

    winw1: $cmd = "C:\k\kubelet.exe $global:KubeletArgs --cert-dir=$env:SYSTEMDRIVE\var\lib\kubelet\pki --config=/var/lib/kubelet/config.yaml --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --hostname-override=$(hostname) --pod-infra-container-image=`"mcr.microsoft.com/oss/kubernetes/pause:1.4.1`" --enable-debugging-handlers --cgroups-per-qos=false --enforce-node-allocatable=`"`" --network-plugin=cni --resolv-conf=`"`" --log-dir=/var/log/kubelet --logtostderr=false --image-pull-progress-deadline=20m --container-runtime=remote --container-runtime-endpoint=npipe:////.//pipe//containerd-containerd --node-ip=$WindowsNodeIP"

[johnSchnake]

So I do see that it tried to set it:

    winw1: # TODO Add dynamic node ip here !
    winw1: $cmd = "C:\k\kubelet.exe $global:KubeletArgs --cert-dir=$env:SYSTEMDRIVE\var\lib\kubelet\pki --config=/var/lib/kubelet/config.yaml --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --hostname-override=$(hostname) --pod-infra-container-image=`"mcr.microsoft.com/oss/kubernetes/pause:1.4.1`" --enable-debugging-handlers --cgroups-per-qos=false --enforce-node-allocatable=`"`" --network-plugin=cni --resolv-conf=`"`" --log-dir=/var/log/kubelet --logtostderr=false --image-pull-progress-deadline=20m --container-runtime=remote --container-runtime-endpoint=npipe:////.//pipe//containerd-containerd --node-ip=$WindowsNodeIP"

But for some reason that value was empty I guess. The IP was set as you showed in the variables file.

I'm going to try again and also see if that env var is still set after provisioning. Not sure if it just set for that step or not. But that will help me trace it back a bit. Thanks for the help so far.

[johnSchnake]

@jayunit100 had mentioned to check the C:/CalicoWindows directory but that doesnt exist locally.

I tried killing the kubelet and restarting it with the ip explitly written but I kept getting a myriad of errors and warnings about the flag values. Since I don't do a lot of kubelet-tweaking I'm not sure if that was a reasonable approach.

Going to try again with a different CNI specified.

[johnSchnake]

So I'm turning in, but I did try again with antrea CNI and got the same situation. The windows box is NotReady due to the CNI and I have the same kubelet cmd issue:

"C:\k\kubelet.exe" --container-runtime=remote --container-runtime-endpoint=npipe:////./pipe/containerd-containerd --pod-infra-container-image=gcr.io/k8s-staging-ci-images/pause:3.4.1 --cert-dir=C:\var\lib\kubelet\pki --config=/var/lib/kubelet/config.yaml
 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --hostname-override=winw1 --pod-infra-container-image=mcr.microsoft.com/oss/kubernetes/pause:1.4.1 --enable-debugging-handlers --cgroups-per-qos=fals
e --enforce-node-allocatable= --network-plugin=cni --resolv-conf= --log-dir=/var/log/kubelet --logtostderr=false --image-pull-progress-deadline=20m --container-runtime=remote --container-runtime-endpoint=npipe:////.//pipe//containerd-containerd --node-ip
=

      message: 'container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
        message:Network plugin returns error: cni plugin not initialized'

[knabben]

how are you provisioning these resources, if the folder does not exists, is because it didnt run calico steps yet.

Try a new fresh environment with:

make 2-vagrant-up 

[knabben]

@johnSchnake pls, ping me @knabben #sig-windows on slack so we can debug this closer.

[knabben]

@johnSchnake awesome you got it solved.

FWIW we are working on a fix for Antrea at: #46

It will probably bring have some issues when you start to use it.. Calico is working though.