kubernetes-sigs / sig-windows-dev-tools Goto Github PK
View Code? Open in Web Editor NEWThis is a batteries included local development environment for Kubernetes on Windows.
License: Apache License 2.0
This is a batteries included local development environment for Kubernetes on Windows.
License: Apache License 2.0
Pulling the windows images often takes a long time, slowing down iteration time.
We should establish a way to prepull the images locally on the host and then move/load them into the Windows machine.
Since you cant do something like docker pull...
for windows images when you're on mac/linux, we need to download them as tarballs and once the Windows machine comes up, load that tarball of images.
Investigate why Windows pods are on crashloopback, logs attached:
vagrant@controlplane:~$ kubectl logs windows-server-iis-7985c648cc-gmtq9
Success Restart Needed Exit Code Feature Result
------- -------------- --------- --------------
True No Success {Common HTTP Features, Default Documen...
Invoke-WebRequest : The remote name could not be resolved:
'dotnetbinaries.blob.core.windows.net'
At line:1 char:32
+ ... Web-Server; Invoke-WebRequest -UseBasicParsing -Uri 'https://dotnetbi ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:Htt
pWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
ll.Commands.InvokeWebRequestCommand
C:\ServiceMonitor.exe : The term 'C:\ServiceMonitor.exe' is not recognized as
the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is
correct and try again.
At line:1 char:311
+ ... ml>' > C:\inetpub\wwwroot\default.html; C:\ServiceMonitor.exe 'w3svc' ...
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\ServiceMonitor.exe:String) [
], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
from Jay:
I guess its failing bc it does some weird DNS request. probably bc that private-network doesnt have public DNS. Thats fine, maybe we can just remove thet Invoke-WebRequest call from the IIS pod
Anything we can do to make the explicit steps obvious will be nice for our overall UX as we add more customizations
all: build-binaries vagrant-up
0-pull-kubernetes:
1-build-binaries:
chmod +x build.sh
./build.sh $(path)
2-vagrant-up:
vagrant destroy -f && vagrant up
# 3-install-cni-antrea:
# 4-install-cni-calico:
4-e2e-test:
sonobuoy run --e2e-focus=...
We would like to have the sonobuoy tests to run
This will require a long term owner - so hope someone wants to sign up for it, really cool project.
We need to build our own vagrant boxes, possibly w/ image-builder and packer.
These would then probably bootstrap and install alot faster.
https://github.com/kubernetes-sigs/image-builder/tree/master/images/capi most likely can be used as the basis for how this is done .
To get started:
Now that we have made some progress with Windows image-builder, let's add Choco to it.
This allows us to take care of other items from our Windows wish list like:
guest tools
vim
More package proposes are welcome.
if we just
python SimpleHTTPServer -p 80 /artifacts/
/artifacts/
and then we have the windows node do
Then we could really easily run fast performant windows provisioning
... IT ALSO allows us to build a customer vagrant LINUX box that hosts all the WINDOWS artifacts,
thus allowing us to leverage ultra fast images that have all the stuff in them, preloaded.
We could use a tool like imgpkg
to contain up all the windows artifacts, and extract them using something like imgpkg extract gcr.io/friedrichwilkenasdfasdfs/my-windows-artifacts
Investigate why windows pods are hung with ContainerCreating
and after ~15 minutes they start to run.
reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlzhm\" (UniqueName: \"kubernetes.io/projected/6471b4a8-1c66-41c2-8542-7e8436d1cf7c-kube-api-access-wlz
hm\") pod \"windows-server-iis-7985c648cc-5xfhr\" (UID: \"6471b4a8-1c66-41c2-8542-7e8436d1cf7c\") "
... after ~15-20 min
fake_memory_manager.go:50] "Add container" pod="default/porter" containerName="porter" containerID="59c9c7636f5783556d8e6c20d24f080b311e9fca28624b8ba4d895e14816c0c5"
fake_topology_manager.go:43] "AddContainer" pod="default/porter" containerName="porter" containerID="59c9c7636f5783556d8e6c20d24f080b311e9fca28624b8ba4d895e14816c0c5"
We used to download and execute Install-Containerd.ps1. Meanwhile, we moved on to use our own version of that script, yet we still have a line somewhere to execute the not existing downloaded version. This causes a (not breaking) error that makes it harder to scan the output for actually relevant error messages:
winw1: C:\k\Install-Containerd.ps1 : The term 'C:\k\Install-Containerd.ps1' is not recognized as the name of a cmdlet,
winw1: function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the
winw1: path is correct and try again.
winw1: At line:1 char:1
winw1: + C:\k\Install-Containerd.ps1
winw1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~
winw1: + CategoryInfo : ObjectNotFound: (C:\k\Install-Containerd.ps1:String) [], CommandNotFoundException
winw1: + FullyQualifiedErrorId : CommandNotFoundException
winw1:
winw1: Running: sync/kubejoin.ps1 as c:\tmp\vagrant-shell.ps1
winw1: At C:\tmp\vagrant-shell.ps1:1 char:4
winw1: + --cri-socket="npipe:////./pipe/containerd-containerd"--cri-socket "n ...
winw1: + ~
winw1: Missing expression after unary operator '--'.
winw1: At C:\tmp\vagrant-shell.ps1:1 char:4
winw1: + --cri-socket="npipe:////./pipe/containerd-containerd"--cri-socket "n ...
winw1: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
winw1: Unexpected token 'cri-socket="npipe:////./pipe/containerd-containerd"--cri-socket' in expression or statement.
winw1: + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
winw1: + FullyQualifiedErrorId : MissingExpressionAfterOperator
I'm trying to get this to run on my Macbook but ran into #20 and also the above issue. Not certain yet how to resolve this one; I was hoping it would be apparent to someone more familiar with the codebase already. Otherwise I'm happy to dig into it a bit.
Pretty soon we'll be tainting our windows nodes, and along the way we'll want to also tolerate them accordingly
good first issue to add these tolerations into our smoke-test.yaml file
spec:
tolerations:
- effect: NoSchedule
key: os
operator: Equal
value: windows
- effect: NoSchedule
key: os
operator: Equal
value: Windows
Easy way to get a dev antrea instance running for CNI
Make sure you use antrea 0.13.2 ! it has a fix to a race condition in binding the vnic in containerd.
Setup antrea on linux nodes:
kubectl apply -f https://github.com/antrea-io/antrea/releases/download/v0.13.2/antrea.yml
Now its running on linux,....
mkdir -Force C:/nssm/ nssm.zip
https://k8stestinfrabinaries.blob.core.windows.net/nssm-mirror/nssm-2.24.ziptar C $global:NssmInstallDirectory -xvf .\nssm.zip --strip-components 2 */$arch/*.exeRemove-Item -Force .\nssm.zip<br class="Apple-interchange-newline">
then enable testsigning
Bcdedit.exe -set TESTSIGNING ON
Restart-Computer
Then do this:
curl.exe -LO https://raw.githubusercontent.com/antrea-io/antrea/main/hack/windows/Install-OVS.ps1
.\Install-OVS.ps1 # Test-only
.\Install-OVS.ps1 -ImportCertificate $false -Local -LocalFile <PathToOVSPackage> # Production
.... Now, follow the steps here .... (some of these steps we already did above)
https://github.com/antrea-io/antrea/blob/main/docs/windows.md#installation-as-a-service-containerd-based-runtimes
I know the windows node takes time to come up but so far its been over an hour and still reads as not ready:
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane,master 78m v1.21.0
winw1 NotReady <none> 68m v1.21.2-rc.0
The reason is:
container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
message:Network plugin returns error: cni plugin not initialized
When I ssh'd into the winw1 box and checked the kubelet process, it was started with --network-plugin=cni
. I dont usually tweak kublet args or CNI configs, so I'm not sure if the problem is within one of the listed config files or if that --node-ip=
should have had an ip value and, since it was empty, is causing the problem.
"C:\k\kubelet.exe" --container-runtime=remote --container-runtime-endpoint=npipe:////./pipe/containerd-containerd --pod-infr
a-container-image=gcr.io/k8s-staging-ci-images/pause:3.4.1 --cert-dir=C:\var\lib\kubelet\pki --config=/var/lib/kubelet/confi
g.yaml --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --hostname-ov
erride=winw1 --pod-infra-container-image=mcr.microsoft.com/oss/kubernetes/pause:1.4.1 --enable-debugging-handlers --cgroups-
per-qos=false --enforce-node-allocatable= --network-plugin=cni --resolv-conf= --log-dir=/var/log/kubelet --logtostderr=false
--image-pull-progress-deadline=20m --container-runtime=remote --container-runtime-endpoint=npipe:////.//pipe//containerd-co
ntainerd --node-ip=
FWIW this was from a cluster made using master as of today.
Can we add a flag to skip cloning kubernetes if I have a local environment setup already? At the moment this clones k8s down but I already have a copy of this on my laptop
weird winrm
error that i just saw...
An error occurred executing a remote WinRM command.
Shell: Cmd
Command: hostname
Message: unknown type: 2577786541
For some folks , virtualbox doesnt install properly...
not sure how we create internal IPs , probably a kubelet metadata issue,
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master Ready control-plane,master 11m v1.20.4 10.0.2.15 <none> Ubuntu 20.04.2 LTS 5.4.0-72-generic docker://20.10.5
winw1 Ready <none> 83s v1.21.0 10.20.30.11 <none> Windows Server 2019 Datacenter Evaluation 10.0.17763.1879 containerd://1.4.1
Taking from kubernetes-sigs/sig-windows-tools#169
make windows-server-image
-> call runme.sh
... with all the docs for what we do with image builder for windows ....
image-builder/
runme.sh
README.md
cloud.json
extraArgs.json
roles/
cni/
csi/
...
...
When I try to execute k.ps1 via winrm on a fresh vm I get this error:
vagrant winrm -c "powershell 'C:\sync\k.ps1'" winw1
powershell.exe : C:\sync\k.ps1 : The term 'C:\sync\k.ps1' is not recognized as the name of a cmdlet, function,
+ CategoryInfo : NotSpecified: (C:\sync\k.ps1 :...let, function, :String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.At line:1 char:1+ C:\sync\k.ps1+ ~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (C:\sync\k.ps1:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException make: *** [Makefile:29: rerun] Error 1
Same happens when I log into Windows (no ssh or winrm, directly over the VirtualBox UI) and try to run k.ps1 from powershell:
PS C:\Windows\system32> powershell "C:\sync\k.ps1"
C:\sync\k.ps1 : The term 'C:\sync\k.ps1' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ C:\sync\k.ps1
+ ~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\sync\k.ps1:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I can see c:\sync from powershell:
PS C:\Windows\system32> cd C:\
PS C:\> ls
Directory: C:\
Mode LastWriteTime Length Name
---- ------------- ------ ----
...
d----l 4/17/2021 5:05 AM sync
...
but I cant see it's content:
PS C:\> cd "C:\sync\"
PS C:\sync> dir
dir : Could not find a part of the path 'C:\sync'.
At line:1 char:1
+ dir
+ ~~~
+ CategoryInfo : ReadError: (C:\sync:String) [Get-ChildItem], DirectoryNotFoundException
+ FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetChildItemCommand
This, however, changes when I simply navigate over to c:\sync\ with the Windows Explorer:
PS C:\sync> ls
ls : Could not find a part of the path 'C:\sync'.
At line:1 char:1
+ ls
+ ~~
+ CategoryInfo : ReadError: (C:\sync:String) [Get-ChildItem], DirectoryNotFoundException
+ FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetChildItemCommand
PS C:\sync> ls
Directory: C:\sync
Mode LastWriteTime Length Name
---- ------------- ------ ----
------ 4/17/2021 5:05 AM 5567 config
------ 4/10/2021 7:12 AM 216 docker.ps1
------ 4/17/2021 3:44 AM 4801 k.ps1
So it looks like the folder needs some 'refresh'. But I don't know yet how to pull that off.
The options to solve this I can see are:
We must create a shell script for full nodes/components report (windows, CNIs, k8s objects), this way we can have a big picture of what is working/not.
Some examples:
Istio have a verify-install
for sanity report.
https://istio.io/latest/docs/reference/commands/istioctl/#istioctl-verify-install.
We should create a base image with hyperv (disabled hypervisor) and containers already setup on server core and vim installed as default for text editing to make the box as fast and small as possible
/kind feature
Kube proxy makes firewall rules , by using golang to make system calls to windows. so this is all synergistic.
:
cc doug landgraff can help intro to kube proxy overall architecture if needed
james sturevant has an example
Antrea CNI works be using the node-ip
address to find a device, and then tries to use that specific Device to pick a gateway
. That default gateway is the uplink
for OVS.
However, it looks like it wants that device to have a destinationPrefix
of 0.0.0.0/0 , as a way to find the default gateway by then extracting the NextHop value.
So the question is... can we hardcode that NextHop value in cases like, this repos, where the node ip doesnt come up w/ a device that is directly connected to a default gateway?
update this is fixed now:) antrea-io/antrea@da17b5d
Just got this at the end of a full cycle of make all
....
Shell: Powershell
Command: if ([System.Net.Dns]::GetHostName() -eq 'winw1') { exit 0 } exit 1
Message: [WSMAN ERROR CODE: 995]: <f:WSManFault Code='995' Machine='127.0.0.1' xmlns:f='http://schemas.microsoft.com/wbem/wsman/1/wsmanfault'><f:Message>The I/O operation has been aborted because of either a thread exit or an application request. </f:Message></f:WSManFault>
make: *** [vagrant-up] Error 1
We should enable SSH on the Windows nodes (because who likes using WinRM...)
Here is some powershell we use in aks-engine to start the ssh server and add authorized keys.
https://github.com/Azure/aks-engine/blob/master/parts/k8s/windowsinstallopensshfunc.ps1
vagrant up
seems to get nodes up but after 20+m of provisioning I still ended here:
winw1: This may take several seconds if the vSwitch needs to be created.
winw1: Waiting for Calico initialisation to finish...
winw1: Waiting for Calico initialisation to finish...StoredLastBootTime , CurrentLastBootTime 20210802101406.813459-420
winw1: Waiting for Calico initialisation to finish...StoredLastBootTime , CurrentLastBootTime 20210802101406.813459-420
winw1: Waiting for Calico initialisation to finish...StoredLastBootTime , CurrentLastBootTime 20210802101406.813459-420
winw1: Waiting for Calico initialisation to finish...StoredLastBootTime , CurrentLastBootTime 20210802101406.813459-420
And kubectl shows both nodes:
kubectl get nodes
NAME STATUS ROLES AGE VERSION
controlplane Ready control-plane,master 31m v1.21.0
winw1 Ready <none> 19m v1.21.2-rc.0
Then when I run sonobuoy it starts running but I can't get logs or status info and the IIS server container seems to have a problem too:
$ kubectl logs windows-server-iis-7985c648cc-k8qnc
Error from server (NotFound): the server could not find the requested resource ( pods/log windows-server-iis-7985c648cc-k8qnc)
$ sonobuoy logs
error streaming logs from container [kube-sonobuoy]: the server could not find the requested resource ( pods/log sonobuoy)%
Originally posted by @johnSchnake in vmware-tanzu/sonobuoy#1320 (comment)
controlplane: Removing insecure key from the guest if it's present...
controlplane: Key inserted! Disconnecting and reconnecting using new SSH key...
==> controlplane: Machine booted and ready!
==> controlplane: Checking for guest additions in VM...
==> controlplane: Setting hostname...
==> controlplane: Configuring and enabling network interfaces...
==> controlplane: Mounting shared folders...
controlplane: /var/sync/linux => C:/Users/jsturtevant/sig-windows-dev-tools/sync/linux
Vagrant was unable to mount VirtualBox shared folders. This is usually
because the filesystem "vboxsf" is not available. This filesystem is
made available via the VirtualBox Guest Additions and kernel module.
Please verify that these guest additions are properly installed in the
guest. This is not a bug in Vagrant and is usually caused by a faulty
Vagrant box. For context, the command attempted was:
mount -t vboxsf -o uid=1000,gid=1000,_netdev var_sync_linux /var/sync/linux
The error output from the command was:
: Invalid argument
lowered cpu/memory variables but otherwise left everything untouched:
linux_ram: 2048
linux_cpus: 2
windows_ram: 4096
windows_cpus: 2
@friedrichwilken lets make a smoke test, based off soemthing IIS, like ....
https://github.com/jayunit100/k8sprototypes/blob/master/windows/smoke-test.yaml
I would like to be able to run a cluster with an alpha build on the control plane as well the windows node, can we create a way to run that from one command like we can for Linux?
part of #46 , is making it so that we can put a local copy of antrea-agent.exe and antrea cni in the windows node. That will make it easy to test antrea modifications, specifically, we want to test the antrea/2344 patch which allows specifying a network interface for pod communication.
this maybe a vxlan networking issue, but it seems like the calico linux
node cant contact the calico windows node because its trying to route to
something using a 100.244.206.65
ip address.
In a BGP cluster, the routes are easier to reason about:
root@calico-worker:/# ip route
default via 172.18.0.1 dev eth0
172.18.0.0/16 dev eth0 proto kernel scope link src 172.18.0.7
192.168.9.128 dev cali868d924bafb scope link
blackhole 192.168.9.128/26 proto bird
192.168.9.129 dev cali73d4c1d05a0 scope link
192.168.9.130 dev calif5885c8f5f0 scope link
192.168.9.131 dev calicb414295737 scope link
192.168.9.132 dev cali05890af621a scope link
192.168.9.133 dev cali4fb2b6d5479 scope link
192.168.88.0/26 via 172.18.0.6 dev eth0 proto bird #### <-- this makes sense :)
But in our clusters, the networks dont really make sense,
vagrant@controlplane:~$ sudo su
root@controlplane:/home/vagrant# ip route
default via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15
10.0.2.2 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100
10.20.30.0/24 dev eth1 proto kernel scope link src 10.20.30.10
100.244.49.65 dev caliee300a6309c scope link
100.244.49.66 dev cali4bad977260a scope link
100.244.49.67 dev cali45019e4d786 scope link
100.244.49.68 dev cali2826d4d9b78 scope link
100.244.49.69 dev calie348bef9edc scope link
100.244.206.64/26 via 100.244.206.65 dev vxlan.calico onlink
The current flow of the code is:
edit yaml file
read makefile and pick a target
run and make sure defaults arent overriding your variables
What we want is:
kow generate --config=variables.yml # after this you can vagrant up manually
kow create --config=variables.yml
kow desroy --config=variables.yml
Implementation:
i.e. containerd
Warning FailedCreatePodSandBox 26m (x2 over 26m) kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "mcr.microsoft.com/oss/kubernetes/pause:1.4.0": failed to pull image "mcr.microsoft.com/oss/kubernetes/pause:1.4.0": failed to pull and unpack image "mcr.microsoft.com/oss/kubernetes/pause:1.4.0": failed to resolve reference "mcr.microsoft.com/oss/kubernetes/pause:1.4.0": failed to do request: Head "https://mcr.microsoft.com/v2/oss/kubernetes/pause/manifests/1.4.0": dial tcp: lookup mcr.microsoft.com: no such host
"mcr.microsoft.com/oss/kubernetes/pause:1.4.1
" --enable-debugging-handlers --cgroups-per-qos=false --enforce-node-allocatable=`"`" --network-plugin=cni --resolv-conf=`"`" --log-dir=/var/log/kubelet --logtostderr=false --image-pull-progress-deadline=20m --container-runtime=remote --container-runtime-endpoint=npipe:////.//pipe//containerd-containerd"
As a workaround to the vbox multiple nics issue.... is it possible to create VMs in VMW workstation/fusion that don't have the dual NIC setup ?
Please can we add an ssh server to the windows box
The steps below are how ansible installs SSH, on windows, these require a relatively up to date server 2019 image, there is a KB that enables this as well but these steps should be enough to understand setting this up
Requires admin rights to install
name: Install OpenSSH
win_shell: Add-WindowsCapability -online -Name OpenSSH.Server~~~~0.0.1.0
become: yes
become_method: runas
become_user: SYSTEM
retries: 5
delay: 3
register: result
until: result is not failed
name: Set default SSH shell to Powershell
win_regedit:
path: HKLM:\SOFTWARE\OpenSSH
state: present
name: DefaultShell
data: '{{ systemdrive.stdout | trim }}\Windows\System32\WindowsPowerShell\v1.0\powershell.exe'
type: string
name: Enable ssh login without a password
win_shell: Add-Content -Path "$env:ProgramData\ssh\sshd_config" -Value "PasswordAuthentication no`nPubkeyAuthentication yes"
name: Set SSH service startup mode to auto and ensure it is started
win_service:
name: sshd
start_mode: auto
state: started
We want to be able to continually re run provisioning
so, lets
Get-Service *containerd
and if installed/ running, skip containerd1 and containerd2estimating this will save us alot of time
Some examples could be
for instance I keep forgetting what the stable k8s images repo is
When creating a dev environment using a standard release the control plan has the following error
controlplane: Hit:5 http://security.ubuntu.com/ubuntu bionic-security InRelease
controlplane: Ign:6 https://apt.kubernetes.io kubernetes-xenial InRelease
controlplane: Err:7 https://apt.kubernetes.io kubernetes-xenial Release
controlplane: Could not handshake: The TLS connection was non-properly terminated. [IP: 34.107.204.206 443]
controlplane: Reading package lists...
controlplane: E
controlplane: :
controlplane: The repository 'https://apt.kubernetes.io kubernetes-xenial Release' does not have a Release file.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
Switching to deb http://packages.cloud.google.com/apt/ kubernetes-xenial main seems to fix this
one possible solution:
add a Makefile command that copies contents of 2/3/4 into cni/
on demand and have vagrant run the scripts in the cni
folder on startup as a default.
in the windows kubelet startup, we try to specify the pause image, but theres no reason to do this:
--pod-infra-container-image=`"mcr.microsoft.com/oss/kubernetes/pause:1.4.1`"
--enable-debugging-handlers
--cgroups-per-qos=false
--enforce-node-allocatable=`"`"
--network-plugin=cni
--resolv-conf=`"`"
--log-dir=/var/log/kubelet
--logtostderr=false
--image-pull-progress-deadline=20m
--container-runtime=remote
--container-runtime-endpoint=npipe:////.//pipe//containerd-containerd"
... But the kubelet itself, if
--container-runtime=remote
```, doesn't actually honor that flag, will not actually use this option
hence we actually only use 1.4.0 in ouur current containerd configs, are the ones that get used.
$config = Get-Content "$global:ConainterDPath\config.toml"
$config = $config -replace "bin_dir = (.)$", "bin_dir = "c:/opt/cni/bin
""
$config = $config -replace "conf_dir = (.)$", "conf_dir = "c:/etc/cni/net.d
""
$config | Set-Content "$global:ConainterDPath\config.toml" -Force
So, i guess this is a longwinded way of saying lets not use the `--pod-infra-container-image` flag at all :) .
i.e. in a running cluster, our windows node has this in the config.toml:
sandbox_image = "mcr.microsoft.com/oss/kubernetes/pause:1.4.0"
hence theres a mismatch that could be confusing for folks long term
this is a minor issue but figured it was worth noting in detail in case people search for it
evidently ross @rancher has a fix to the host ip issues
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.