mirantis / cri-dockerd

dockerd as a compliant Container Runtime Interface for Kubernetes

cri-dockerd

This adapter provides a shim for Docker Engine that lets you control Docker via the Kubernetes Container Runtime Interface.

Take a look at the official docs for more information.

IMPORTANT

For users running 0.2.5 or above, the default network plugin is cni. Kubernetes 1.24+ has removed kubenet and other network plumbing from upstream as part of the dockershim removal/deprecation. In order for a cluster to become operational, Calico, Flannel, Weave, or another CNI should be used.

For CI workflows, basic functionality can be provided via containernetworking/plugins.

Motivation

Mirantis and Docker have agreed to partner to maintain the shim code standalone outside Kubernetes, as a conformant CRI interface for the Docker Engine API. For Mirantis customers, that means that Docker Engine’s commercially supported version, Mirantis Container Runtime (MCR), will be CRI compliant. This means that you can continue to build Kubernetes based on the Docker Engine as before, just switching from the built in dockershim to the external one.

Mirantis and Docker intend to work together on making sure it continues to work as well as before and that it passes all the conformance tests and continues to work just like the built in version did. Mirantis will be using this in Mirantis Kubernetes Engine, and Docker will continue to ship this shim in Docker Desktop.

You can find more information about the context for this tool in Don't Panic: Kubernetes and Docker and on the Mirantis blog.

Community

We can be found on the Kubernetes Slack in the #cri-dockerd channel.

Using cri-dockerd

Install

Refer to the install page for instructions on how to install cri-dockerd using a package manager.

Advanced Setup

Installing manually

If you want to run cri-dockerd on an unsupported platform, instructions can be found on the manual install page.

To use with Kubernetes

If you want to use cri-dockerd with Kubernetes, you can find instructions on the Kubernetes page.

Developing cri-dockerd

We welcome contributions to cri-dockerd. If you would like to contribute, please refer to the development section of the official docs.

Documentation

The docs are generated using Hugo and the Geekdocs theme. Hugo will need to be installed to generate the docs found in the docs/ directory.

Editing Docs

The docs can be ran locally with hot-reloading to make editing easier. To do so, run the following command in the project's root directory:

make docs

This will launch the development server that is included with Hugo. You can then access the docs at http://localhost:1313/

cri-dockerd's Issues

metrics performance

Metrics data being pulled through cri-dockerd is much slower than querying docker directly.

Details here: rancher/rke#2716

Add a user-facing documentation home page, for installing and configuring

Currently there is some developer documentation on GitHub, and MCR documentation on docs.mirantis.com

• https://github.com/Mirantis/cri-dockerd/#build-and-install

• https://docs.mirantis.com/mcr/20.10/single/#use-mcr-with-kubernetes

But there is no information for Docker Engine users, how install a cri-dockerd package and configure crictl.

One could use GitHub Pages for this home page, perhaps ?

Ideally it should have a vanity domain like cri-dockerd.io

Like:

• https://containerd.io
• https://cri-o.io

Ubuntu 20.04 LTS deb package??

How come there is no 20.04 LTS? 22.04 is still not available in Azure so its a bit early to close up shop?

Change the documentation to match the systemd unit files

Need the new "cri-dockerd" name, too. So that it matches the README

systemctl enable --now cri-dockerd.socket

Since:

• #24

---

Alternatively, the README could be updated to reflect the systemd units ?

That is: keep the "cri-docker" name, and make sure everyone uses that.

systemctl enable --now cri-docker.socket

`stats` command failing to retrieve memory

$ time docker stats --no-stream
CONTAINER ID   NAME                                                                                                          CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O     PIDS
61cad08c09c0   k8s_metrics-server_metrics-server-8595bd7d4c-5nvnl_kube-system_c4c13514-6af6-460f-b6d4-cadabb1ca23d_0         0.34%     14.67MiB / 5.805GiB   0.25%     1.44MB / 2.42MB   0B / 8.19kB   13
af9d6b482eda   k8s_storage-provisioner_storage-provisioner_kube-system_91efee2f-9d1c-4962-8847-fc515d518896_0                0.27%     8.922MiB / 5.805GiB   0.15%     0B / 0B           0B / 0B       9
af46c0c8fa90   k8s_kube-proxy_kube-proxy-qwhqs_kube-system_bae1cac7-f02e-4836-ba60-efdf8ce1bc2b_0                            0.08%     9.422MiB / 5.805GiB   0.16%     0B / 0B           0B / 16.4kB   9
f423e70923c8   k8s_POD_kube-proxy-qwhqs_kube-system_bae1cac7-f02e-4836-ba60-efdf8ce1bc2b_0                                   0.00%     308KiB / 5.805GiB     0.01%     0B / 0B           0B / 0B       1
4f2260ec6dec   k8s_coredns_coredns-6d4b75cb6d-dhvgf_kube-system_508a1c0d-547c-45c9-8256-0c6a6bb0008f_0                       0.26%     12.04MiB / 170MiB     7.08%     257kB / 248kB     0B / 0B       11
d595622f36bc   k8s_POD_coredns-6d4b75cb6d-dhvgf_kube-system_508a1c0d-547c-45c9-8256-0c6a6bb0008f_0                           0.00%     232KiB / 5.805GiB     0.00%     257kB / 248kB     0B / 0B       1
1c5f241c831e   k8s_POD_storage-provisioner_kube-system_91efee2f-9d1c-4962-8847-fc515d518896_0                                0.00%     172KiB / 5.805GiB     0.00%     0B / 0B           0B / 0B       1
e46a35791ca9   k8s_POD_metrics-server-8595bd7d4c-5nvnl_kube-system_c4c13514-6af6-460f-b6d4-cadabb1ca23d_0                    0.00%     236KiB / 5.805GiB     0.00%     1.44MB / 2.42MB   0B / 0B       1
427cec7f3a62   k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_09d2e6db6299ac08c1c74b56109ba3d0_0   3.20%     39.98MiB / 5.805GiB   0.67%     0B / 0B           0B / 0B       16
08c6bf8af5bb   k8s_kube-apiserver_kube-apiserver-minikube_kube-system_1b39c2135b30a71bf5fa75156498160b_0                     5.78%     280MiB / 5.805GiB     4.71%     0B / 0B           0B / 0B       18
f8902332b034   k8s_etcd_etcd-minikube_kube-system_906edd533192a4db2396a938662a5271_0                                         1.75%     31.13MiB / 5.805GiB   0.52%     0B / 0B           0B / 54.3MB   12
0afbee693806   k8s_kube-scheduler_kube-scheduler-minikube_kube-system_e731e44e54402bb1350402612c5f28bb_0                     0.80%     14.12MiB / 5.805GiB   0.24%     0B / 0B           0B / 0B       11
2fcee2fd796c   k8s_POD_kube-controller-manager-minikube_kube-system_09d2e6db6299ac08c1c74b56109ba3d0_0                       0.00%     180KiB / 5.805GiB     0.00%     0B / 0B           0B / 0B       1
f032ed29ad23   k8s_POD_etcd-minikube_kube-system_906edd533192a4db2396a938662a5271_0                                          0.00%     248KiB / 5.805GiB     0.00%     0B / 0B           73.7kB / 0B   1
4ef53131c420   k8s_POD_kube-apiserver-minikube_kube-system_1b39c2135b30a71bf5fa75156498160b_0                                0.00%     176KiB / 5.805GiB     0.00%     0B / 0B           0B / 0B       1
5a53b2ae1ae8   k8s_POD_kube-scheduler-minikube_kube-system_e731e44e54402bb1350402612c5f28bb_0                                0.00%     176KiB / 5.805GiB     0.00%     0B / 0B           0B / 0B       1

real	0m2.631s
user	0m0.037s
sys	0m0.028s

$ crictl stats
CONTAINER           CPU %               MEM                 DISK                INODES
08c6bf8af5bbc       7.02                0B                  0B                  0
0afbee6938067       0.51                0B                  0B                  0
427cec7f3a622       3.28                0B                  0B                  0
4f2260ec6decd       0.29                0B                  0B                  0
61cad08c09c0f       0.43                0B                  0B                  0
af46c0c8fa906       0.07                0B                  2.294kB             0
af9d6b482eda5       0.26                0B                  0B                  0
f8902332b0346       2.45                0B                  0B                  0

real	0m32.294s
user	0m0.017s
sys	0m0.025s

There is no memory being reported back, but docker is outputting memory fine. Also, the command takes 32 seconds vs the 2.6 seconds with docker.

cri-tools 1.24 pre-release does not work with cri-dockerd 0.2.1 pre-release

The API has been upgraded to v1, which breaks the containerd and docker runtimes.

$ ./crictl info
FATA[0000] getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService 

• crictl version 1.23.0-18-g0871ebbc

• cri-dockerd 0.2.0 (a4d1895)

There is an issue to allow fallback to CRI v1alpha2:

• kubernetes-sigs/cri-tools#883

---

It is not a strict requirement to update crictl to the same version of Kubernetes.

It works, when running the latest release (cri-tools v1.23.0), but not master.

$ ./crictl-1.23.0 info
{
  "status": {
    "conditions": [
      {
        "type": "RuntimeReady",
        "status": true,
        "reason": "",
        "message": ""
      },
      {
        "type": "NetworkReady",
        "status": true,
        "reason": "",
        "message": ""
      }
    ]
  }
}

So it is possible to use cri-tools 1.19 or 1.21 or 1.23, even when using k8s 1.24.

cri-dockerd 0.2.2 self-identifies as 0.2.1

$ ls -l cri-dockerd
-rwxr-xr-x    1 root     root      52363914 Jun  8 23:11 cri-dockerd
$ ./cri-dockerd --version
cri-dockerd 0.2.1 (HEAD)

Because version.go wasn't updated:

cri-dockerd/cmd/version/version.go

Lines 4 to 5 in d627d3e

	// Version of the product
	Version = "0.2.1"

Issue on building the code

While building on vm using the build command :

cd src && go get && go build -o ../bin/cri-dockerd

The logs are:

package github.com/Mirantis/cri-dockerd/cmd: cannot find package "github.com/Mirantis/cri-dockerd/cmd" in any of:
        /usr/src/github.com/Mirantis/cri-dockerd/cmd (from $GOROOT)
        /home/vagrant/go/src/github.com/Mirantis/cri-dockerd/cmd (from $GOPATH)
package io/fs: unrecognized import path "io/fs" (import path does not begin with hostname)
package go.opentelemetry.io/otel/exporters/otlp/otlpgrpc: cannot find package "go.opentelemetry.io/otel/exporters/otlp/otlpgrpc" in any of:
        /usr/src/go.opentelemetry.io/otel/exporters/otlp/otlpgrpc (from $GOROOT)
        /home/vagrant/go/src/go.opentelemetry.io/otel/exporters/otlp/otlpgrpc (from $GOPATH)`

Add support for pinning the infra container image

KEP 2040: Kubelet CRI support

https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2040-kubelet-cri#pinned-images

• kubernetes/kubernetes#106893

Requires CRI API 1.23

Typos in systemd unit file

There are some typos, both cosmetical:

   Docs: https://docs.micrantis.com

And more severe ones, failing the start:

unknown flag: --networkplugin

README Instructions don't seem to work for 0.2.0

mkdir -p /usr/local/bin
install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
cp -a packaging/systemd/* /etc/systemd/system
sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-dockerd.service

sed: can't read /etc/systemd/system/cri-dockerd.service: No such file or directory

systemctl daemon-reload
systemctl enable cri-dockerd.service
systemctl enable --now cri-dockerd.socket

Failed to enable unit: Unit file cri-dockerd.socket does not exist.

So looks like releases would need some kind of regression testing ? (Or manual)

When doing last-minute changes, such as 50c048c

Document how to enable CNI network plugin

It seems like currently the "noop" plugin is always used ?

ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=

level=info msg="Docker cri networking managed by network plugin kubernetes.io/no-op"

In order to use CNI, one needs to supply additional params:

--network-plugin=cni --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni/cache --cni-conf-dir=/etc/cni/net.d

level=info msg="Docker cri networking managed by network plugin cni"

---

The previous default values seem to be missing, compared to dockershim ?

BEFORE (1.23)

      --network-plugin string                                    The name of the network plugin to be invoked for various events in kubelet/pod lifecycle. This docker-specific flag only works when container-runtime is set to docker. (DEPRECATED: will be removed along with dockershim.)
      --network-plugin-mtu int32                                 The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU. This docker-specific flag only works when container-runtime is set to docker. (DEPRECATED: will be removed along with dockershim.)
      --cni-bin-dir string                                       A comma-separated list of full paths of directories in which to search for CNI plugin binaries. This docker-specific flag only works when container-runtime is set to docker. (default "/opt/cni/bin") (DEPRECATED: will be removed along with dockershim.)
      --cni-cache-dir string                                     The full path of the directory in which CNI should store cache files. This docker-specific flag only works when container-runtime is set to docker. (default "/var/lib/cni/cache") (DEPRECATED: will be removed along with dockershim.)
      --cni-conf-dir string                                      The full path of the directory in which to search for CNI config files. This docker-specific flag only works when container-runtime is set to docker. (default "/etc/cni/net.d") (DEPRECATED: will be removed along with dockershim.)

AFTER (1.24)

      --network-plugin string                   <Warning: Alpha feature> The name of the network plugin to be invoked for various events in kubelet/pod lifecycle.
      --network-plugin-mtu int32                <Warning: Alpha feature> The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU.
      --cni-bin-dir string                      <Warning: Alpha feature> A comma-separated list of full paths of directories in which to search for CNI plugin binaries.
      --cni-cache-dir string                    <Warning: Alpha feature> The full path of the directory in which CNI should store cache files.
      --cni-conf-dir string                     <Warning: Alpha feature> The full path of the directory in which to search for CNI config files

Systemd unit starting the wrong socket

It seems like the socket/service pair, is still starting /var/run/dockershim.sock ?

When adding the missing fd:// for socket-activation, this feature looks missing ?

ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=""

failed to listen on "fd://": protocol "fd" not supported

As a workaround, one can ignore systemd and start unix:///var/run/cri-docker.sock

$ sudo cri-dockerd --container-runtime-endpoint unix:///var/run/cri-docker.sock

Does cri-dockerd support journald logging ??

Wanted to leverage cri-dockerd as CRI for k8s.

We tested the shim code and noticed kubectl logs doesnt work anymore when docker was configured with journald as logging driver.

Is there an option I need to enable to make cri-dockerd as proxy and leverage the functionality of docker as is??

Error logs:

-sh-4.2# 
kubectl logs test-9ng99 -n test -c test -f
failed to try resolving symlinks in path "/var/log/pods/test_test-9ng99_e5c59673-3af9-4b01-a764-4ebb6adc4313/test/5.log": lstat /var/log/pods/test_test-9ng99_e5c59673-3af9-4b01-a764-4ebb6adc4313/test/5.log: no such file or directory
-sh-4.2# 

docker info

-sh-4.2# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.12.0)

Server:
 Containers: 139
  Running: 59
  Paused: 0
  Stopped: 80
 Images: 734
 Server Version: 20.10.9
 Storage Driver: btrfs
  Build Version: Btrfs v4.9.1
  Library Version: 102
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc version: v1.1.1-0-g52de29d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.62.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 48
 Total Memory: 125.6GiB
 Name: test.local.net
 ID: 6IQV:5TXZ:ATQ5:FQK7:HSDQ:OULR:BAAN:CRME:K2GO:GXAB:U4OL:IRT7
 Docker Root Dir: /cowdata/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels: 
 Experimental: false
 Insecure Registries:
  test.loca.net:5000
  127.0.0.0/8
 Live Restore Enabled: false

Large image pull fails

I am running cri-dockerd on the socket below. I am trying to pull a large image onto the machine with crictl and it always times out after 2 mins . Is there a tunable that i can change in cri-dockerd to fix the current issue ?

I am running cri-dockerd-v0.2.0 with the following options.

cri-dockerd-v0.2.0 --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --container-runtime-endpoint=unix:///var/run/dockershim.sock --network-plugin=cni --image-pull-progress-deadline=5m0s

[root@qct32 ~]# time /usr/bin/crictl -i unix:///var/run/dockershim.sock -r unix:///var/run/dockershim.sock pull docker.io/slightlytyler/large-image-test
FATA[0119] pulling image: rpc error: code = Unknown desc = context deadline exceeded

real 1m59.087s
user 0m0.034s
sys 0m0.031s

export: not a valid identifier

Sorry to bother, I have trouble installing.
When I follow README and execute export CRI_DOCKERD_LDFLAGS=-ldflags "-X github.com/Mirantis/cri-dockerd/version.Version=${VERSION} -X github.com/Mirantis/cri-dockerd/version.PreRelease=${PRERELEASE} -X github.com/Mirantis/cri-dockerd/version.BuildTime=${BUILD_DATE} -X github.com/Mirantis/cri-dockerd/version.GitCommit=${REVISION}", the output is

-bash: export: `-X github.com/Mirantis/cri-dockerd/version.Version=0.2.0 -X github.com/Mirantis/cri-dockerd/version.PreRelease= -X github.com/Mirantis/cri-dockerd/version.BuildTime= -X github.com/Mirantis/cri-dockerd/version.GitCommit=13b3b70': not a valid identifier

I am using go with version 1.16.10. I would appreciate any help. Thanks!

Can we move the contents of src to the root

In other projects

import "github.com/Mirantis/cri-dockerd/cmd"

github.com/Mirantis/cri-dockerd/cmd: module github.com/Mirantis/cri-dockerd@latest found (v0.2.0), but does not contain package github.com/Mirantis/cri-dockerd/cmd

import "github.com/Mirantis/cri-dockerd/src/cmd"

github.com/Mirantis/cri-dockerd/src/cmd: github.com/Mirantis/cri-dockerd/[email protected]: parsing go.mod:
        module declares its path as: github.com/Mirantis/cri-dockerd
                but was required as: github.com/Mirantis/cri-dockerd/src

// go.mod
replace github.com/Mirantis/cri-dockerd => github.com/Mirantis/cri-dockerd/src v0.2.0

github.com/Mirantis/[email protected]: reading github.com/Mirantis/cri-dockerd/src/src/go.mod at revision src/v0.2.0: unknown revision src/v0.2.0

Need method of providing git commit information

When not using git to build, but a release tarball

From #19 (comment)

Optionally also support SOURCE_DATE_EPOCH.

https://reproducible-builds.org/

default socket path on Linux/Windows

hello,
i see different values for the socket path in the packaged systemd socket file:
https://github.com/Mirantis/cri-dockerd/blob/master/packaging/systemd/cri-docker.socket#L6
(has ListenStream=%t/cri-docker.sock)

and the default value in:

cri-dockerd/src/cmd/cri/options/options.go

Lines 44 to 47 in 542e27d

	if runtime.GOOS == "linux" {
	remoteRuntimeEndpoint = "unix:///var/run/cri-dockerd.sock"
	} else if runtime.GOOS == "windows" {
	remoteRuntimeEndpoint = "npipe:////./pipe/cri-dockerd"

the flag has .../cri-dockerd.sock (notice the trailing d). Windows packaging seems to not overload the default npipe:////./pipe/cri-dockerd

we have a pending kubeadm PR for k8s 1.24 where we are hardcoding the cri-dockerd "known" endpoints to the same values as in the cri-dockerd options.go above for Linux/Windows:
kubernetes/kubernetes#107317

i'm assuming that the missing d is a typo?
are we safe to use npipe:////./pipe/cri-dockerd for Windows and unix:///var/run/cri-dockerd.sock for Linux in kubeadm defaults?

cc @evol262 @afbjorklund

CRI Docker deb package failing.

.deb packages can be built from this directory with the following syntax

make deb

Artifacts will be located in debbuild under the following directory structure: debbuild/$distro-$distro_version/

Specifying a specific distro
make ubuntu

I'm trying to create cri-dockerd deb package as mentioned in the readme but getting the below error.

$make deb
Output truncated
...
dpkg-buildpackage: warning:     debian/changelog(l1): version '5:not-0~ubuntu-bionic' is invalid: version number does not start with digit
LINE: cri-docker (5:not-0~ubuntu-bionic) bionic; urgency=low
dpkg-buildpackage: error: version number does not start with digit
dpkg-buildpackage: info: source package cri-docker
dpkg-buildpackage: info: source version unknown
Makefile:70: recipe for target 'ubuntu-bionic' failed
make: *** [ubuntu-bionic] Error 255

OS details:

$pwd
cri-dockerd/packaging/deb
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

GitCommit shows as HEAD (for 0.2.0)

$ cri-dockerd --version
cri-dockerd 0.2.0 (HEAD)

Apparently the support for supplying the version and commit at build-time never materialized ?

var (
	// Version of the product
	Version = "0.2.0"
	// PreRelease is set during the build
	PreRelease = ""
	// GitCommit is set during the build
	GitCommit = "HEAD"
	// BuildTime is set during the build
	BuildTime = "<unknown>"
)

https://github.com/Mirantis/cri-dockerd/blob/master/src/version/version.go

installation issue

Not sure, whether it is the right place to ask, moreover it's not an issue seeking information.

I am new to Kubernetes, just trying to install the cluster. ---

seeing an issue while running the commands to install in Linux.
exactly at cd cri-dockerd

I don't see a directory anywhere, how can I move forward?

error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: nodes "xxxx" not found

after kubeadm init:

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 5.501531 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[kubelet-check] Initial timeout of 40s passed.
error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: nodes "xxxx" not found

Does not work if there is no `docker` group

I just installed Ubuntu 22.04 server on a new VM. The install process offered to "install" "docker", and I took the offer. Later I found out this install was done with snap. It turns out that snap install docker, at least on Ubuntu 22.04, does NOT create a usergroup named docker. But the cri-docker.socket systemd unit here assumes the existence of such a group.

how to configure cri-dockerd with calico

We pursued an implementation of cri-dockerd in kubespray kubernetes-sigs/kubespray#8623 , an ansible based kubernetes deployment tool.

In the linked PR we discovered that the cri-dockerd does not use the configured CNI resulting in incorrect pod addressing.

Failed CI runs:

• https://github.com/kubernetes-sigs/kubespray/pull/8623/checks?check_run_id=5547765685
• https://github.com/kubernetes-sigs/kubespray/pull/8623/checks?check_run_id=5547629216

The configured pod CIDR for the CI is 10.233.64.0/18 but the pods get IP addresses like 172.17.0.6 and 172.17.0.7 which are allocated from the default docker network.

I tried forcing the use of calico with setting the cri-dockerd command line like this:

ExecStart=/usr/local/bin/cri-dockerd --container-runtime-endpoint fd:// --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni --cni-conf-dir=/etc/cni/net.d --network-plugin=calico --pod-cidr=10.233.64.0/18

But I get the following error when starting cri-dockerd:

Mar 15 09:57:08 localhost cri-dockerd[48989]: time="2022-03-15T09:57:08Z" level=info msg="Using CNI configuration file /etc/cni/net.d/10-calico.conflist"
Mar 15 09:57:08 localhost cri-dockerd[48989]: time="2022-03-15T09:57:08Z" level=fatal msg="didn't find compatible CNI plugin with given settings &{HairpinMode:none NonMasqueradeCIDR:10.0.0.0/8 PluginName:calico PluginBinDirString:/opt/cni/bin PluginBinDirs:[/opt/cni/bin] PluginConfDir:/etc/cni/net.d PluginCacheDir:/var/lib/cni MTU:0}: network plugin \"calico\" not found"
Mar 15 09:57:08 localhost systemd[1]: cri-dockerd.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 09:57:08 localhost systemd[1]: cri-dockerd.service: Failed with result 'exit-code'.
Mar 15 09:57:08 localhost systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Scheduled restart job, restart counter is at 3.
Mar 15 09:57:10 localhost systemd[1]: Stopped CRI Interface for Docker Application Container Engine.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Start request repeated too quickly.
Mar 15 09:57:10 localhost systemd[1]: cri-dockerd.service: Failed with result 'exit-code'.
Mar 15 09:57:10 localhost systemd[1]: Failed to start CRI Interface for Docker Application Container Engine.

The test CNI configuration:

(venv) root@instance-1:~/kubespray# ls -l /opt/cni/bin/
total 163508
-rwxr-xr-x 1 root root  3990548 Mar 15 09:42 bandwidth
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 calico
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 calico-ipam
-rwxr-xr-x 1 root root  3357992 Mar 15 09:42 flannel
-rwxr-xr-x 1 root root  3402808 Mar 15 09:42 host-local
-rwsr-xr-x 1 root root 47026188 Mar 15 09:42 install
-rwxr-xr-x 1 root root  3472123 Mar 15 09:42 loopback
-rwxr-xr-x 1 root root  3924908 Mar 15 09:42 portmap
-rw-r--r-- 1 root root  4555575 Mar 15 09:42 tags.txt
-rwxr-xr-x 1 root root  3622648 Mar 15 09:42 tuning
(venv) root@instance-1:~/kubespray# ls -l /etc/cni/net.d/
total 12
-rw-r--r-- 1 root root  709 Mar 15 09:42 10-calico.conflist
-rw-r--r-- 1 root root  715 Mar 15 09:42 calico.conflist.template
-rw------- 1 root root 2824 Mar 15 09:42 calico-kubeconfig
(venv) root@instance-1:~/kubespray# cat /etc/cni/net.d/10-calico.conflist 
{
  "name": "cni0",
  "cniVersion":"0.3.1",
  "plugins":[
    {
      "datastore_type": "kubernetes",
      "nodename": "localhost",
      "type": "calico",
      "log_level": "info",
      "log_file_path": "/var/log/calico/cni/cni.log",
      "ipam": {
        "type": "calico-ipam",
        "assign_ipv4": "true",
        "ipv4_pools": ["10.233.64.0/18"]
      },
      "policy": {
        "type": "k8s"
      },
      "kubernetes": {
        "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
      }
    },
    {
      "type":"portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type":"bandwidth",
      "capabilities": {
        "bandwidth": true
      }
    }
  ]
}

The `amd64` go architecture is hardcoded in the Makefile

GOARCH=amd64

packaging/static/Makefile-.PHONY: static-linux
packaging/static/Makefile-static-linux:
packaging/static/Makefile-      mkdir -p build/linux/cri-dockerd
packaging/static/Makefile:      cd $(APP_DIR) && go get && env CGO_ENABLED=$(CGO_ENABLED) GOOS=linux GOARCH=amd64 go build ${CRI_DOCKERD_LDFLAGS} -o cri-dockerd
packaging/static/Makefile-      mv $(APP_DIR)/cri-dockerd build/linux/cri-dockerd/cri-dockerd
packaging/static/Makefile-      tar -C build/linux -c -z -f build/linux/cri-dockerd-$(VERSION).tgz cri-dockerd
packaging/static/Makefile-

Which means that it won't build for arm64, for instance.

ERROR: architecture for "/usr/bin/cri-dockerd" is "Advanced Micro Devices X86-64", should be "AArch64"

---

There is also a hardcoded go version, but it doesn't seem to have any affect (it uses go, from the PATH)

packaging/common.mk:GO_VERSION:=1.16.8
packaging/static/Makefile:GO_VERSION=$(shell grep "ARG GO_VERSION" $(APP_DIR)/dockerfiles/Dockerfile.dev | awk -F'=' '{print $$2}')

So it still uses the correct version, currently 1.18.1 (using the same go compiler as for the Kubernetes)

Error during install from README.md instructions

sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-dockerd.service
sed: can't read /etc/systemd/system/cri-dockerd.service: No such file or directory

Should it be 'cri-docker.service' not 'cri-dockerd.service'?

with docker-cri, k8s v1.24.1 uses docker0 as network for pod's

Hi,
I have deployed k8s 1.24.1 and used docker-cri for the first time here as dockershim is removed.
The nodes went to ready state after cluster initialization without deploying any cni, earlier with dockershim till 1.23.x I used to deploy calico then only nodes go in ready state.
I noticed that docker0 is used as network for Pod's, all Pod's take an IP from docker0. does cri comes with an inbuilt cni ??
also I found that 2 pod's are taking same IP, let's suppose 2 pod's are running on diff worker nodes, they will take IP from respective node docker0 network, hence same IP's.
Also in such a case how can pod's talk that are running on different worker node ? as each pod takes IP from respective docker0 network
Can we by any chance switch to any other cni like calico or flannel or something else ??

Please make a cri-dockerd release and package

Now that Kubernetes 1.23 is in alpha, it would be nice with a 0.1.0 release and some binaries... (like static and deb/rpm)

https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/

There is a Makefile with a few targets. make deb or make rpm will probably have you covered, and you can install the packages as normal. If you’re using a different distribution, make static will give you raw binaries you can invoke.

I expect to be able to download these, from the GibHub release "assets"

"Error deleting network when building cni runtime conf: could not retrieve port mappings: key is not found"

Hello,

I have upgraded kubernetes from 1.23 to 1.24 and added cri-dockerd between docker and kubernetes. Also upgraded cni network plugins to 1.1.1. I was also using flannel 16.3, this one I did not upgrade.

When i try to delete a pod that i already had i get:

  Normal   Killing        14m                   kubelet  Stopping container metrics-server
  Warning  FailedKillPod  4m22s (x50 over 14m)  kubelet  error killing pod: failed to "KillPodSandbox" for "fc41f3c5-292a-4e31-99c1-db87fc6f4fd5" with KillPodSandboxError: "rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"metrics-server-77dc799f5f-rh8dr_kube-system\" network: could not retrieve port mappings: key is not found"

[cloud-user@razvan-ssd bin]$ sudo systemctl status cri-docker
● cri-docker.service - CRI Interface for Docker Application Container Engine
   Loaded: loaded (/etc/systemd/system/cri-docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2022-05-11 14:49:13 UTC; 11min ago
     Docs: https://docs.mirantis.com
 Main PID: 1773162 (cri-dockerd)
    Tasks: 10
   Memory: 21.7M
   CGroup: /system.slice/cri-docker.service
           └─1773162 /usr/bin/cri-dockerd --network-plugin=cni --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni/cache --cni-conf-dir=/etc/cni/net.d --pod-infra-container-image=k8s.gcr.io/pause:3.7

May 11 15:00:14 razvan-ssd.novalocal cri-dockerd[1773162]: time="2022-05-11T15:00:14Z" level=error msg="CNI failed to delete loopback network: could not retrieve port mappings: key is not found"
May 11 15:00:14 razvan-ssd.novalocal cri-dockerd[1773162]: time="2022-05-11T15:00:14Z" level=error msg="Error deleting network when building cni runtime conf: could not retrieve port mappings: key is not found"

Please advise.

malformed import path

I am getting errors trying to build cri-dockerd in support of a new Kubernetes installation, Kubernetes version 1.24. The error messages are

go: downloading google.golang.org/appengine v1.6.5
malformed import path "github.com/Mirantis/cri-dockerd/version.Version=0.2.2": invalid char '='
malformed import path "-X": leading dash
malformed import path "github.com/Mirantis/cri-dockerd/version.PreRelease=": invalid char '='
malformed import path "github.com/Mirantis/cri-dockerd/version.BuildTime=": invalid char '='
malformed import path "github.com/Mirantis/cri-dockerd/version.GitCommit=b872f96": invalid char '='
malformed import path "-o": leading dash
package cri-dockerd is not in GOROOT (/home/tjcw/.go/src/cri-dockerd)

and the command I was trying to run was

$ go get && go build ${CRI_DOCKERD_LDFLAGS} -o cri-dockerd

I am using go version 1.18.3 on a linux/amd64 system (Ubuntu 20.04).

Can anyone help me with what the problem is ?

add CGO_ENABLED=0 to static builds

Otherwise it's possible to get issues like this..

cri-dockerd --help
cri-dockerd: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by cri-dockerd)

Enabled like this..

--- a/packaging/static/Makefile
+++ b/packaging/static/Makefile
@@ -23,7 +23,7 @@ static: static-linux cross-mac cross-win cross-arm ## create all static packages
 .PHONY: static-linux
 static-linux:
        mkdir -p build/linux/cri-dockerd
-       cd $(APP_DIR) && go get && env GOOS=linux GOARCH=amd64 go build -o cri-dockerd
+       cd $(APP_DIR) && go get && env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o cri-dockerd
        mv $(APP_DIR)/cri-dockerd build/linux/cri-dockerd/cri-dockerd
        tar -C build/linux -c -z -f build/linux/cri-dockerd-$(STATIC_VERSION).tgz cri-dockerd
 
@@ -35,14 +35,14 @@ hash_files:
 .PHONY: cross-mac
 cross-mac:
        mkdir -p build/mac/cri-dockerd
-       cd $(APP_DIR) && go get && env GOOS=darwin GOARCH=amd64 go build -o cri-dockerd-darwin-amd64
+       cd $(APP_DIR) && go get && env CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -o cri-dockerd-darwin-amd64
        mv $(APP_DIR)/cri-dockerd-darwin-amd64 build/mac/cri-dockerd/cri-dockerd
        tar -C build/mac -c -z -f build/mac/cri-dockerd-$(STATIC_VERSION).tgz cri-dockerd
 
 .PHONY: cross-win
 cross-win:
        mkdir -p build/win/cri-dockerd
-       cd $(APP_DIR) && go get && env GOOS=windows GOARCH=amd64 go build -o cri-dockerd-windows-amd64
+       cd $(APP_DIR) && go get && env CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -o cri-dockerd-windows-amd64
        mv $(APP_DIR)/cri-dockerd-windows-amd64 build/win/cri-dockerd/cri-dockerd.exe
        if ! grep -sq 'docker\|lxc' /proc/1/cgroup; then \
            docker run --rm -v $(CURDIR)/build/win:/v -w /v alpine sh -c 'apk update && apk add zip && zip -r cri-dockerd-$(STATIC_VERSION).zip cri-dockerd'; \
@@ -52,6 +52,6 @@ cross-win:
 .PHONY: cross-arm
 cross-arm: ## create tgz with linux armhf client only
        mkdir -p build/arm/cri-dockerd
-       cd $(APP_DIR) && go get && env GOOS=linux GOARCH=arm64 go build -o cri-dockerd-arm64
+       cd $(APP_DIR) && go get && env CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o cri-dockerd-arm64
        mv $(APP_DIR)/cri-dockerd-arm64 build/arm/cri-dockerd/cri-dockerd
        tar -C build/arm -c -z -f build/arm/cri-dockerd-$(STATIC_VERSION).tgz cri-dockerd

Clarifications on "Build and install" steps

I am trying the steps in build and install on Ubuntu 18.04 and am running into some issues ...

mkdir bin

Is this command supposed to be run inside a clone of the project?

export CRI_DOCKERD_LDFLAGS=-ldflags "-X github.com/Mirantis/cri-dockerd/version.Version=${VERSION} -X github.com/Mirantis/cri-dockerd/version.PreRelease=${PRERELEASE} -X github.com/Mirantis/cri-dockerd/version.BuildTime=${BUILD_DATE} -X github.com/Mirantis/cri-dockerd/version.GitCommit=${REVISION}"

This command seems to be failing when run in bash. Does it need some other setting?

export CRI_DOCKERD_LDFLAGS=-ldflags "-X github.com/Mirantis/cri-dockerd/version.Version=${VERSION} -X github.com/Mirantis/cri-dockerd/version.PreRelease=${PRERELEASE} -X github.com/Mirantis/cri-dockerd/version.BuildTime=${BUILD_DATE} -X github.com/Mirantis/cri-dockerd/version.GitCommit=${REVISION}"
bash: export: `-X github.com/Mirantis/cri-dockerd/version.Version=0.2.1 -X github.com/Mirantis/cri-dockerd/version.PreRelease= -X github.com/Mirantis/cri-dockerd/version.BuildTime= -X github.com/Mirantis/cri-dockerd/version.GitCommit=60a25a1': not a valid identifier

go get && go build ${CRI_DOCKERD_LDFLAGS} -o ../bin/cri-dockerd

Based on the -o flag, the output is supposed to be places in ../bin/cri-dockerd. Is this the same directory where the project was cloned?

unable to install rpm on centos8, missing dependency `or`

rpm -ivh cri-dockerd-0.2.1.20220525024524.cb92d7b-0.el8.x86_64.rpm
error: Failed dependencies:
or is needed by cri-dockerd-3:0.2.1.20220525024524.cb92d7b-0.el8.x86_64

rpm -qpR cri-dockerd-0.2.1.20220525024524.cb92d7b-0.el8.x86_64.rpm
/bin/sh
/bin/sh
/bin/sh
container-selinux >= 2:2.74
containerd.io >= 1.2.2-3
device-mapper-libs >= 1.02.90-1
iptables
libc.so.6()(64bit)
libc.so.6(GLIBC_2.2.5)(64bit)
libcgroup
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
libpthread.so.0(GLIBC_2.3.2)(64bit)
libseccomp >= 2.3
nftables
or
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1
systemd
tar
xz

unable to install rpm on centos-8

Failed ReopenContainerLog from runtime interface

There is an error when I use cri-dockerd as container-runtime.

Kubelet tries to reopen log file, but get the error from container runtime.

E0209 21:28:39.416074 1071258 remote_runtime.go:1134] "ReopenContainerLog from runtime service failed" err="rpc error: code = Unknown desc = docker does not support reopening container log files" containerID=....

E0209 21:28:39.981058 1071258 container_log_manager.go:244] "Container log doesn't exist, reopen container log failed" err="rpc error: code = Unknown desc = docker does not support reopening container log files" containerID=.....

It stuck when I try to run kubernetes metrics-server.
When I run the same helm chart with kubelet build in dockerhim it works fine.

kubectl version:

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23+", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25",  BuildDate:"2022-01-28T07:59:30Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

OS:  5.10.93-flatcar

Also, I find the same error as mine here rancher/rke#2716

Do you have any ideas how to fix this, thanks?

cri-docker unit binds to containerd by mistake

probably a copy/paste error, from the docker unit ?

BindsTo=containerd.service

it only needs to wait for and talk to the docker.service

since "containerd.service" doesn't even have to exist...

Consider switching back to standard layout (without "src")

After the introduction of "src", it is harder to work with the repo with standard go tools.

They sort of assume that go.mod and the resulting vendor is at the top of the repo...

Could this be reverted ?

commit 278a1fc

not support ipv6dualstak

while run k8s in dualstak, only one ip returned.
I see the codes in network/plugins.go using k8s.io/apiserver/pkg/util/feature.DefaultFeatureGate to check is ipv6DualStack enabled, but no args exports, how to enable this featuregate? should upgrade k8s after 1.23
utilfeature.DefaultFeatureGate.Enabled(kubefeatures.IPv6DualStack)

Release 0.2.1 with binary packages (deb / rpm) for end-user installation ?

Would it be possible to get a new release of cri-dockerd ?
The current 0.2.0 release still has the old socket location

Missing 50c048c

Also missing the .deb and .rpm files, mentioned in https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/

As a workaround, we are shipping v0.2.0-11-ga4d1895
(building from source, rather than using vendor binaries)

cri-dockerd 0.2.0 (a4d1895)

Support for user namespaces in Kubernetes

Hi!

I'm working on the KEP that will be implemented in 1.25 (next k8s release) to support user namespaces. We are creating an implementation for containerd and CRIO, but it will be nice if dockershim implemented that too.

I think there are some limitations docker needs to fix as a pre-requisite for the implementation. IIUC docker only supports a single ID mappings shared by all containers running in the host. There is not support for multiple ID mappings yet. However, for isolation reasons, we are using a different ID mappings for each pod in Kubernetes, which doesn't overlap with mappings of other pods either. So, we will need to use multiple ID mappings for containers, not just a single mapping shared by all containers as docker currently supports.

Some very old comments on the linked moby issue mention that this limitation might be simpler to solve once containerd 1.0 is used, which is already the case. Do you know if this limitation is indeed "easy" to fix now?

It would be great if you can implement userns support for Kubernetes pods in dockershim :)

`go get` doesn't work anymore

How do I install this thing using go install, which is apparently the new "right way"? Thanks! I'll gladly submit a README patch once I get it working.

Don't enable the cri-dockerd service (only the socket)

Since it is socket-activated, it will be started on-demand (when requested).

systemctl daemon-reload
systemctl enable cri-dockerd.service
systemctl enable --now cri-dockerd.socket

Lots of obscure error logging

I set up cri-dockerd on Ubuntu 20.04 alongside latest stable Docker and minikube (using driver 'none'), to enable minikube to work now it uses Kubernetes 1.24.

It does work, but it's furiously logging the following "error" several times per second (with varying hexadecimal values):

Jun 26 07:07:23 a cri-dockerd[941]: time="2022-06-26T07:07:23Z" level=error msg="ContainerStats resp: {0x40008700c0 linux}"

Unable to get logs

Unable to get logs after using cri-dockerd with Kubelet, everything else looks good.

crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock  logs f96a85c2686f4
FATA[0000] failed to try resolving symlinks in path "/var/log/pods/kube-system_kube-apiserver-zsm-test-0_cb68ef30a75a5673ba80d69dedf5b217/kube-apiserver/2.log": lstat /var/log/pods/kube-system_kube-apiserver-zsm-test-0_cb68ef30a75a5673ba80d69dedf5b217/kube-apiserver/2.log: no such file or directory

kubectl logs -f -n kube-system           kube-apiserver-zsm-test-0
failed to try resolving symlinks in path "/var/log/pods/kube-system_kube-apiserver-zsm-test-0_cb68ef30a75a5673ba80d69dedf5b217/kube-apiserver/2.log": lstat /var/log/pods/kube-system_kube-apiserver-zsm-test-0_cb68ef30a75a5673ba80d69dedf5b217/kube-apiserver/2.log: no such file or directory

interested in integrating with minikube

hello, in minikube we support docker, containerd and cri-o runtimes. we like to keep supporting docker runtime. I would like to know if there has already been work started for a separate dockershim that we could use in minikube?

v0.2.2: cri-dockerd --version reports v0.2.1

The application binary, at least as present in the debian buster amd64 package, reports itself as the wrong version

Bugs in version output

$ cri-dockerd --version
Kubernetes v0.0.0-master+$Format:%h$

It was supposed to show "cri-dockerd", not "Kubernetes".

And whatever the git command was it tried to use, it failed.
(There is a known bug with magic git files in the base version*)

* But cri-dockerd is supposed to output its own version, anyway.

---

VERSION=0.1.0-dev

make static

Please help in this error while installing cri-dockerd

root@Ria-PC:/home/ria# rm -r bin/
root@Ria-PC:/home/ria# mkdir bin
root@Ria-PC:/home/ria# VERSION=$((git describe --abbrev=0 --tags | sed -e 's/v//') || echo $(cat VERSION)-$(git log -1 --pretty='%h')) PRERELEASE=$(grep -q dev <<< "${VERSION}" && echo "pre" || echo "") REVISION=$(git log -1 --pretty='%h')
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
root@Ria-PC:/home/ria# go get && go build -ldflags="-X github.com/Mirantis/cri-dockerd/version.Version='$VERSION}' -X github.com/Mirantis/cri-dockerd/version.PreRelease='$PRERELEASE' -X github.com/Mirantis/cri-dockerd/version.BuildTime='$BUILD_DATE' -X github.com/Mirantis/cri-dockerd/version.GitCommit='$REVISION'" -o cri-dockerd
can't load package: package .: no Go files in /home/ria
root@Ria-PC:/home/ria#

Is this project still being maintained ?

Or should the issues be opened elsewhere instead ?

https://www.mirantis.com/blog/mirantis-to-take-over-support-of-kubernetes-dockershim-2/

Otherwise, if you’re using the open source Docker Engine, the dockershim project will be available as an open source component, and you will be able to continue to use it with Kubernetes; it will just require a small configuration change, which we will document.

ping @evol262

debian/ubuntu packages are not installable: containerd vs containerd.io

The debian packages, as-is, are not installable in a default debian or ubuntu environment: they depend on containerd.io instead of containerd. I'm not sure if the former is a package name provided by Docker's upstream packaging, to go with docker-ce, but the docker.io & related packages that ship with Debian & Ubuntu use the containerd package name.

I think this could be fixed simply by changing this line:

cri-dockerd/packaging/deb/common/control

Line 29 in 60a25a1

Depends: containerd.io (>= 1.2.2-3), iptables, libseccomp2 (>= 2.3.0), ${shlibs:Depends}

To use Depends: containerd.io (>= 1.2.2-3) | containerd(>= 1.2.2-3), so that either one will satisfy the requirements