containerd / cri

Moved to https://github.com/containerd/containerd/tree/master/pkg/cri . If you wish to submit issues/PRs, please submit to https://github.com/containerd/containerd

Moved to github.com/containerd/containerd/pkg/cri

On October 7, 2020, the contents of this repo were merged into the containerd/containerd repo. For example, the source code previously stored under containerd/cri/pkg was moved to containerd/containerd/pkg/cri package.

Pull requests are no longer accepted in the master branch of this repo.

Bug-fix PRs for release/1.3 and release/1.4 branches are still accepted in this repo. However, the master branch for containerd/cri integration work is now located in the containerd/containerd repository, and as such new commits should be merged there.

This repo will be archived after the EOL of containerd 1.4.

---

cri

cri is a containerd plugin implementation of Kubernetes container runtime interface (CRI).

With it, you could run Kubernetes using containerd as the container runtime.

Current Status

cri is a native plugin of containerd 1.1 and above. It is built into containerd and enabled by default.

cri is in GA:

• It is feature complete.
• It (the GA version) works with Kubernetes 1.10 and above.
• It has passed all CRI validation tests.
• It has passed all node e2e tests.
• It has passed all e2e tests.

See test dashboard

Support Metrics

CRI-Containerd Version	Containerd Version	Kubernetes Version	CRI Version
v1.0.0-alpha.x		1.7, 1.8	v1alpha1
v1.0.0-beta.x		1.9	v1alpha1
End-Of-Life	v1.1 (End-Of-Life)	1.10+	v1alpha2
	v1.2 (Extended)	1.10+	v1alpha2
	v1.3	1.12+	v1alpha2
	v1.4	1.19+	v1alpha2

Note: The support table above specifies the Kubernetes Version that was supported at time of release of the containerd - cri integration.

The following is the current support table for containerd CRI integration taking into account that Kubernetes only supports n-3 minor release versions.

Containerd Version	Kubernetes Version	CRI Version
v1.2	1.15+	v1alpha2
v1.3	1.15+	v1alpha2
v1.4	1.19+	v1alpha2

Production Quality Cluster on GCE

For a production quality cluster on GCE brought up with kube-up.sh refer here.

Installing with Ansible and Kubeadm

For a multi node cluster installer and bring up steps using ansible and kubeadm refer here.

Custom Installation

For non ansible users, you can download the cri-containerd release tarball and deploy kubernetes cluster using kubeadm as described here.

Getting Started for Developers

Binary Dependencies and Specifications

The current release of the cri plugin has the following dependencies:

• containerd
• runc
• CNI

See versions of these dependencies cri is tested with.

As containerd and runc move to their respective general availability releases, we will do our best to rebase/retest cri with these releases on a weekly/monthly basis. Similarly, given that cri uses the Open Container Initiative (OCI) image and runtime specifications, we will also do our best to update cri to the latest releases of these specifications as appropriate.

Install Dependencies

1. Install development libraries:

• libseccomp development library. Required by cri and runc seccomp support. libseccomp-dev (Ubuntu, Debian) / libseccomp-devel (Fedora, CentOS, RHEL). On releases of Ubuntu <=Trusty and Debian <=jessie a backport version of libseccomp-dev is required. See travis.yml for an example on trusty.
• btrfs development library. Required by containerd btrfs support. btrfs-tools(Ubuntu, Debian) / btrfs-progs-devel(Fedora, CentOS, RHEL)

2. Install pkg-config (required for linking with libseccomp).
3. Install and setup a Go 1.15.14 development environment.
4. Make a local clone of this repository.
5. Install binary dependencies by running the following command from your cloned cri/ project directory:

# Note: install.deps installs the above mentioned runc, containerd, and CNI
# binary dependencies. install.deps is only provided for general use and ease of
# testing. To customize `runc` and `containerd` build tags and/or to configure
# `cni`, please follow instructions in their documents.
make install.deps

Build and Install cri

To build and install a version of containerd with the cri plugin, enter the following commands from your cri project directory:

make
sudo make install

NOTE: The version of containerd built and installed from the Makefile is only for testing purposes. The version tag carries the suffix "-TEST".

Build Tags

cri supports optional build tags for compiling support of various features. To add build tags to the make option the BUILD_TAGS variable must be set.

make BUILD_TAGS='seccomp apparmor selinux'

Build Tag	Feature	Dependency
seccomp	syscall filtering	libseccomp development library
selinux	selinux process and mount labeling
apparmor	apparmor profile support

Validate Your cri Setup

A Kubernetes incubator project called cri-tools includes programs for exercising CRI implementations such as the cri plugin. More importantly, cri-tools includes the program critest which is used for running CRI Validation Testing.

Run the CRI Validation test to validate your installation of containerd with cri built in:

make test-cri

Running a Kubernetes local cluster

If you already have a working development environment for supported Kubernetes version, you can try cri in a local cluster:

1. Start the version of containerd with cri plugin that you built and installed above as root in a first terminal:

sudo containerd

2. From the Kubernetes project directory startup a local cluster using containerd:

CONTAINER_RUNTIME=remote CONTAINER_RUNTIME_ENDPOINT='unix:///run/containerd/containerd.sock' ./hack/local-up-cluster.sh

Test

See here for information about test.

Using crictl

See here for information about using crictl to debug pods, containers, and images.

Configurations

See here for information about how to configure cri plugins and here for information about how to configure containerd

Documentation

See here for additional documentation.

Communication

For async communication and long running discussions please use issues and pull requests on this github repo. This will be the best place to discuss design and implementation.

For sync communication we have a community slack with a #containerd channel that everyone is welcome to join and chat about development.

Slack: Catch us in the #containerd and #containerd-dev channels on dockercommunity.slack.com. Click here for an invite to docker community slack.

Other Communications

As this project is tightly coupled to CRI and CRI-Tools and they are Kubernetes projects, some of our project communications take place in the Kubernetes' SIG: sig-node.

For more information about sig-node, CRI, and the CRI-Tools projects:

• sig-node community site
• Slack: #sig-node channel in Kubernetes (kubernetes.slack.com)
• Mailing List: https://groups.google.com/forum/#!forum/kubernetes-sig-node

Reporting Security Issues

If you are reporting a security issue, please reach out discreetly at [email protected].

Licenses

The containerd codebase is released under the Apache 2.0 license. The README.md file, and files in the "docs" folder are licensed under the Creative Commons Attribution 4.0 International License under the terms and conditions set forth in the file "LICENSE.docs". You may obtain a duplicate copy of the same license, titled CC-BY-4.0, at http://creativecommons.org/licenses/by/4.0/.

Project details

cri is a containerd sub-project. This project was originally established in April of 2017 in the Kubernetes Incubator program. After reaching the Beta stage, In January of 2018, the project was merged into containerd. As a containerd sub-project, you will find the:

• Project governance,
• Maintainers,
• and Contributing guidelines

information in our containerd/project repository.

cri's Issues

Add a document about how to use crictl with cri-containerd.

crictl is a CLI to interact with CRI directly for debugging purpose.

We should add a document to tell user/developer how to use crictl to help them debug.

Don't need to check `os: process already finished` any more.

This is covered by containerd now. See here.

Create permanent network namespace for sandbox.

Currently, the network teardown logic assumes that once the network namespace is gone, we don't need to teardown the network again.

However, it doesn't work well with current implementation of cri-containerd. Currently, cri-containerd creates a sandbox container for each sandbox and use the network namespace of this container. But the problem is that, once the container dies, we'll not be able to find the network namespace by /proc/${PID}/ns/net.

So based on current logic, once the sandbox container dies itself, we'll not be able to find the network namespace, thus not teardown network for it. This will cause resource leakage.

We need to maintain a permanent network namespace for each sandbox, which is similar with cri-o.
We should remove the permanent network namespace as soon as we successfully tear down the network, so as to avoid multiple teardown to the same network in most cases.

/cc @xlgao-zju

Additional work needed on image apis

1. add config.json image/image.config struct to the resolver helper
2. remaining nit TODOs
3. add auth impl
4. add list filters
5. add name resolution code

Add unit test for image management code.

The image management code is merged #21.

However, we don't have unit test for it, because:

1. The fake containerd service is not ready.
2. Containerd is still changing their image side api.

We should:

1. Add unit test as much as we can for now, e.g. current ListImages and ImageStatus purely based on metadata store, it's testable.
2. Add unit test with fake containerd service when containerd stabilizes their api and #20 is finished.

An example cni config.

We need an example cni config in user guide.

Developer Guide

It's too early to add a developer guide, but we should add one in the future when the build and development process is finalized.

Update CRI version

Kubernetes 1.7 code freeze started, we should update CRI version to the newest version kubernetes/kubernetes#45614.

Stop container with image default stop signal

Currently, when stopping a container, currently we send SIGTERM.

In the new image spec, a StopSignal field is added. After updating containerd, we should send the stop signal in the image config if it is specified.

Use containerd client.

Containerd is adding a new client, which will make integration and our unit test much easier.

We should use it after it lands.

containerd/containerd#904

Change all Sirupsen to sirupsen

Ref

• containerd/containerd#1266;
• moby/moby#34272;
• cri-o/cri-o#712.

We should change vendored Sirupsen to sirupsen after dependencies are resolved.

Enable Travis CI

We should enable travis to run presubmit build and unit test.

Check whether permanent network namespace is removed in SandboxRemove

Currently, we stop sandbox container and remove permanent network namespace when stop sandbox.

When remove sandbox, we check whether sandbox container is still running, but we should also check whether network namespace is properly removed.

A practical way to create a Kubernetes cluster.

We need to design, implement and document a practical way to create a Kubernetes cluster using containerd as container runtime.

kubeadm seems to be the way to go, and https://www.projectatomic.io/blog/2017/06/using-kubeadm-with-cri-o/ is very good reference.

We can start to run some cluster e2e test after this is done.

/cc @kubernetes-incubator/maintainers-cri-containerd

We need fake containerd services.

To better unit test functions of cri-containerd, we need a group of fake containerd services (similar with fake docker client)

Containerd now has 4 kinds of services:

• content
• execution
• images
• rootfs

We can add a fake execution service first, because the execution api is more stable. And add others later when image management code is added and needs test.

/cc @kubernetes-incubator/maintainers-cri-containerd
@heartlock Are you interested in this? This is a good start to get familiar with containerd and the code base.

Use context in the right way.

Currently, we pass the grpc handler context to containerd client directly.

Once the context is cancelled by the client, no functions calls could be made to containerd client.

The good thing is that, we can avoid proceeding when user want to cancel the context. However, the bad thing is that we the cleanup functions in defer will not be able to run, either.

Note the problem here, so that we could revisit this in the future.

Use new execution api.

The execution api of containerd is updated just now containerd/containerd#894.

This is a good move for us, we could potentially checkpoint state-independent data into the container labels, like what we are doing for docker today.

However, we still need to do checkpoint for the state-dependent data, such as exit status, container state, timestamp.

We should update containerd version and use the new execution api. This basically involves:

1. Update containerd version to containerd/containerd#894
2. Update current implementation to use new execution api.
3. Write fake client for the new execution api and add unit test.

Update ocicni vendoring

Update ocicni vendoring to master once cri-o/ocicni#1 is merged.

Put containerd-shim into pod cgroup.

We should put containerd-shim into pod cgroup, so that we could charge the resource usage to pod instead.

Ref containerd/containerd#1032 and containerd/containerd#1134.

/cc @lanchongyizu

Disable Pid namespace sharing.

Kubernetes enabled Pid namespace sharing inside a pod with Docker 1.13+.

However, some customers reported that they could not run their container, because their containers made assumptions about the pid 1.

Before resolving this issue, we could not use pid namespace sharing in production. In cri-containerd, we should also disable it by default, and probably introduce a flag to enable it.

Should not use dot in namespace.

Currently we are using k8s.io.

However, containerd will enforce dns label. containerd/containerd#1059

We should get rid of the dot on our side.

Add Container Metrics Support

According to CRI, container runtime is responsible to provide container metrics, including:

• container CPU usage.
• container memory usage.
• container writable layer size.
• Image FS usage.

See here

Containerd should have provided all those information:

• container CPU/memory usage: Provided through Prometheus metrics endpoint.
• container writable layer size: Provided by snapshotter.
• Image FS usage:
  • May need to count size of each snapshot;
  • May need to consider content store.

We should support these container metrics.

/cc @kubernetes-incubator/maintainers-cri-containerd

Pull Image Authentication

Containerd should have already supported pull image authentication. containerd/containerd#783

We should figure it out whether it's enough for us and use it.

We need a fake OS.

We need a fake OS for unit test, similar with https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/container/testing/os.go.

Checkpoint and restart recovery

There are several restart recovery problems with current cri-containerd:

1. cri-containerd restart. Because cri-containerd maintains all internal state in-memory, including sandbox list, container list and image list, once restarted all state will be lost.
2. containerd restart. When containerd restart and reconnect, there may be state mismatch between containerd and cri-containerd, e.g. a container dies during containerd is down.

To fix this, we should recover/reconcile state during cri-containerd start or after containerd restart and reconnect.

There are 3 kinds of internal state:

1. Image list. Containerd has all the information we need, we just need to list images from containerd and recover the image list.
2. Sandbox/container metadata: Most of the metadata is not provided by containerd, we need to checkpoint them for restart recovery. However, because metadata is constant, we could save it into containerd container label so as to leverage containerd metadata store to save it for us.
3. Container status: Container status is not persisted by containerd, we need to persist it ourselves. And because it's constantly changing, we may not want to abuse containerd container label to save it. So we need to maintain its checkpoint ourselves.

/cc @kubernetes-incubator/maintainers-cri-containerd

Update containerd version to fix a bug.

Ref containerd/containerd#1386

Use `repo-infra` when it is ready for our use case.

https://github.com/kubernetes/repo-infra is great tool.

Now we copied the boilerplate verification part into the repo, but removed the other go code verification.

It would be better to consolidate this in the future and only use repo-infra when it is ready for our use case.

Several related issue:

1. kubernetes/repo-infra#15
2. kubernetes/repo-infra#12
3. We don't need the build part now.
4. Subtree the repo causes DCO problem.

• If we squash commits, @mikebrow found that he could not sign the squashed commit.
• If we don't squash, that will be quite spammy.

Support UpdateContainerResources

A new CRI function UpdateContainerResources is added to update container resource constraints kubernetes/kubernetes#46105.

This is relatively lower priority for now, because Kubelet hasn't started using it yet.

However, since containerd already has corresponding interface, it's easy to do. So let's support it. :)

@kubernetes-incubator/maintainers-cri-containerd

Rename `container` to `task`

The previous container concept in containerd is changed to task.
In the refactoring PR, I've changed most of them to task.

However, there are some left, we should clean them up.

CRI Containerd Integration TODOs

Required Internal Services:

• [P0] Metadata store. A metadata store is required to store metadata and system state #9.
  • [P0] In-memory metadata store #9.
  • [P1] On-disk metadata store. (file-based or db-based)
• [P1] Container manager. A container manager keeps containerd running in runtime cgroup. (See dockershim container manager)

Container Runtime Interface Functions:

• Sandbox Lifecycle
  • [P0] RunPodSandbox
  • [P0] StopPodSandbox
  • [P0] ListPodSandbox
  • [P0] PodSandboxStatus
  • [P0] RemovePodSandbox
• Container Lifecycle
  • [P0] CreateContainer
  • [P0] StartContainer
  • [P0] ListContainers
  • [P0] ContainerStatus
  • [P0] StopContainer
  • [P0] RemoveContainer
• Image Management
  • [P0] PullImage
  • [P0] ListImages
  • [P0] ImageStatus
  • [P0] RemoveImage
• Others
  • [P0] Version
  • [P0] UpdateRuntimeConfig
• Logging/Streaming
  • [P1] Container Logging
  • [P1] ExecSync
  • [P2] Exec
  • [P2] Attach
  • [P2] PortForward
• TBDs
  • [P2] Container Metrics
  • [P2] Image Filesystem Metrics

Error happened when run `make lint`

Error happened when run make lint

root@ubuntu:/home/vcap/go-project/src/github.com/kubernetes-incubator/cri-containerd# make lint
checking lint
for directory ./cmd ...
for directory ./cmd/cri-containerd ...
WARNING: deadline exceeded by linter interfacer on ./cmd/cri-containerd (try increasing --deadline)
WARNING: deadline exceeded by linter gosimple on ./cmd/cri-containerd (try increasing --deadline)
WARNING: deadline exceeded by linter structcheck on ./cmd/cri-containerd (try increasing --deadline)
Makefile:31: recipe for target 'lint' failed
make: *** [lint] Error 2

check so slow that exceed deadline
@mikebrow @Random-Liu

Cleanup the image management code to use containerd client

After #113, cri-containerd will be using containerd client.

However, there are still some image specific code needs to cleanup, e.g. getImageInfo, localResolve etc.

We should add required functions into containerd client, and clean those functions up.

/cc @kubernetes-incubator/maintainers-cri-containerd
@abhinandanpb

Update Test Document for Node E2E

We added make test-e2e-node in #145.

We should document that in the test document https://github.com/kubernetes-incubator/cri-containerd/blob/master/docs/testing.md.

Get rid of nsenter and socat

Currently we are using nsenter for both CNI plugin and port forword.

Ideally, we should be able to use NetNS.Do to run code inside a specific network namespace.

However, golang has an issue that it may spawn new threads from locked thread, which will inherit the network namespace. golang/go#20676

We should be able to get rid of all nsenters after this is fixed.

Support NoNewPrivileges

Kubernetes 1.8 and runc both support it, we should update CRI version and support it.

Create a Makefile

We need a Makefile to manage the build, test, package and release process.

Update Kubernetes version to fix a bug

kubernetes/kubernetes#50176.

The parameters of InitCNI should be filled in reverse order in NewCRIContainerdService function

In InitCNI, the first parameter "pluginDir " means the path of cni config file.
"cniDirs" means the path of bin.

func NewCRIContainerdService(containerdEndpoint, rootDir, networkPluginBinDir, networkPluginConfDir,
	streamAddress, streamPort string) (CRIContainerdService, error) {
        ......
        netPlugin, err := ocicni.InitCNI(networkPluginBinDir, networkPluginConfDir)     <--here
	

func InitCNI(pluginDir string, cniDirs ...string) (CNIPlugin, error) {
	plugin := probeNetworkPluginsWithVendorCNIDirPrefix(pluginDir, cniDirs, "")

boilerplate verification is needed.

The initial Makefile is merged in #7.

Currently, we already have lint and fmt verification. We need a boilerplate verification.

Ref https://github.com/kubernetes/repo-infra.

Maintain resolv.conf for pod.

Previously, Docker maintains a resolv.conf for containers sharing the same network namespace.

Now, when using containerd, we need to do these ourselves. We need to:

1. Maintain a resolv.conf when creating/removing sandbox if not using HostNetwork. And initialize resolv.conf with DNSOptions in PodSandboxConfig.
2. Bind mount the resolv.conf into each container.

This logic is similar with cri-o. (See https://github.com/kubernetes-incubator/cri-o/blob/master/server/sandbox_run.go#L144-L159)

/cc @xlgao-zju

CRI-Containerd Missing Pieces

cri-containerd already supports basic container/sandbox lifecycle and image management today. However, there are still many missing pieces. List them here with relative proprieties:

Missing Features

• [P0] Privileged support. (Issue: #29, PR: #51)
• [P0] Stop container with image default stop signal. (Issue: #61, PR: #83)
• [P1] Container logging. (PR: #56)
• [P1] ExecSync. ExecSync is very useful for testing, so prioritize it.
• [P1] Pull image authentication. (Issue: #58, PR: #88)
• [P1] Sandbox /etc/resolv.conf. (Issue: #28, PR: #50)
• [P1] Sandbox /etc/hosts. (PR: #60)
• [P1] Sandbox /dev/shm. (PR: #67)
• [P1] Device mapping. (PR: #51)
• [P1] Set user/username. (PR: #146, #168)
• [P1] ExecSync timeout. (PR: #137)
• [P2] Selinux options/label. (PR: #157)
• [P2] Container streaming.
  • Exec (PR: #115)
  • Attach
  • Portforward (PR: #130)
• [P2] Support systemd cgroup. (PR: #290)
• [P3] Image list filter.
• [P2] CRI conformance. Stop running container when stopping sandbox, remove container when removing sandbox etc. (PR: #77)
• [P2] OOM Event. Handle containerd event, and set container status exit reason correspondingly. (PR: #91)
• [P2] Apparmor. (PR: #159)
• [P2] Seccomp. (PR: #219)
• [P2] Sysctl (PR: #119).
• [P2] Other pod sandbox security context (user/selinux etc.). Figure out what this means to sandbox container.
• [P2] Container metrics. (PR: #265)
• [P2] Image filesystem metrics. (PR: #257)
• [P2] Container manager. Add container manager to ensure containerd and cri-containerd are in runtime cgroup. (Issue: #181, PR: containerd/containerd#1443, #184)
• [P2] Host port. (PR: #154)

Improvements

• [P0] Switch to containerd api. Including change the implementation and update unit test (add mock client etc.). (Issue: #49)
• [P1] Switch to new containerd client. (PR: #113)
• [P1] Refactor metadata store. (PR: #66)
  • Add pure in-memory cache for image management;
  • Add in-memory wrapper for sandbox/container metadata store, because there are several things which don't need to be checkpointed.
  • Reconsider what we should store in container labels.
• [P1] Create permanent network namespace. (Issue: #43, PR: #54)
• [P2] Add unit test for image. With new containerd client, it's much easier to add unit test for image part. (Issue: #36)
• [P3] Check image config and top level snapshot existence when list/Inspect image. (Issue: containerd/containerd#1514, #303)
• [P1] Default sandbox container resource limit. (PR: #92)
• [P3] Add truncindex for image and container id. (PR: #235)
• [P2] Reliable containerd event handling. Requeue event on error. (#628)
• [P2] Checkpoint versioning. (Metadata and status are all versioned now)
• [P2] Handle recovery from cri-containerd and containerd restart.

Containerd Missing Features

• [P0] V2 Schema 1 image support. (Issue: #35, containerd/containerd#851)
• [P1] Image (content/snapshot) garbage collection.
• [P1] Containerd version. Containerd doesn't report semver because of a bug.

Support Privileged.

Privileged is initially a Docker concept. Docker grants a group of permissions and mounts a group of directories into a privileged pod.

In Kubernetes, we should define our own Privileged kubernetes/kubernetes#44503. For now, we could do whatever docker is doing.

Things to do:

1. Figure out what docker is doing to Privileged container.
2. Do the same in cri-contained.

/cc @heartlock

Support schema 1 manifest.

Currently, containerd only supports schema 2 manifest.

For cri-containerd, not supporting schema 1 means:

1. Users who are still using schema 1 image could not use cri-containerd integration.
2. Some system container images are still using schema 1 manifest, e.g. pause image.

Regarding to 1), it is fine for our alpha release (this quarter), but it should be fixed before beta (probably the end of the year).

For 2), we need to rebuild our schema 1 system container image.

Ref containerd/containerd#851.

Cleanup logs and errors.

We should cleanup logs and errors:

1. We should reconsider log level, level 4 for list/status operations is too spammy;
2. We should make sure we are using %+v for struct, %q for string, and %v for error;
3. We should make sure necessary information is included in the log/error, e.g. container/sandbox/image id.

/cc @mikebrow

Integrate with kubernetes network plugin.

CRI expects the runtime to handle container networking.

As is expected, containerd itself doesn't provide container networking. We should integrate cri-containerd with kubernetes network plugin to initialize the network namespace.

Things need to do:

1. Add flags for network plugin settings and initialize network plugin in cri-containerd. (See https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockershim/docker_service.go#L197-L204)
2. Implement interface needed by network plugin, include namespaceGetter and portMappingGetter. (See https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockershim/docker_service.go#L119-L137)
3. Setup and teardown network when run/stop pod sandbox. (See https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockershim/docker_sandbox.go#L128)

/cc @xlgao-zju

Should we share UTS namespace?

In current implementation, we let container inside a sandbox share the same uts namespace. This is what we want in Kubernetes kubernetes/kubernetes#1615.

However, docker doesn't support UTS namespace sharing. We should figure out:

1. Why they don't share it?
2. Whether there is any problem with UTS sharing?

Add `--version` flag.

Add cri-containerd version.

We should version cri-containerd based on the repo tag and commit number. The version will be used:

1. As part of the name when building a release package or any docker image in the future.
2. As part of --version output.

Refer kubernetes/node-problem-detector#71.

Add unit test back

Many of our unit tests rely on fake containerd services. Unit test does give us some values:

1. Make sure the options we pass to containerd are as expected;
2. Make sure the workflow is as expected especially for error handling.

However, since containerd is still in alpha, they are still getting feedbacks and may change api again and again, it's a pain and a bit waste of effort to maintain the fake services up to date.

We decided to temporarily remove the unit tests which use fake services, and rely on the integration test (CRI validation test and node e2e test soon) for now.

We'll add the unit test back after:

1. Containerd api is beta or GA.
2. cri-containerd starts to use containerd client.

The commit before removing unit test is cbd936b.

@mikebrow @yujuhong
/cc @stevvooe @crosbymichael

Upgrade containerd api.

Containerd is still making changes to API, e.g. containerd/containerd#1047, containerd/containerd#1062.

We should upgrade once more after they finalize the api.

Add support for annotations and labels

generateSandboxContainerSpec() is currently filtering all annotations received through the CRI api vs copying them to the runtime spec. Consider if some of the annotations should be filtered or if all should be copied over. For example:

annotations := config.GetAnnotations()
for key, value := range annotations {
 		g.AddAnnotation(key, value)
}

Similarly, labels received over CRI are not being processed / handed over to containerd.

Consider the patterns enabled/being used for kubernetes when connected to dockershim and CRI-O, should we be unique here?