Use context in the right way. about cri

[cpuguy83]

Would it make sense to create a new context in the defer?
Did you have something else in mind for this?

[Random-Liu]

We actually hit this.
In the test failure https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-cri-containerd-node-e2e-serial/343, kubelet got restarted in the middle of the test, and all on-going contexts got cancelled.

After that, task being started was left in a weird state, because all cleanup operations failed:

time="2018-03-06T22:50:42Z" level=info msg="StartContainer for "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293""
time="2018-03-06T22:50:42Z" level=debug msg="Start writing log file "/var/log/pods/cbf9f844-2190-11e8-b64e-42010a80001f/guaranteedcbf1db79-2190-11e8-a254-42010a80001f/0.log""
time="2018-03-06T22:50:42Z" level=debug msg="Start writing log file "/var/log/pods/cbf9f844-2190-11e8-b64e-42010a80001f/guaranteedcbf1db79-2190-11e8-a254-42010a80001f/0.log""
time="2018-03-06T22:50:42Z" level=debug msg="get snapshot mounts" key=b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293
time="2018-03-06T22:50:42Z" level=info msg="shim /home/containerd/usr/local/bin/containerd-shim started" address="/containerd-shim/k8s.io/b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293/shim.sock" debug=false pid=14405
time="2018-03-06T22:50:42Z" level=debug msg="event published" ns=k8s.io topic="/tasks/create" type=containerd.events.TaskCreate
time="2018-03-06T22:50:42Z" level=error msg="Failed to pipe stdout of container "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293"" error="read /proc/self/fd/18: file already closed"
time="2018-03-06T22:50:42Z" level=error msg="Failed to pipe stderr of container "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293"" error="read /proc/self/fd/24: file already closed"
time="2018-03-06T22:50:42Z" level=error msg="Failed to delete containerd task "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293"" error="context canceled: unknown"
time="2018-03-06T22:50:42Z" level=info msg="Finish piping stdout of container "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293""
time="2018-03-06T22:50:42Z" level=debug msg="Finish redirecting log file "/var/log/pods/cbf9f844-2190-11e8-b64e-42010a80001f/guaranteedcbf1db79-2190-11e8-a254-42010a80001f/0.log""
time="2018-03-06T22:50:42Z" level=info msg="Finish piping stderr of container "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293""                                                                                                                                                                                                    
time="2018-03-06T22:50:42Z" level=debug msg="Finish redirecting log file "/var/log/pods/cbf9f844-2190-11e8-b64e-42010a80001f/guaranteedcbf1db79-2190-11e8-a254-42010a80001f/0.log""
time="2018-03-06T22:50:42Z" level=error msg="StartContainer for "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293" failed" error="failed to start containerd task "b15874cb3bedc785ae0a0186f266cb60d33b473e3819f569bffc80313cda4293": context canceled: unknown"

We should fix this.

Would it make sense to create a new context in the defer?
Did you have something else in mind for this?

@cpuguy83 Sorry for missing your reply... Yeah, as you said, we should create a new context for cleanup in defer.