dyrnq / dist Goto Github PK

Error: failed to create containerd container: get apparmor_parser version: exec: "apparmor_parser": executable file not found in $PATH

Jan 13 08:45:23 m223 containerd[664]: time="2023-01-13T08:45:23.953973499+08:00" level=error msg="CreateContainer within sandbox \"5da31a211dd52593b2326553a11c55f216a745b55ae530af931f5dd5e4fa22c8\" for &ContainerMetadata{Name:nginx,Attempt:0,} failed" error="failed to create containerd container: get apparmor_parser version: exec: \"apparmor_parser\": executable file not found in $PATH"

Jan 13 08:46:48 m223 kubelet[720]: E0113 08:46:48.012377     720 remote_runtime.go:442] "CreateContainer in sandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create containerd container: get apparmor_parser version: exec: \"apparmor_parser\": executable file not found in $PATH" podSandboxID="5da31a211dd52593b2326553a11c55f216a745b55ae530af931f5dd5e4fa22c8"
Jan 13 08:46:48 m223 kubelet[720]: E0113 08:46:48.012632     720 kuberuntime_manager.go:862] container &Container{Name:nginx,Image:nginx:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:http,HostPort:0,ContainerPort:80,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maven-cache,ReadOnly:false,MountPath:/usr/share/nginx/html,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:kube-api-access-czkgt,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod test-nginx-65c6d4bf48-n55zn_default(77c2bdb2-9386-45da-8461-0cbc571b47d1): CreateContainerError: failed to create containerd container: get apparmor_parser version: exec: "apparmor_parser": executable file not found in $PATH
Jan 13 08:46:48 m223 kubelet[720]: E0113 08:46:48.012718     720 pod_workers.go:965] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nginx\" with CreateContainerError: \"failed to create containerd container: get apparmor_parser version: exec: \\\"apparmor_parser\\\": executable file not found in $PATH\"" pod="default/test-nginx-65c6d4bf48-n55zn" podUID=77c2bdb2-9386-45da-8461-0cbc571b47d1

pod status "Initialized":"False"; message: "containers with incomplete status: [place-tools]"

Name:             test-nginx-65c6d4bf48-nsn7v
Namespace:        default
Priority:         0
Service Account:  default
Node:             m204/192.168.6.204
Start Time:       Fri, 13 Jan 2023 09:43:45 +0800
Labels:           app=test-nginx
                  pod-template-hash=65c6d4bf48
Annotations:      <none>
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/test-nginx-65c6d4bf48
Containers:
  nginx:
    Container ID:   
    Image:          nginx:latest
    Image ID:       
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /usr/share/nginx/html from maven-cache (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn54s (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  my-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  maven-cache:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  maven-cache
    ReadOnly:   false
  kube-api-access-wn54s:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age    From               Message
  ----     ------       ----   ----               -------
  Normal   Scheduled    3m11s  default-scheduler  Successfully assigned default/test-nginx-65c6d4bf48-nsn7v to m204
  Warning  FailedMount  98s    kubelet            MountVolume.SetUp failed for volume "pvc-5db7ec30-91d7-49a9-92eb-28019bc6f05a" : rpc error: code = Internal desc = mount bind fail. stagingTargetPath:/var/lib/kubelet/plugins/kubernetes.io/csi/csi.cubefs.com/25d95a4317311a75df8c06f904a95e38c2456b6244b295da5feea65607cbd4e9/globalmount, targetPath:/var/lib/kubelet/pods/6c41c418-3b15-467a-afdb-36a9da6c1505/volumes/kubernetes.io~csi/pvc-5db7ec30-91d7-49a9-92eb-28019bc6f05a/mount error:mount --bind /var/lib/kubelet/plugins/kubernetes.io/csi/csi.cubefs.com/25d95a4317311a75df8c06f904a95e38c2456b6244b295da5feea65607cbd4e9/globalmount to /var/lib/kubelet/pods/6c41c418-3b15-467a-afdb-36a9da6c1505/volumes/kubernetes.io~csi/pvc-5db7ec30-91d7-49a9-92eb-28019bc6f05a/mount fail: exit status 32
  Warning  FailedMount  68s    kubelet            Unable to attach or mount volumes: unmounted volumes=[maven-cache], unattached volumes=[kube-api-access-wn54s maven-cache]: timed out waiting for the condition

Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"batch/v1\",\"kind\":\"Job\",\"metadata\":{\"annotations\":{\"helm.sh/hook\":\"pre-install,pre-upgrade\",\"helm.sh/hook-delete-policy\":\"before-hook-creation,hook-succeeded\"},\"labels\":{\"app.kubernetes.io/component\":\"admission-webhook\",\"app.kubernetes.io/instance\":\"ingress-nginx\",\"app.kubernetes.io/managed-by\":\"Helm\",\"app.kubernetes.io/name\":\"ingress-nginx\",\"app.kubernetes.io/version\":\"0.49.0\",\"helm.sh/chart\":\"ingress-nginx-3.36.0\"},\"name\":\"ingress-nginx-admission-create\",\"namespace\":\"ingress-nginx\"},\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"app.kubernetes.io/component\":\"admission-webhook\",\"app.kubernetes.io/instance\":\"ingress-nginx\",\"app.kubernetes.io/managed-by\":\"Helm\",\"app.kubernetes.io/name\":\"ingress-nginx\",\"app.kubernetes.io/version\":\"0.49.0\",\"helm.sh/chart\":\"ingress-nginx-3.36.0\"},\"name\":\"ingress-nginx-admission-create\"},\"spec\":{\"containers\":[{\"args\":[\"create\",\"--host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc\",\"--namespace=$(POD_NAMESPACE)\",\"--secret-name=ingress-nginx-admission\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"docker.io/jettech/kube-webhook-certgen:v1.5.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"create\"}],\"nodeSelector\":{\"kubernetes.io/os\":\"linux\"},\"restartPolicy\":\"OnFailure\",\"securityContext\":{\"runAsNonRoot\":true,\"runAsUser\":2000},\"serviceAccountName\":\"ingress-nginx-admission\"}}}}\n"},"labels":{"app.kubernetes.io/version":"0.49.0","helm.sh/chart":"ingress-nginx-3.36.0"}},"spec":{"template":{"metadata":{"labels":{"app.kubernetes.io/version":"0.49.0","helm.sh/chart":"ingress-nginx-3.36.0"}},"spec":{"nodeSelector":{"kubernetes.io/os":"linux"}}}}}
to:
Resource: "batch/v1, Resource=jobs", GroupVersionKind: "batch/v1, Kind=Job"
Name: "ingress-nginx-admission-create", Namespace: "ingress-nginx"
for: "10-deployments.yaml": Job.batch "ingress-nginx-admission-create" is invalid: spec.template: Invalid value: core.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"ingress-nginx-admission-create", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/component":"admission-webhook", "app.kubernetes.io/instance":"ingress-nginx", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"ingress-nginx", "app.kubernetes.io/version":"0.49.0", "controller-uid":"1d03be1c-6b4c-4401-87d7-3792cff809d0", "helm.sh/chart":"ingress-nginx-3.36.0", "job-name":"ingress-nginx-admission-create"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:core.PodSpec{Volumes:[]core.Volume(nil), InitContainers:[]core.Container(nil), Containers:[]core.Container{core.Container{Name:"create", Image:"docker.io/jettech/kube-webhook-certgen:v1.5.1", Command:[]string(nil), Args:[]string{"create", "--host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc", "--namespace=$(POD_NAMESPACE)", "--secret-name=ingress-nginx-admission"}, WorkingDir:"", Ports:[]core.ContainerPort(nil), EnvFrom:[]core.EnvFromSource(nil), Env:[]core.EnvVar{core.EnvVar{Name:"POD_NAMESPACE", Value:"", ValueFrom:(*core.EnvVarSource)(0xc00c9bc620)}}, Resources:core.ResourceRequirements{Limits:core.ResourceList(nil), Requests:core.ResourceList(nil)}, VolumeMounts:[]core.VolumeMount(nil), VolumeDevices:[]core.VolumeDevice(nil), LivenessProbe:(*core.Probe)(nil), ReadinessProbe:(*core.Probe)(nil), StartupProbe:(*core.Probe)(nil), Lifecycle:(*core.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*core.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]core.EphemeralContainer(nil), RestartPolicy:"OnFailure", TerminationGracePeriodSeconds:(*int64)(0xc01619c538), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"ingress-nginx-admission", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", SecurityContext:(*core.PodSecurityContext)(0xc017027110), ImagePullSecrets:[]core.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*core.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]core.Toleration(nil), HostAliases:[]core.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), PreemptionPolicy:(*core.PreemptionPolicy)(nil), DNSConfig:(*core.PodDNSConfig)(nil), ReadinessGates:[]core.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), Overhead:core.ResourceList(nil), EnableServiceLinks:(*bool)(nil), TopologySpreadConstraints:[]core.TopologySpreadConstraint(nil)}}: field is immutable
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"batch/v1\",\"kind\":\"Job\",\"metadata\":{\"annotations\":{\"helm.sh/hook\":\"post-install,post-upgrade\",\"helm.sh/hook-delete-policy\":\"before-hook-creation,hook-succeeded\"},\"labels\":{\"app.kubernetes.io/component\":\"admission-webhook\",\"app.kubernetes.io/instance\":\"ingress-nginx\",\"app.kubernetes.io/managed-by\":\"Helm\",\"app.kubernetes.io/name\":\"ingress-nginx\",\"app.kubernetes.io/version\":\"0.49.0\",\"helm.sh/chart\":\"ingress-nginx-3.36.0\"},\"name\":\"ingress-nginx-admission-patch\",\"namespace\":\"ingress-nginx\"},\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"app.kubernetes.io/component\":\"admission-webhook\",\"app.kubernetes.io/instance\":\"ingress-nginx\",\"app.kubernetes.io/managed-by\":\"Helm\",\"app.kubernetes.io/name\":\"ingress-nginx\",\"app.kubernetes.io/version\":\"0.49.0\",\"helm.sh/chart\":\"ingress-nginx-3.36.0\"},\"name\":\"ingress-nginx-admission-patch\"},\"spec\":{\"containers\":[{\"args\":[\"patch\",\"--webhook-name=ingress-nginx-admission\",\"--namespace=$(POD_NAMESPACE)\",\"--patch-mutating=false\",\"--secret-name=ingress-nginx-admission\",\"--patch-failure-policy=Fail\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"docker.io/jettech/kube-webhook-certgen:v1.5.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"patch\"}],\"nodeSelector\":{\"kubernetes.io/os\":\"linux\"},\"restartPolicy\":\"OnFailure\",\"securityContext\":{\"runAsNonRoot\":true,\"runAsUser\":2000},\"serviceAccountName\":\"ingress-nginx-admission\"}}}}\n"},"labels":{"app.kubernetes.io/version":"0.49.0","helm.sh/chart":"ingress-nginx-3.36.0"}},"spec":{"template":{"metadata":{"labels":{"app.kubernetes.io/version":"0.49.0","helm.sh/chart":"ingress-nginx-3.36.0"}},"spec":{"nodeSelector":{"kubernetes.io/os":"linux"}}}}}
to:
Resource: "batch/v1, Resource=jobs", GroupVersionKind: "batch/v1, Kind=Job"
Name: "ingress-nginx-admission-patch", Namespace: "ingress-nginx"
for: "10-deployments.yaml": Job.batch "ingress-nginx-admission-patch" is invalid: spec.template: Invalid value: core.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"ingress-nginx-admission-patch", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/component":"admission-webhook", "app.kubernetes.io/instance":"ingress-nginx", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"ingress-nginx", "app.kubernetes.io/version":"0.49.0", "controller-uid":"cc7b86cd-4ffc-4f89-8b98-feaa5ed80c37", "helm.sh/chart":"ingress-nginx-3.36.0", "job-name":"ingress-nginx-admission-patch"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:core.PodSpec{Volumes:[]core.Volume(nil), InitContainers:[]core.Container(nil), Containers:[]core.Container{core.Container{Name:"patch", Image:"docker.io/jettech/kube-webhook-certgen:v1.5.1", Command:[]string(nil), Args:[]string{"patch", "--webhook-name=ingress-nginx-admission", "--namespace=$(POD_NAMESPACE)", "--patch-mutating=false", "--secret-name=ingress-nginx-admission", "--patch-failure-policy=Fail"}, WorkingDir:"", Ports:[]core.ContainerPort(nil), EnvFrom:[]core.EnvFromSource(nil), Env:[]core.EnvVar{core.EnvVar{Name:"POD_NAMESPACE", Value:"", ValueFrom:(*core.EnvVarSource)(0xc004105600)}}, Resources:core.ResourceRequirements{Limits:core.ResourceList(nil), Requests:core.ResourceList(nil)}, VolumeMounts:[]core.VolumeMount(nil), VolumeDevices:[]core.VolumeDevice(nil), LivenessProbe:(*core.Probe)(nil), ReadinessProbe:(*core.Probe)(nil), StartupProbe:(*core.Probe)(nil), Lifecycle:(*core.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*core.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]core.EphemeralContainer(nil), RestartPolicy:"OnFailure", TerminationGracePeriodSeconds:(*int64)(0xc015cd6138), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"ingress-nginx-admission", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", SecurityContext:(*core.PodSecurityContext)(0xc00dfd6d20), ImagePullSecrets:[]core.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*core.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]core.Toleration(nil), HostAliases:[]core.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), PreemptionPolicy:(*core.PreemptionPolicy)(nil), DNSConfig:(*core.PodDNSConfig)(nil), ReadinessGates:[]core.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), Overhead:core.ResourceList(nil), EnableServiceLinks:(*bool)(nil), TopologySpreadConstraints:[]core.TopologySpreadConstraint(nil)}}: field is immutable

Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": failed to call webhook: Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": dial tcp 10.103.239.113:443: connect: connection refused

mysql 04:02:24.46 
mysql 04:02:24.48 Welcome to the Bitnami mysql container
mysql 04:02:24.49 Subscribe to project updates by watching https://github.com/bitnami/containers
mysql 04:02:24.50 Submit issues and feature requests at https://github.com/bitnami/containers/issues
mysql 04:02:24.50 
mysql 04:02:24.50 INFO  ==> ** Starting MySQL setup **
mysql 04:02:24.53 INFO  ==> Validating settings in MYSQL_*/MARIADB_* env vars
mysql 04:02:24.54 INFO  ==> Initializing mysql database
mysql 04:02:24.57 WARN  ==> The mysql configuration file '/opt/bitnami/mysql/conf/my.cnf' is not writable. Configurations based on environment variables will not be applied for this file.
mysql 04:02:24.59 INFO  ==> Installing database
mysql 04:02:35.21 INFO  ==> Starting mysql in background
mysql 04:02:37.22 INFO  ==> Configuring authentication
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql 04:02:37.30 INFO  ==> Running mysql_upgrade
mysql 04:02:37.33 INFO  ==> Stopping mysql
mysql 04:02:39.35 INFO  ==> Starting mysql in background
mysql 04:02:51.42 INFO  ==> Loading user's custom files from /docker-entrypoint-startdb.d
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1046 (3D000) at line 24 in file: '/docker-entrypoint-startdb.d/initdb.sql': No database selected
mysql 04:02:51.57 ERROR ==> Failed executing /docker-entrypoint-startdb.d/initdb.sql
mysql 04:02:51.57 INFO  ==> Stopping mysql

git pull
fatal: unable to access 'https://github.com/dyrnq/dist.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated.

kubectl exec -it pod/csi-s3-test-nginx -- bash -c "ls -l /usr/share/nginx/html/s3"
ls: cannot access '/usr/share/nginx/html/s3': Transport endpoint is not connected
command terminated with exit code 2