zebrium / ze-kubernetes-collector Goto Github PK
View Code? Open in Web Editor NEWDEPRICATED: Zebrium's log collector for Kubernetes environments
DEPRICATED: Zebrium's log collector for Kubernetes environments
Well, I have some machines running RancherOS, and they don't have /etc/localtime
as shown:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned zebrium/zlog-collector-4hs64 to docker-vmss-1
Warning Failed 2m21s kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/2087d8e3a8d2db5dcc669407a2879c028c83157b0875ff88d7d9feb66626ff82/merged\\\" at \\\"/var/lib/docker/overlay2/2087d8e3a8d2db5dcc669407a2879c028c83157b0875ff88d7d9feb66626ff82/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 2m21s kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/8e48fe511efa2e4ed858e3457865f2a0d60e15101ba6d416da69f08b383fd4c8/merged\\\" at \\\"/var/lib/docker/overlay2/8e48fe511efa2e4ed858e3457865f2a0d60e15101ba6d416da69f08b383fd4c8/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 2m5s kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/1d3cb742ce8715861be2649fab32730508ff658c0829bc7446da53e6b1a697be/merged\\\" at \\\"/var/lib/docker/overlay2/1d3cb742ce8715861be2649fab32730508ff658c0829bc7446da53e6b1a697be/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 98s kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/c99a4e457ebe5fdebb108441220a14676976444624fbe94dc13e663fb24f4af3/merged\\\" at \\\"/var/lib/docker/overlay2/c99a4e457ebe5fdebb108441220a14676976444624fbe94dc13e663fb24f4af3/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Normal Created 52s (x5 over 2m22s) kubelet, docker-vmss-1 Created container zlog-collector
Normal Pulled 52s (x5 over 2m22s) kubelet, docker-vmss-1 Container image "zebrium/zlog-collector:latest" already present on machine
Warning Failed 52s kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/etc/localtime\\\" to rootfs \\\"/var/lib/docker/overlay2/74e0367cc231875c78b83c77a31895dc0cb48a18fe6730a20dc40e2e8d9d0f1f/merged\\\" at \\\"/var/lib/docker/overlay2/74e0367cc231875c78b83c77a31895dc0cb48a18fe6730a20dc40e2e8d9d0f1f/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning BackOff 11s (x9 over 2m20s) kubelet, docker-vmss-1 Back-off restarting failed container
trying to mount directly my timezone also won't work:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned zebrium/zlog-collector-l5qt4 to docker-vmss-1
Warning Failed 14m kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/dbd01800fd4de6382c806dc1c950cc0c99cbfa9085822a61fc8b8c984390bd0d/merged\\\" at \\\"/var/lib/docker/overlay2/dbd01800fd4de6382c806dc1c950cc0c99cbfa9085822a61fc8b8c984390bd0d/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 14m kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/a9906d9d3957a8f653ac97a92173f10a8c3bb5fcd02eb3b2bed8f5cb657a79b6/merged\\\" at \\\"/var/lib/docker/overlay2/a9906d9d3957a8f653ac97a92173f10a8c3bb5fcd02eb3b2bed8f5cb657a79b6/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 14m kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/3a772d8cb92cedde9c6594dbb0f8a2242b42b37f10253087f762a075eb8d19f2/merged\\\" at \\\"/var/lib/docker/overlay2/3a772d8cb92cedde9c6594dbb0f8a2242b42b37f10253087f762a075eb8d19f2/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning Failed 14m kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/ac9e09ba373401fedd924586fd9fd5ae9a47bc93790057f6d7b47dd36635fcad/merged\\\" at \\\"/var/lib/docker/overlay2/ac9e09ba373401fedd924586fd9fd5ae9a47bc93790057f6d7b47dd36635fcad/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Normal Created 13m (x5 over 14m) kubelet, docker-vmss-1 Created container zlog-collector
Normal Pulled 13m (x5 over 14m) kubelet, docker-vmss-1 Container image "zebrium/zlog-collector:latest" already present on machine
Warning Failed 13m kubelet, docker-vmss-1 Error: failed to start container "zlog-collector": Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/usr/share/zoneinfo/Europe/Lisbon\\\" to rootfs \\\"/var/lib/docker/overlay2/00dbf0e393b4c806d0a70f28bfeeed209817a7a7a73ebdecf15e5ec3d666efd9/merged\\\" at \\\"/var/lib/docker/overlay2/00dbf0e393b4c806d0a70f28bfeeed209817a7a7a73ebdecf15e5ec3d666efd9/merged/usr/share/zoneinfo/America/Los_Angeles\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Warning BackOff 4m49s (x45 over 14m) kubelet, docker-vmss-1 Back-off restarting failed container
I think there might be no tzdata
on this OS, so it might be helpful to have a way to select it by env var or something.
My solution ATM was to add an initContainer linking the expected timezone:
initContainers:
- name: tzdata-fixer
image: busybox
imagePullPolicy: IfNotPresent
args:
- ln
- -fs
- /usr/share/zoneinfo/Europe/Lisbon
- /etc/localtime
I tried to use ze-kubernetes-collector and found a issue.
We are using /path/to/docker
as a docker-root-directory.
It is not mounted as default and cannot be added through custom values file.
So log-collector cannot access to our container's stdout/err logs.
So I added volumeMounts
and volumes
in Daemonset spec.
I think it is not just our special case.
It will be nice to be able to configure root directory in values file
Currently the chart always creates the secret and there is no way to disable it.
having a secret.enabled
option would be very useful.
We have an OPA agent running and, as everyone should, we have a requirement that all containers must specify resource limits. This is more than necessary to keep a cluster healthy. While installing the log collector, I got denied:
Error: admission webhook "validating-webhook.openpolicyagent.org" denied the request: container "zlog-collector" requires resource limits
You guys should really set this up. Any idea what good values would be for these limits?
There are several configuration options that require manual edits of the templates to change
- name: EXCLUDE_POD_REGEX
value: ""
- name: TAIL_FROM_HEAD
value: "true"
- name: FLUENTD_LOG_FILE
value: ""
- name: ZE_LOG_LEVEL
value: "info"
- name: NODE_LOGS_PATH
value: "/var/log/*.log,/var/log/syslog,/var/log/messages,/var/log/secure"
- name: EXCLUDE_PATH
value: "[\"/var/log/boot.log\",\"/var/log/lastlog\"]"
- name: FLUSH_INTERVAL
value: "30s"
- name: VERIFY_SSL
value: "false"
A nice to have would be to allow overriding this by putting it in the values.yaml
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.