Comments (4)
Can you paste the log from the fluentd container on STDOUT/STDERR ?
Also, have you pulled the latest version?
Please provide more details on your configuration
from fluentd-kubernetes-sumologic.
Thanks Steve for taking a look. Here is the config.
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: fluentd
labels:
app: fluentd
version: v1
spec:
template:
metadata:
labels:
name: fluentd
spec:
volumes:
- name: pos-files
emptyDir: {}
- name: host-logs
hostPath:
path: /var/log/
- name: docker-logs
hostPath:
path: /var/lib/docker/containers
containers:
- image: sumologic/fluentd-kubernetes-sumologic:latest
resources:
limits:
cpu: 50m
memory: 100Mi
requests:
cpu: 30m
memory: 70Mi
name: fluentd
imagePullPolicy: Always
volumeMounts:
- name: host-logs
mountPath: /mnt/log/
readOnly: true
- name: docker-logs
mountPath: /var/lib/docker/containers
readOnly: true
- name: pos-files
mountPath: /mnt/pos/
env:
- name: COLLECTOR_URL
valueFrom:
secretKeyRef:
name: sumologic
key: collector-url
---
apiVersion: v1
data:
collector-url: {{ sumologic_kubernetes_url|b64encode }}
kind: Secret
metadata:
name: sumologic
namespace: default
type: Opaque
from fluentd-kubernetes-sumologic.
@stevezau I'm having the same issue - 1000s of logs as above. I've disabled kube-apiserver log to prevent running out of space in sumo.
It's been deployed yesterday to kubernetes 1.5.2 based cluster, off the master of this repo.
Here's the manifest I used:
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: fluentd
labels:
app: fluentd
version: v1
spec:
template:
metadata:
labels:
name: fluentd
spec:
volumes:
- name: pos-files
emptyDir: {}
- name: host-logs
hostPath:
path: /var/log/
- name: docker-logs
hostPath:
path: /var/lib/docker
containers:
- image: sumologic/fluentd-kubernetes-sumologic:latest
name: fluentd
imagePullPolicy: Always
volumeMounts:
- name: host-logs
mountPath: /mnt/log/
readOnly: true
- name: docker-logs
mountPath: /var/lib/docker/
readOnly: true
- name: pos-files
mountPath: /mnt/pos/
env:
- name: COLLECTOR_URL
valueFrom:
secretKeyRef:
name: sumologic
key: collector-url
This is an example of logs from fluentd:
{"log":"2017-02-15 21:04:55 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-controller-manager\" time=1487192691 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"6\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-75-85 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-controller-manager\", :category=\u003e\"kubernetes/kube-controller-manager\"}}\n","stream":"stdout","time":"2017-02-15T21:04:55.371179902Z"}
{"log":"2017-02-15 21:04:55 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-scheduler\" time=1487192691 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"5\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-32-29 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-scheduler\", :category=\u003e\"kubernetes/kube-scheduler\"}}\n","stream":"stdout","time":"2017-02-15T21:04:55.372112493Z"}
{"log":"2017-02-15 21:04:57 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-controller-manager\" time=1487192694 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"6\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-75-85 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-controller-manager\", :category=\u003e\"kubernetes/kube-controller-manager\"}}\n","stream":"stdout","time":"2017-02-15T21:04:57.37121862Z"}
{"log":"2017-02-15 21:05:00 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-scheduler\" time=1487192695 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"5\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-32-29 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-scheduler\", :category=\u003e\"kubernetes/kube-scheduler\"}}\n","stream":"stdout","time":"2017-02-15T21:05:00.371432512Z"}
{"log":"2017-02-15 21:05:01 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-controller-manager\" time=1487192697 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"6\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-75-85 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-controller-manager\", :category=\u003e\"kubernetes/kube-controller-manager\"}}\n","stream":"stdout","time":"2017-02-15T21:05:01.371181377Z"}
{"log":"2017-02-15 21:05:02 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-scheduler\" time=1487192699 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"5\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-32-29 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-scheduler\", :category=\u003e\"kubernetes/kube-scheduler\"}}\n","stream":"stdout","time":"2017-02-15T21:05:02.371416978Z"}
{"log":"2017-02-15 21:05:04 +0000 [warn]: dump an error event: error_class=KeyError error=\"key not found: \\\"kubernetes\\\"\" tag=\"kube-scheduler\" time=1487192701 record={\"severity\"=\u003e\"I\", \"pid\"=\u003e\"5\", \"source\"=\u003e\"leaderelection.go:247\", \"message\"=\u003e\"lock is held by ip-172-20-32-29 and has not yet expired\", \"_sumo_metadata\"=\u003e{:log_format=\u003e\"json\", :source=\u003e\"k8s_kube-scheduler\", :category=\u003e\"kubernetes/kube-scheduler\"}}\n","stream":"stdout","time":"2017-02-15T21:05:04.372356299Z"}
from fluentd-kubernetes-sumologic.
Should be fixed in latest release thanks to @fikander
from fluentd-kubernetes-sumologic.
Related Issues (20)
- Release as an independent GEM HOT 3
- Sourcecategories in Sumo appending numbers HOT 12
- Feature request - Migrating .conf files into configMap
- Kubernetes_meta and Kubernetes_meta_reduce break when condition is met.
- Add ability to control which metadata you want to include/exclude when reducing the metadata
- Memory usage growing constantly
- Duplicate Logs because of Redundant SourceCategories HOT 1
- emit transaction failed: error_class=NoMethodError error="undefined method `+' HOT 4
- fluentd daemonset unable to read log files under OCI OKE
- Sumologic sourceCategory not respecting sumologic.com/sourceCategory annotation HOT 2
- RBAC fluentd daemonset unable to read log files under OCI OKE
- ls: cannot access '/fluentd/etc/fluent.file.conf': No such file or directory HOT 4
- Allow ability to set annotations in the daemonSet in Helm Chart HOT 3
- Windows support HOT 1
- Worker 0 continuously restarts HOT 1
- Question: Can I reduce Kubernetes metadata but leave labels? HOT 1
- support for CRI-O logs HOT 1
- Docker build failing for v2.4.2 HOT 1
- POD metadata is not extracted to sourceName
- Memory consumption grows to max and never goes down HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluentd-kubernetes-sumologic.