Comments (7)
Thanks for detailed report @edsiper - seems linked to fluent-plugins-nursery/fluent-plugin-cloudwatch-logs#65
from fluentd-kubernetes-daemonset.
Hey @kenden , I have a question:
I've been trying to deploy this daemon to integrate fluentd + CloudWatch with little success.
Even though I've followed these instructions I keep getting this error:
#42
Have you ever seen it before? I'm running out of options here, maybe you can help
from fluentd-kubernetes-daemonset.
@so0k, after a lot of tests it looks like the issue is related to the version of the fluent-plugin-cloudwatch-logs
gem. If you update it to 0.4.0 the encoding error goes away.
I build a docker image using
FROM fluent/fluentd:v0.12.33-debian
USER root
WORKDIR /home/fluent
ENV PATH /home/fluent/.gem/ruby/2.3.0/bin:$PATH
RUN buildDeps="sudo make gcc g++ libc-dev ruby-dev libffi-dev" \
&& apt-get update \
&& apt-get install \
-y --no-install-recommends \
$buildDeps \
&& echo 'gem: --no-document' >> /etc/gemrc \
&& gem install fluent-plugin-secure-forward \
&& gem install fluent-plugin-record-reformer \
&& gem install fluent-plugin-cloudwatch-logs -v 0.4.0 \
&& gem install fluent-plugin-kubernetes_metadata_filter \
&& gem install ffi \
&& gem install fluent-plugin-systemd -v 0.0.8 \
&& SUDO_FORCE_REMOVE=yes \
apt-get purge -y --auto-remove \
-o APT::AutoRemove::RecommendsImportant=false \
$buildDeps \
&& rm -rf /var/lib/apt/lists/* \
&& gem sources --clear-all \
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
# Copy configuration files
COPY ./conf/fluent.conf /fluentd/etc/
COPY ./conf/systemd.conf /fluentd/etc/
COPY ./conf/kubernetes.conf /fluentd/etc/
# Copy plugins
COPY plugins /fluentd/plugins/
# Environment variables
ENV FLUENTD_OPT=""
ENV FLUENTD_CONF="fluent.conf"
# jemalloc is memory optimization only available for td-agent
# td-agent is provided and QA'ed by treasuredata as rpm/deb/.. package
# -> td-agent (stable) vs fluentd (edge)
#ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
# Run Fluentd
CMD exec fluentd -c /fluentd/etc/$FLUENTD_CONF -p /fluentd/plugins $FLUENTD_OPT
Uploaded it to the Docker Hub (https://hub.docker.com/r/alterx/fluentd-cloudwatch/) and it works like a charm. As you can see in the Dockerfile
above, the only thing I did was bump the version of the fluent-plugin-cloudwatch-logs
gem from 0.3.3 to 0.4.0
from fluentd-kubernetes-daemonset.
@alterx Thanks for the feedback. there's a PR to update the version of the plugin:
#44
from fluentd-kubernetes-daemonset.
Perfect, will keep an eye on that PR :D
from fluentd-kubernetes-daemonset.
@alterx The PR was merged, the version of the ruby package fluent-plugin-cloudwatch-logs used is now 0.4.0.
from fluentd-kubernetes-daemonset.
Fixed by #44
from fluentd-kubernetes-daemonset.
Related Issues (20)
- It is not an issue but query HOT 2
- High CVEs found in latest Docker image "fluent/fluentd-kubernetes-daemonset:v1.16.2-debian-logzio-amd64-1.1" HOT 7
- High CVEs in fluent/fluentd-kubernetes-daemonset:v1.16.2-debian-forward-1.0 image HOT 2
- Missing fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-s3-amd64-1.0 in Docker Hub HOT 5
- High CVEs in fluent/fluentd-kubernetes-daemonset:v1.16.2-debian-forward-1.0 image #1467 Issue HOT 5
- 42 CVEs(including Critical and High) found in latest Docker image "fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-opensearch-1.0" HOT 5
- Fluentd not pushing latest logs from a specific namespace in Kubernetes HOT 2
- `free(): invalid pointer` with latest fluent/fluentd-kubernetes-daemonset:v1-debian-forward-arm64 image HOT 7
- Fluentd worker crashing on startup when connecting to Graylog HOT 9
- Support ILM settings for elasticsearch8 in /templates/conf/fluent.conf.erb HOT 2
- fluent/fluentd-kubernetes-daemonset:v1.16.3-debian-elasticsearch8-2.1 systemd plugin segementation fault causing log not send to the destination HOT 3
- Fluentd encountered a memory error (free(): invalid pointer) HOT 5
- Error ruby build docker-image v1.16 debian-syslog HOT 1
- Image vulnerabilities on fluentd-kubernetes-daemonset:v1.16.3-debian-opensearch-2.1 HOT 1
- Kubernetes attribute is missing in cloudwatch logs when fluentd read application logs from files system different from /var/log/containers/ HOT 2
- debian-kafka2 after v1.16.3-debian-kafka2-2.0 and above fails to start kafka output plugin HOT 8
- fluentd collect k8s containerd log do not use the timestamp from realy time HOT 1
- Clarify rule to tag HOT 3
- free(): invalid pointer (segmentation fault with systemd plugin and jemalloc)
- Splunk HEC add support for v1.x
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluentd-kubernetes-daemonset.