graylog2 / documentation Goto Github PK
View Code? Open in Web Editor NEWArchived Graylog documentation. See https://docs.graylog.org/ for the new documentation.
Home Page: http://archivedocs.graylog.org/
License: Other
Archived Graylog documentation. See https://docs.graylog.org/ for the new documentation.
Home Page: http://archivedocs.graylog.org/
License: Other
I would be beneficial to have an official description of how to backup the graylog configuration that is persisted into mongodb ...
Hi
When I send a log with default level { 0 or 5} it shows different log level in Graylog-web
Note
we use ruby module -- gelf notifier
Looking at the current documentation for 1.3, I couldn't find anything regarding AND/OR stream rules. We should add some documentation for that.
As a Graylog user would like to see better LDAP documentation. When I search for the word LDAP or Active Directory, I don't see any documentation that describes how to configure it etc.
It might be good to refactor the documentation a bit, and to divide information a bit like
Anyways, Securing Graylog should collect together for example
every field you send and prefix with a _ (underscore) will be treated as an additional field. Allowed characters in field names are any word character (letter, number, underscore), dashes and dots. The verifying regular expression is: ^[w.-]*$
This is not what that regular expression does. Or maybe I'm misunderstanding something? Did you mean \w\.-
? Also this regex does not force the initial underscores
Change http://docs.graylog.org/ to redirect always to the latest documentation.
Now that you have released v1 of your product (congrats for that btw) I would really appreciate some official documentation for writing plugins. I have by now somehow managed to create all standard plugin types, but really would like to know the official way on this ought to be done.
One of my customers really complains that it is currently still very hard to start from scratch - especially for input (yes, there are examples by now) and rest resource (where I could not find an example) it was quite hard to find out.
In addition it would be beneficial to get more information about the built-in transports and codecs.
Or I can offer to write down something (already have some parts) if you tell me what you expect.
Documentation is key to get a community adding plugins so you can better compete with other products like splunk and logstash.
Otherwise just keep up the good work! Looking forward to seeing Graylog evolve! Ronald
When using some proxy in front of Graylog, some configuration might be necessary in order to make the websockets loading metrics to work. By default they use the /a/metrics
route. We need to add documentation for this.
When going to system/nodes/available input types in Graylog there is a Kafka documentation link linking to the input overview http://docs.graylog.org/en/1.0/pages/sending_data.html#json-path-from-http-api-input
I can not find any documentation on Kafka inputs at all. This either needs to be written or the link should be removed.
In the README file of greylog-setup-1.0.0, in "Graylog Server" section following command is missing a "-" switch for XX:+CMSConcurrentMTEnabled.
java -Xms1g -Xmx1g -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Djava.net.preferIPv4Stack=true -Djava.library.path=lib/sigar -jar graylog/graylog.jar server -p run/graylog.pid -f conf/graylog.conf
There are several implementation details missing in the GELF specification, e. g. that using GELF via TCP mandates terminating messages with a null character (\0
) or that compression cannot be used (due to the null character being used for message framing).
See https://www.loggly.com/blog/five-invaluable-techniques-to-improve-regex-performance/ for a good example.
e.g. /pages/installation.html#ubuntu-14-04
Links like http://readthedocs.org/projects/graylog2-docs/downloads/epub/2.0/ point to 404. Documentation for 1.3 is correctly downloadable (which is awesome!)
Instead of just linking the DEB/RPM packages which create the repository configuration on the local machine, the documentation should also list the actual URL to the repository and link to the GnuPG public key being used to sign the packages.
This simplifies setting up these repositories in configuration management products like Puppet, Chef, Ansible, etc.
We should keep all release and upgrade notes in here to make them accessible and to find them easier than digging through old blog articles.
http://docs.graylog.org/en/1.3/pages/installation/graylog_ctl.html#configuration-commands
sudo graylog-ctl set-email-config <smtp server> [–port=<smtp port> –user=<username> –password=<password> –no-tls –no-ssl]
For at least the port and no-tls and no-ssl parameters they seem to require two dashes instead of one.
we only have some minor information about how to raise heap for elasticsearch but not how this could be done for graylog server and why.
The section of the document titled "Create a Stream Rule" reads as:
"Pick the Syslog UDP Input, and click Add stream rule."
[Image for picking the item & add stream rule button.]
Followed by:
"Then, type in the values shown below and hit save.
Then click I’m done!
We have just configured this stream to process in real time all the messages that come in from the security/authorization facility.
Now let’s create the alert."
The issue is that, at the line stating, "Then, type in the values show below and hit save", there is no image or values given.
I'm very much a novice & new to Graylog, but I tried several times to glean the information from text & sections following & didn't find that information to create the initial/test stream rule, in that location...
The Graylog 2.0 documentation requires Elasticsearch 2.1.x or later while it also advises to add script.disable_dynamic: true
to the elasticsearch.yml
file here. I did that using Elasticsearch 2.3.2 and it resulted in error:
Exception in thread "main" java.lang.IllegalArgumentException: script.disable_dynamic is not a supported setting, replace with fine-grained script settings.
Dynamic scripts can be enabled for all languages and all operations by replacing `script.disable_dynamic: false` with `script.inline: on` and `script.indexed: on` in elasticsearch.yml
This issue says that such an error is thrown from ElasticSearch 2.x branch up. ElasticSearch docs on this topic are here. I am not sure if setting the most secure options:
script.inline: false
script.indexed: false
script.file: false
won't hinder Graylog communication to ElasticSearch, so I can't suggest specific values, but definitely that part of Graylog documentation should be updated.
For me it would be enough if the documentation said: "Make sure to add script.disable_dynamic: true
to the elasticsearch.yml
file if you use ElasticSearch <2.0 ..."
Hi,
The only mention of configuring ES in the docker page is that one can pass in an ES_MEMORY field
http://docs.graylog.org/en/1.1/pages/installation/docker.html
I've started a docker Graylog2, and am getting the 'Elasticsearch nodes with too low open file limit' message. I can see there is a page for configuring ES:
http://docs.graylog.org/en/1.1/pages/configuring_es.html
But I'm not sure where elasticsearch.yml or graylog.conf is stored. Docker seems to be a black box. But maybe I'm misunderstanding docker. Must I 'go into' the container somehow, to find these files?
Is the docker install a viable production option? "We strongly recommend to use a dedicated Elasticsearch cluster for your Graylog setup." - (I'm using the one that comes with the docker file, so I presume I'm ok?).
Regards
Daniel
Pls add the note that there is the possibility to use mathematical functions in drools rules.
Bcs there are some users who want this feature but in Graylog itself, so they can get the field values in drools and add a new one with the outcome of the mathematical function.
sudo graylog-ctl https://<public ip>:443/api
Should be
sudo graylog-ctl set-external-ip https://<public ip>:443/api
Also would be nice to specify if it could be a hostname instead of the IP or not.
Network-based inputs have configuration options for setting the receive buffer size which is dependent on the system limits.
We should add an informational section to the documentation which explains how to set those limits, similar to https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html
There are several new Kafka-based inputs in Graylog 1.1.x which need to be documented.
Refs Graylog2/graylog2-server#322 and Graylog2/graylog2-server#1165
The https section in the aws installation has the following command:
sudo graylog-ctl https://<public ip>:443/api
But that command seems not to be valid. It results in: "I don't know that command."
I am running graylog version 2
http://docs.graylog.org/en/2.0/pages/installation/operating_system_packages.html
URL tells us the version is 2.0, but the download .deb link is 1.3, and I test it on Ubuntu 14.04 following those steps, installation fails.
On several pages of the live documentation on docs.graylog.org double hypens "--" in source blocks are being rendered as an en dash "–" instead which looks like just a single hyphen, and when copied and pasted into a terminal is copied over as a single hyphen. The documentation source on Github correctly has double hyphens, rendering the documentation source with Sphinx correctly renders double hyphens, so not a problem with the source or config. It appears this is a known bug, and the live site is likely using an outdated version of smartypants and just needs to update their sphinx install.
Some examples of en dashes instead of double hyphens from http://docs.graylog.org/en/1.3/pages/installation/graylog_ctl.html
Live Site:
sudo graylog-ctl set-email-config <smtp server> [–port=<smtp port> –user=<username> –password=<password> –no-tls –no-ssl]
Should Read:
sudo graylog-ctl set-email-config <smtp server> [--port=<smtp port> --user=<username> --password=<password> --no-tls --no-ssl]
Live Site:
sudo parted -a optimal – /dev/sdb unit \
compact mkpart primary ext3 “1” “-1”
Should Read:
sudo parted -a optimal -- /dev/sdb unit \
compact mkpart primary ext3 "1" "-1"
We should make a list for all the widgets in the documentation.
I am logging this issue from "make latexpdf". This is on Fedora 22 OS with texlive packages all installed.
Chapter 5. [43] [44] Underfull \hbox (badness 10000) in paragraph at lines 2376--2377 []\T1/ptm/m/n/10 Change into the ex-tracted col-lec-tor di-rec-tory and cre-ate a col-lec-tor con-fig-u-ra-tion file in [45 <./coll ector_win_install_1.png>] [46 <./collector_win_install_2.png>] Underfull \hbox (badness 10000) in paragraph at lines 2432--2433 []\T1/ptm/m/n/10 If you choose the op-er-at-ing sys-tem in-stal-la-tion method, the con-fig-u-ra-tion file de-faults to [47 <./collector_win_install_3.png>] Underfull \hbox (badness 10000) in paragraph at lines 2506--2507 []\T1/ptm/m/n/10 Please make sure to es-cape the \T1/pcr/m/n/10 \ \T1/ptm/m/n/1 0 char-ac-ter in Win-dows paths: \T1/pcr/m/n/10 path = "C:\\Program Underfull \hbox (badness 10000) in paragraph at lines 2513--2514 []\T1/ptm/m/n/10 Please make sure to es-cape the \T1/pcr/m/n/10 \ \T1/ptm/m/n/1 0 char-ac-ter in Win-dows paths: \T1/pcr/m/n/10 path = "C:\\Program [48] Overfull \hbox (29.49223pt too wide) in paragraph at lines 2554--2555 []\T1/ptm/m/n/10 A usual glob/wildcard string you know from other tools might b e \T1/pcr/m/n/10 /var/log/apache2/**/*.{access,error}.log\T1/ptm/m/n/10 . Underfull \hbox (badness 10000) in paragraph at lines 2554--2555 \T1/ptm/m/n/10 This means you are in-ter-ested in all log files which names end with \T1/pcr/m/n/10 .access.log \T1/ptm/m/n/10 or Underfull \hbox (badness 10000) in paragraph at lines 2554--2555 \T1/pcr/m/n/10 .error.log \T1/ptm/m/n/10 and which are in a sub di-rec-tory of \T1/pcr/m/n/10 /var/log/apache2\T1/ptm/m/n/10 . Ex-am-ple: [49] [50] [51] [52] ! Package inputenc Error: Unicode char \u8:✔ not set up for use with LaTeX. See the inputenc package documentation for explanation. Type H for immediate help. ... l.2877 \hline\end{tabulary} ?
hello,
I have a problem when I use graylog
Is that when I want to use the data type conversion when there was a problem.
This page has introduced about how to make data type conversion, http://docs.graylog.org/en/2.0/pages/extractors.html
The page wrote:
Grok directly supports converting field values by adding ;datatype at the end of the pattern, like:
len=%{NUMBER:length;int} src=%{IP:srcip} sport=%{NUMBER:srcport} dst=%{IP:dstip} dport=%{NUMBER:dstport}
I write configuration is as follows:
%{NUMBER:byes;int}
but,I found the conversion is not successful !!!
The field is not of type int.
When I try to use the "Generate chart" drawing, tip:
Could not create field graph
Field graphs are only available for numeric fields.
I don't know what to do, I don't know why can't the converted format correct drawing.
I need your help! Please
The installation docs are currently hardcoded to 0.92 versions. The old docs had a macro that was used and we should have that again.
Hi
I like to understand if this project will accept pull request for graylog related diagrams draw by tool like Inkscape,Dia ...
In 'The manual setup' section, am I wrong that there appears to be a big leap between downloading and untarring the graylog.tar file and adding to the conf and running it from there?
If I follow the steps word for word, I have a graylog folder in my home directory, a conf file at /etc/graylog/server/server.conf (the parent folders have nothing in them after /etc) and I'm starting the server by manually running a script in ~/graylog-1.0.1/bin
That's quite a dearth of information information there...and a none functioning setup? I'm more confused than anything that this is the full documentation on installing a well used application.
So then, moving onto the ubuntu 14.04 installation instructions, I run them as instructed and I am left with an apparently installed instance - but no init.d script is in existence and I can do service greylog-server and web start and they do something...but, I have no idea what is supposed to happen from here, the next section is about setting up elasticsearch, and then receiving logs.
I have no graylog-ctl, I have nothing at 127.0.0.1:9000
I'm usually pretty resourceful, but these seem so utterly confusing I have no idea what I'm meant to do. I originally followed this, and had something working, but followed the advice to update the application.
https://www.digitalocean.com/community/tutorials/how-to-install-graylog2-and-centralize-logs-on-ubuntu-14-04
Sadly there are no issues for v1 of graylog, all old versions. So this is even more confusing.
On top of that you say
"It is important to remember that the quick setup app is not meant to create production ready setups. We strongly recommend to use one of the other installation methods for a Graylog setup that is intended to run in production."
So I don't even see the point in that? Sorry to be pessimistic but I've spent hours trying to crawl through this documentation and I'm no where closer to having this installed. It was the same when installing older versions, really obscure dependencies buried in some stack exchange article from 2012.
Hi,
i graylog seems to have issues when running in docker for windws (hyperv vm). following errors occure as soon as i open the webinterface:
graylog_1 | 2016-06-08 14:03:37,111 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:39,113 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:41,235 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local: unknown error (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:43,122 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:45,105 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:47,125 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
graylog_1 | 2016-06-08 14:03:49,109 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://docker.local:12900/system/metrics/multiple on node <8055968d-b53e-46c7-8153-ff9810633a27>, caught exception: docker.local (class java.net.UnknownHostException)
the docker-compose file i am using:
mongo:
image: "mongo:3"
elasticsearch:
image: "elasticsearch:2"
command: "elasticsearch -Des.cluster.name='graylog'"
graylog:
image: graylog2/server:2.0.0-1
environment:
GRAYLOG_PASSWORD_SECRET: somepasswordpepper
GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
GRAYLOG_REST_TRANSPORT_URI: http://docker.local:12900
links:
- mongo
- elasticsearch
ports:
- "9000:9000"
- "12900:12900"
- "5555:5555/udp"
The site in general is running, but it seems that i cannot successfully add inputs.
Was looking through several bug forums, and as far as i could find out, the GRAYLOG_REST_TRANSPORT_URI: http://docker.local:12900
must be a url the browser can access (which it is...)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.