t0xa / gelfj

Graylog Extended Log Format (GELF) implementation in Java and log4j appender without any dependencies.

GELFJ - A GELF Appender for Log4j and a GELF Handler for JDK Logging

Downloading

Add the following dependency section to your pom.xml:

<dependencies>
  ...
  <dependency>
    <groupId>org.graylog2</groupId>
    <artifactId>gelfj</artifactId>
    <version>1.1.16</version>
    <scope>compile</scope>
  </dependency>
  ...
</dependencies>

What is GELFJ

It's very simple GELF implementation in pure Java with the Log4j appender and JDK Logging Handler. It supports chunked messages which allows you to send large log messages (stacktraces, environment variables, additional fields, etc.) to a Graylog2 server.

Following transports are supported:

• TCP
• UDP
• AMQP

How to use GELFJ

Drop the latest JAR into your classpath and configure Log4j to use it.

Log4j appender

GelfAppender will use the log message as a short message and a stacktrace (if exception available) as a long message if "extractStacktrace" is true.

To use GELF Facility as appender in Log4j (XML configuration format):

<appender name="graylog2" class="org.graylog2.log.GelfAppender">
    <param name="graylogHost" value="192.168.0.201"/>
    <param name="originHost" value="my.machine.example.com"/>
    <param name="extractStacktrace" value="true"/>
    <param name="addExtendedInformation" value="true"/>
    <param name="facility" value="gelf-java"/>
    <param name="Threshold" value="INFO"/>
    <param name="additionalFields" value="{'environment': 'DEV', 'application': 'MyAPP'}"/>
</appender>

and then add it as a one of appenders:

<root>
    <priority value="INFO"/>
    <appender-ref ref="graylog2"/>
</root>

Or, in the log4j.properties format:

# Define the graylog2 destination
log4j.appender.graylog2=org.graylog2.log.GelfAppender
log4j.appender.graylog2.graylogHost=graylog2.example.com
log4j.appender.graylog2.originHost=my.machine.example.com
log4j.appender.graylog2.facility=gelf-java
log4j.appender.graylog2.layout=org.apache.log4j.PatternLayout
log4j.appender.graylog2.extractStacktrace=true
log4j.appender.graylog2.addExtendedInformation=true
log4j.appender.graylog2.additionalFields={'environment': 'DEV', 'application': 'MyAPP'}

# Send all INFO logs to graylog2
log4j.rootLogger=INFO, graylog2

AMQP Configuration:

log4j.appender.graylog2=org.graylog2.log.GelfAppender
log4j.appender.graylog2.amqpURI=amqp://amqp.address.com
log4j.appender.graylog2.amqpExchangeName=messages
log4j.appender.graylog2.amqpRoutingKey=gelfudp
log4j.appender.graylog2.amqpMaxRetries=5
log4j.appender.graylog2.facility=test-application
log4j.appender.graylog2.layout=org.apache.log4j.PatternLayout
log4j.appender.graylog2.layout.ConversionPattern=%d{HH:mm:ss,SSS} %-5p [%t] [%c{1}] - %m%n
log4j.appender.graylog2.additionalFields={'environment': 'DEV', 'application': 'MyAPP'}
log4j.appender.graylog2.extractStacktrace=true
log4j.appender.graylog2.addExtendedInformation=true

Options

GelfAppender supports the following options:

• graylogHost: Graylog2 server where it will send the GELF messages; to use TCP instead of UDP, prefix with tcp:
• graylogPort: Port on which the Graylog2 server is listening; default 12201 (optional)
• originHost: Name of the originating host; defaults to the local hostname (optional)
• extractStacktrace (true/false): Add stacktraces to the GELF message; default false (optional)
• addExtendedInformation (true/false): Add extended information like Log4j's NDC/MDC; default false (optional)
• includeLocation (true/false): Include caller file name and line number. Log4j documentation warns that generating caller location information is extremely slow and should be avoided unless execution speed is not an issue; default true (optional)
• facility: Facility which to use in the GELF message; default "gelf-java"
• amqpURI: AMQP URI (required when using AMQP integration)
• amqpExchangeName: AMQP Exchange name - should be the same as setup in graylog2-radio (required when using AMQP integration)
• amqpRoutingKey: AMQP Routing key - should be the same as setup in graylog2-radio (required when using AMQP integration)
• amqpMaxRetries: Retries count; default value 0 (optional)

Automatically populating fields from a JSON message

GelfJsonAppender is also available at org.graylog2.log.GelfJsonAppender. This appender is exactly the same as GelfAppender except that if you give it a parseable JSON string in the log4j message, then it will automatically set additional fields according to that JSON.

For example, given the log4j message "{\"simpleProperty\":\"hello gelf\"}", the GelfJsonAppender will automatically add the additional field simpleProperty to your GELF logging. These fields are in addition to everything else.

The GelfJsonAppender is fail safe. If the given log4j message cannot be parsed as JSON, then the message will still be logged, but there will be no additional fields derived from the message.

Logging Handler

Configured via properties as a standard Handler like

handlers = org.graylog2.logging.GelfHandler

.level = ALL

org.graylog2.logging.GelfHandler.level = ALL
org.graylog2.logging.GelfHandler.graylogHost = syslog.example.com
#org.graylog2.logging.GelfHandler.graylogPort = 12201
#org.graylog2.logging.GelfHandler.extractStacktrace = true
#org.graylog2.logging.GelfHandler.additionalField.0 = foo=bah
#org.graylog2.logging.GelfHandler.additionalField.1 = foo2=bah2
#org.graylog2.logging.GelfHandler.facility = local0

.handlers=org.graylog2.logging.GelfHandler

What is GELF

The Graylog Extended Log Format (GELF) avoids the shortcomings of classic plain syslog:

• Limited to length of 1024 byte
• Not much space for payloads like stacktraces
• Unstructured. You can only build a long message string and define priority, severity etc.

You can get more information here: http://www.graylog2.org/about/gelf

gelfj's Issues

documentation: additionalFields option missing

The very useful option "additionalFields" is not documented on the project start page (https://github.com/t0xa/gelfj)

GelfAppender::close() throws NPE on update if Graylog was not available before

in case Graylog server wasn't available and now the log4j property file changes (with "watch"), the close method will fail on NPE. Here's the fix:

public class MyGelfAppender extends GelfAppender {

    @Override
    public void close() {
        GelfSender x = this.getGelfSender();
        if (x != null) {
            x.close();
        }
    }
}

Gelf chunk message ID is not unique

I'm testing the library in a busy environment where we have around 10 JVMs on a single vmware VM and around 30 VMs total. This generates about 8k messages per second.
The last 4 bytes of the message ID of a gelf chunk is formed by last 4 bytes of the host name, but in a standard environment many hosts would be in the same domain, hence for all hosts the last 4 bytes would be the same for example ".com", ".foo", etc. In my environment these are 300 JVMs setting the same last four bytes the same for their chunks. In that regard I would prefer to avoid using "originHost" parameter to try to workaround the issue.
About the first four bytes, these are formed by the currentTimeMillis and
there are many cases where two different threads would get the same time to generate the message ID. Bottom line is that there is quite a big chance to generate the same message ID for two different messages which will make graylog server to drop the messge(s).
The duplicate chunk metric for last 12 hours on my graylog server is 2141.
I was thinking of introducing some more random way of generating those message IDs. Do anyone have any ideas of what would be a better way (than the current) to do it?

Problem with the how to article

Hello,

I'm a beginner and have trouble with the implementation of graylog2.

My AM-Q doesn't have a pom.xml, so I will use a jar file. I downloaded the jar file gelfj 1.1.7 (http://mvnrepository.com/artifact/org.graylog2/gelfj/1.1.7) and implement the file to my classpath. Then I configurated my graylog2 appender and change the log4j.rootLogger.

But it doesn't work. Didi I forgot something? What is with a graylog handler?

Thank you.

Regards
lnix1988

Host ip is 127.0.1.1 on Debian Systems

Hey t0xa,

I've noticed that the 'host' field is set to 127.0.1.1 (Ubuntu 10.04, see http://stackoverflow.com/questions/2381316/java-inetaddress-getlocalhost-returns-127-0-0-1-how-to-get-real-ip).

My ideas are 1) allow the hostname to be set explicitly in the appender, 2) allow the hostname to be set via the 'host' JSON value of the additional fields. 3) have some boolean flag to flip between InetAddress.getLocalHost().getHostAddress() and InetAddress.getLocalHost().getHostName() (although it looks like getHostName() can do a looking, which shouldn't be done for every message instance); . I think #1 probably makes the most sense.

You can see how gelf4j does it (but it's nothing special): https://github.com/pstehlik/gelf4j/blob/master/src/main/groovy/com/pstehlik/groovy/gelf4j/appender/Gelf4JAppender.groovy

Let me know if you want me to get a working solution + pull request.

Thanks,
-nick

New downloads location?

Since GitHub removed the ability to host arbitrary files for download, I guess the gelfj JARs need a new home. It'd be good to get this info on the wiki and in the README.

Curly Braces around Position parameter String leads to exception in GelfHandler

When using a positional parameter string like the following:
{%1$s}
the GelfHandler throws an uncatched exception.

It probably is a good idea to either make this behaviour configurable (to disable MessageFormat) or
change the ordering of the formatters or to catch the exceptions. Other loggers we use do not show this behaviour .
Is there any workaround for that?

I added a reproducer under e5a4730.

RabbitMQ client should be an optional dependency

In the pom.xml for the RabbitMQ client dependency I believe it can be and should be optional:

    <dependency>
        <groupId>com.rabbitmq</groupId>
        <artifactId>amqp-client</artifactId>
        <version>3.0.4</version>
        <optional>true</optional>
    </dependency>

Notice the <optional>true</optional>.

As long as GelfAMQPSender is not instantiated (which would be the case if you did not use AMQP) I believe everything will be fine (ie no class missing exceptions).

No maven repository!

Instructions on using github to provide a maven repository for your project here:

http://cemerick.com/2010/08/24/hosting-maven-repos-on-github/

add additional setter for additionalField(s)

JBoss AS 7, only set the Handler properties if the Handler has corresponding setters. see custom-handler (https://docs.jboss.org/author/display/AS71/Logging+Configuration).
JBoss/Wildfly does invoke setters multiple times, so multiple 'additionalField' keys are processed correctly.

Threshold parameter not being honored.

I see in the log4j.xml example you can set the threshold, I added this parameter to my log4j.properties file but it doesn't seem to be honoring that. Is this parameter supported?

Have to call stream.finish() with GZipOutputStream

GZipOutputStream doesn't always return good results if you don't call stream.finish() before calling stream.close(). Weird, buggy, but true.

Appender Intermittently Stops Sending Messages

Hello,
I'm running the latest code (as of 31 January 2013), and I've noticed that our hosts using this appender will stop sending logs to graylog2 intermittently. Once this occurs, only a restart of the service fixes the problem. This is particularly bad for our production hosts, where a restart is simply not possible most times. I've not been able to track down the root cause, but I will comment on this issue if I get any more details.

Couldn't send message to graylog

I tried just to send one simple message to graylog2 server.
I got gelfj-1.1.14.jar,json-simple-1.1.1.jar and put them into classpath.

Then tried:

GelfMessage message = new GelfMessage("Short message", "Long message", new Date(), "1");
 message.setHost("origin-host");
 GelfUDPSender gelfSender = new GelfUDPSender(host,port);
 GelfSenderResult gelfSenderResult = gelfSender.sendMessage(message);
 if (!GelfSenderResult.OK.equals(gelfSenderResult)) {
             reportError("Error during sending GELF message. Error code: " + gelfSenderResult.getCode() + ".", gelfSenderResult.getException(), ErrorManager.WRITE_FAILURE);
 }

GelfSenderResult.OK.equals(gelfSenderResult) has true value, but i cannot see my message on graylog server. Is it enough to start before trying to use GELFAppender?

log4j2 support

Is there any chance of supporting log4j2 anytime soon?

GelfAppender does not support JSON message parsing into additional fields

When I log a JSON message to GelfAppender it does not understand JSON. I therefore get a single 'message' field in Graylog with a JSON string.

What I really wanted is for the GelfAppender to 'understand' a JSON message and populate the additional fields for me. There doesn't seem to be a way to do this, although please correct me if there is?

If not, then I think an easy way to do this is to extend GelfAppender to GalfJsonAppender and set the additional fields on append method. If you agree, I am happy to do a PR.

Regards, James

Unit test failure (GelfAppenderTest.handleNDC)

Tried to compile the latest version (as of today) with "mvn package":
(NOTE: "mvn test" results in a success!)

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building gelfj 1.0.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ gelfj ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\Java\gelfj-master\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:compile (default-compile) @ gelfj ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ gelfj ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:testCompile (default-testCompile) @ gelfj ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ gelfj ---
[INFO] Surefire report directory: C:\Java\gelfj-master\target\surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.graylog2.GelfMessageTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.118 sec
Running org.graylog2.log.GelfAppenderTest
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 21.088 sec <<< FAILURE!
Running org.graylog2.logging.GelfHandlerTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec

Results :

Failed tests:   handleNDC(org.graylog2.log.GelfAppenderTest): expected:<Foobar[]> but was:<Foobar[ Foobar]>

Tests run: 17, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22.809s
[INFO] Finished at: Thu Apr 04 12:21:47 BST 2013
[INFO] Final Memory: 6M/120M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project gelfj: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Java\gelfj-master\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Update log4j dependency to 1.2.17

The old version of log4j in the pom.xml is causing problems when sending messages to graylog. Can you update the log4j dependency to version 1.2.17?

Compatibility issue with JDK 1.5

In the pom.xml the project used JDK 1.5, while GelfMessage.java used some methods (Arrays.CopyOf(), Arrays.CopyOfRange()) that are belonged to JDK 1.6 only.

Chunking implmentation seems to be broken

While developing gelfino (a tiny gelf server) I used your gelf client as a bench to test message send, the sequence and count fields are wrong (I get 52 53 as values) I think its related to the way int's are converted to bytes (gelf4r seems to do this right)

Iv used both gelf4r and gelf4j and they work fine,

Thanks
Ronen

Server hostname is cached for the duration of the process requiring restarts to refresh

It looks like both GelfUDPSender and GelfTCPSender caches the hostnames of the graylog server for the duration of the process, which makes it unnecessary hard to fail-over / replace the server without restarting all the clients so they can resolve the hostname again.

Please correct me if I've read the source code wrong.

Is there any reason why this cannot be done upon every reconnect on the GelfTCPSender and periodically (i.e default every 10 min, configurable via a new setting) for the GelfUDPSender?

Add a changelog

It'd be very helpful if there was a changelog with a high-level overview of the changes and any notable upgrade steps for each release. Changelogs are very helpful in knowing when an upgrade is necessary and guarding against any potential mishaps.

GelfMessageFactory (1.1.12) does not handle MDC correctly

GelfMessageFactory is assuming that GelfAppender is used directly and not wrapped in an AsyncAppender. The MDC access breaks if the used with AsyncAppender. The fix is pretty simple:

Replace

    // Get MDC and add a GELF field for each key/value pair
    Map<String, Object> mdc = MDC.getContext();

with:

    // Get MDC and add a GELF field for each key/value pair
    Map<String, Object> mdc = event.getProperties();

If there is an AsyncAppender, Log4J will have populated the mdcCopy within the LoggingEvent already. If there is no AsyncAppender, Log4J will populate the mdcCopy when getProperties() is invoked, and return the MDC.

Timestamp handled differently in different constructors

There are three constructors in the GelfMessage class. In two of them, the Long timestamp is divided by 1000L, but in the other (the middle one, shorter of the two with the timestamp passed in as a Long), the timestamp is not divided by 1000L. Is there a reason for this that I'm missing?

NullPointerExceptions

I'm getting null pointer exceptions that appear to be originating from the graylog appender. Looking a bit into the code it appears the exception could be thrown if bytesList array contains null entries possibly? I have some log messages that dump an entire http response for debugging and I think perhaps the chunking logic doesn't properly handle large messages maybe?

java.lang.NullPointerException
at sun.nio.ch.IOUtil.write(IOUtil.java:147)
at sun.nio.ch.DatagramChannelImpl.write0(DatagramChannelImpl.java:439)
at sun.nio.ch.DatagramChannelImpl.write(DatagramChannelImpl.java:456)
at java.nio.channels.DatagramChannel.write(DatagramChannel.java:435)
at org.graylog2.GelfSender.sendDatagrams(GelfSender.java:41)
at org.graylog2.GelfSender.sendMessage(GelfSender.java:36)
at org.graylog2.log.GelfAppender.append(GelfAppender.java:139)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.debug(Log4jLoggerAdapter.java:281)

Common logging pattern log.warn(null, ex) doesn't work

It is common to log exceptions without accompanying message like this:

    try {
      throw new IOException();
    } catch (Exception ex) {
      LOG.error(null, ex);
      throw new RuntimeException(ex);
    }

but the message is not processed by GELFJ. It is rejected by GelfMessage.isValid due to empty shortMessage field and while there is a test that seems to test this pattern is does not reveal the problem due to overriden behaviour in test sender.

GelfHandler does not map log4j log levels correctly

If I'm using the GelfHandler, all logs are reported as "INFO", because they don't have the proper mapping to Syslog levels.

I made a "quick fix" in 1741c9b (in my forked repo), but my problems with that:

1. It's only a fix for log4j, I'm not familiar with other logging libraries, like slf4j, etc.
2. Unit test not yet provided, I have to figure out how to do it properly.

Please comment if you have some time...

Regards,

Zoltan

Ability to add additional fields via log4j

Hey,

I'm looking for a way to specify some additional fields via the log4j.properties. In my specific use-case, I'd like to add an environment (dev/staging/production) field, as well as an application (ThisService,ThatService) field. Wondering if you had any thoughts on a general solution. Maybe there could be an 'additionalFields' property whose setter could take query-string like values ('environment=dev&application=myApp'). I could modify the appender for my needs, but this feels like a pretty common use-case. Let me know if you have any ideas. Thanks,

-nick

Data encoding is not correct when you send it to stream

when data contains non latin symbols (may be non UTF strings) then it will broken on server side, because server use UTF-8.
You need to use encoding when data in string format are being converted to bytes.
One of that place is GelfMessage->gzipMessage.

  try {
            GZIPOutputStream stream = new GZIPOutputStream(bos);
            stream.write(message.getBytes(<b>"UTF-8"</b>));
...

Patch text:

--- gelfj/src/main/java/org/graylog2/GelfMessage.java (revision 12b0e90)
+++ gelfj/src/main/java/org/graylog2/GelfMessage.java (revision )
@@ -124,7 +124,7 @@

         try {
             GZIPOutputStream stream = new GZIPOutputStream(bos);
-            stream.write(message.getBytes());
+            stream.write(message.getBytes("UTF-8"));
             stream.finish();
             stream.close();
             byte[] zipped = bos.toByteArray();

Please publish sources to maven central

There are no sources in here...
http://repo1.maven.org/maven2/org/graylog2/gelfj/1.1.14/

Could you please publish sources to maven central which will make it much easier to use when developing?

Thanks, James

Add context to gelf message?

Hi, I am trying to figure out the way to add a context to logged gelf message (noob in java), with no lock for now.

I.e. I want to log a message "User Logged In", with additional fields (ip, browser, someInfo) which would be translated in ctxt_ on graylog. Is it possible to do this in Java?

noob question

In the documentation for "How to use GELFJ" it says:

"Grab latest JAR from the downloads section and drop it into your classpath. You will also need com.googlecode.json-simple, json-simple and log4j."

Forgive my ignorance, but what is the difference between "com.googlecode.json-simple" and "json-simple"? When I search for "com.googlecode.json-simple" they look like they're the same thing. If they're different, where do I find them?

Thanks!

Tag releases

It'd be very helpful if each of the releases was tagged so they can be easily checked out of source control and worked with.

Only snapshot available on clojars

It would be awesome to push a Release version into clojars, at the moment
https://clojars.org/org.graylog2/gelfj only a SNAPSHOT version is there.

Our release policy forbids to include SNAPSHOT dependencies..

Duplicate Log Entries

We are seeing duplicate log entries with the same millisecond timestamp but have verified that the application is only outputting one entry to the local log files.

here is our log4j.properties file

# Set the default level to WARN
# Each log appender can override this setting, but this keeps imported libraries from spewing forth junk
# this is the version of the file from /projects/logging
log4j.debug=TRUE
log4j.rootLogger=WARN, LFS, I,D, graylog2

# Log clearwater code at debug
log4j.logger.com.ca=DEBUG, graylog2
log4j.logger.com.clearwateranalytics=DEBUG, graylog2
# Set the request package to log at INFO to save logging, this includes the RequestMethod
log4j.logger.com.ca.wsclient.request=INFO, graylog2

# INFO Logger
log4j.appender.I=org.apache.log4j.RollingFileAppender
log4j.appender.I.layout=org.apache.log4j.PatternLayout
log4j.appender.I.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSSS} %p %t %c %m%n
log4j.appender.I.Threshold=INFO
log4j.appender.I.MaxFileSize=50MB
log4j.appender.I.File=/var/log/tomcat6/donner-ws.info
# how many 50MB files to keep
log4j.appender.I.MaxBackupIndex=5

# DEBUG Logger
log4j.appender.D=org.apache.log4j.RollingFileAppender
log4j.appender.D.layout=org.apache.log4j.PatternLayout
log4j.appender.D.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSSS} %p %t %c %m%n
log4j.appender.D.Threshold=DEBUG
log4j.appender.D.MaxFileSize=50MB
log4j.appender.D.File=/var/log/tomcat6/donner-ws.debug
# how many 50MB files to keep
log4j.appender.D.MaxBackupIndex=2

# Define the graylog2 destination
log4j.appender.graylog2=org.graylog2.log.GelfAppender
log4j.appender.graylog2.graylogHost=${grayloghost}
log4j.appender.graylog2.facility=donner-ws
log4j.appender.graylog2.layout=org.apache.log4j.PatternLayout
log4j.appender.graylog2.extractStacktrace=true
log4j.appender.graylog2.addExtendedInformation=true
log4j.appender.graylog2.Threshold=INFO
log4j.appender.graylog2.additionalFields={'application': 'donner-ws'}

Not sure if this issue has been seen before or if we have a bad configuration but any help would be appreciated.

error in sending GELF messages to logstash

Hi, I am using gelfj module in my application log4j.xml file to sending application logs to logstash which are in turn getting indexed in Elasticsearch.

I have observed that at times when my application starts, I see below error in catalina.out. If I restart my application, GELF initializes and i don't see the error. This is happening randomly. Could you help me in figuring out what could be the issue?

We are using gelf-1.1.7.jar and sending logs to remote logstash server.

Error:
log4j:ERROR Could not send GELF message

GelfMessage#setAdditionalFields() modifies original map

The setAdditionalFields() method stores the original map in this.additionalFields and the addField() method (and others) modifies that map later. This can lead to concurrency problems and breaks if an immutable map is passed as parameter.

    public void setAdditonalFields(Map<String, Object> additonalFields) {
        this.additonalFields = additonalFields;
    }

    public GelfMessage addField(String key, String value) {
        getAdditonalFields().put(key, value);
        return this;
    }

gelfj will not forward messages if originHost is unset or empty

Due to the changes from commit e43ac6a gelfj (or more precisely GelfSender) won't forward messages to the configured Graylog2 server if the originHost property is unset or empty.

The fallback (getting the local host's hostname) only kicks in when GelfMessage#toJson is being executed. Otherwise GelfMessage#isValid will return false due to GelfMessage.host being empty.

Make including caller file name and line number optional

Log4j documentation warns that generating caller location information is
extremely slow and should be avoided unless execution speed is not an
issue

AMQP Appender does not initiate the channel correctly for confirms

The current AMQP sender does not initiate the channels for confirms correctly and causes connections to be instantiated on every logging request.

The channel will be null every time because an exception will be thrown if you try to publish to the channel with out stating that you want confirms

From GelfAMQPSender:

            try {
                // establish the connection the first time
                if (channel == null) {
                    connection = factory.newConnection();
                    channel = connection.createChannel();
                    //MISSING channel.confirmSelect();
                }
                // SNIP property setting
                channel.basicPublish(exchangeName, routingKey, properties, message.toAMQPBuffer().array());
                channel.waitForConfirms();

                return true;
            } catch (Exception e) {
                channel = null;
                tries++;
            }

(I tested this with RabbitMQ 3.2.3. Perhaps other versions don't require stating confirmSelect() otherwise I'm not sure how it could have worked)

Also two features need to be implemented for this to appender to be production ready:

1. We need a configuration to disable waitForConfirms. RabbitMQ and AMQP is already rather reliable and confirms just slow it down. In fact one fix to the above could be to just not to use confirms.
2. We may need to add our own internal buffer queue to improve performance. The reason is that the RabbitMQ clients internal queue is not configurable. The Spring AMQP project has its own log4j appender and does what I'm talking about. Thus I think we need a AsynchGelfSender that use Java concurrent queues and could be used by the TCP and UDP appenders.

Facility not being set to custom value

Hi there,

We're using the gelf appender and the facility is always set to to be gelf-java. I'm having a look through the source code at the moment and I can see that gelf-java is the default set on a GelfMessage. My guess is that it is not being read correctly from the log4j file and therefore not being set.

I'm trying to write a unit test that would prove this either way, but I can't see the code that converts a log4j file into a GelfAppender. If you can help me out I will try to write a failing test.

Thanks!

Closing connection through timeout

Hi.
Can you add support of closing connection after a period of inactivity

Here is the simple example what I want (class GelfTCPSender):

try {
    // reconnect if necessary or 5 min timeout occurred
    if (socket == null || os == null || lastSendTime + 5 * 60 * 1000 < System.currentTimeMillis()) {
        if (socket != null){
            socket.close();
        }
        socket = new Socket(host, port);
        os = socket.getOutputStream();
    }
    os.write(message.toTCPBuffer().array());
    lastSendTime = System.currentTimeMillis();

    return true;
} catch (IOException e) {
    // if an error occours, signal failure
    socket = null;
    return false;
}

I have some problems with firewall and keep-alive sessions.

GelfHandler does not implement Handler interface correctly

During initialization process WebSphere Application Server initializes JUL multiple times. Bug in Websphere causes reinitialization process to leave closed handlers attached to loggers (If I remember correctly from my previous studies, WebSphere's LogManager does try to remove handler, but it does it incorrectly) Specification allows having closed handlers, so getting IBM to fix this will be hard. If GelfHandler implemented close() interface according to specification, problem would not exist.

Current GelfHandler.close closes connection (sender) and sets it to null. This is not a valid implementation, because publish method has logic that reopens connection when it is set to null.

Solution is to add a boolean flag to mark connection as closed.

Handler javadoc:

public abstract void close()
throws SecurityException

Close the Handler and free all associated resources.

The close method will perform a flush and then close the Handler. After close has been called this Handler should no longer be used. Method calls may either be silently ignored or may throw runtime exceptions.

regular jar also contains dependencies

You regular jar..
http://repo1.maven.org/maven2/org/graylog2/gelfj/1.1.14/gelfj-1.1.14.jar

...seems to also contain dependencies just like this one does...
http://repo1.maven.org/maven2/org/graylog2/gelfj/1.1.14/gelfj-1.1.14-jar-with-dependencies.jar

Could you please ensure that the regular jar does not contain dependencies? This will make it easier to to work with maven and allow us to specify different log4j versions at runtime.

Thanks, James

Issue with tcp: and udp: prefixed hosts in log4j appender config

log4j appender has a little bug. when graylog host is specified with protocol (tcp: or udp:), the url is lost.

A case of wrong substring, should be substring(4), not substring(0, 4) in the two places:

if (graylogHost.startsWith("tcp:")) {
String tcpGraylogHost = graylogHost.substring(0, 4);
gelfSender = new GelfTCPSender(tcpGraylogHost, graylogPort);
} else if (graylogHost.startsWith("udp:")) {
String udpGraylogHost = graylogHost.substring(0, 4);
gelfSender = new GelfUDPSender(udpGraylogHost, graylogPort);
} else {
gelfSender = new GelfUDPSender(graylogHost, graylogPort);
}

Chunks are sent incorrectly

When a chunked message is sent to graylog2 server the server will never beable to decode it:

2012-11-01 11:18:45,742 DEBUG: org.graylog2.messagehandlers.gelf.ChunkedGELFClientHandler - Got GELF message chunk: GELFClientChunk:
    Hash: bbb1b89a6c657373  Sequence: 0/4   Arrival: 1351768725 Data size: 4802

As you can see the datasize is larger than the max chunk-size, this because of the automatic concatting of the ByteBuffers in the channel.write()

Will submit a pull request resolving the issue making the request in graylog2 look like:

2012-11-01 11:21:38,017 DEBUG: org.graylog2.messagehandlers.gelf.GELFClientHandlerThread - Received message is chunked. Handling now.
2012-11-01 11:21:38,017 DEBUG: org.graylog2.messagehandlers.gelf.ChunkedGELFClientHandler - Got GELF message chunk: GELFClientChunk:
    Hash: bbb459916c657373  Sequence: 0/4   Arrival: 1351768898 Data size: 1420
2012-11-01 11:21:38,018 DEBUG: org.graylog2.messagehandlers.gelf.GELFClientHandlerThread - Received message is chunked. Handling now.
2012-11-01 11:21:38,018 DEBUG: org.graylog2.messagehandlers.gelf.ChunkedGELFClientHandler - Got GELF message chunk: GELFClientChunk:
    Hash: bbb459916c657373  Sequence: 1/4   Arrival: 1351768898 Data size: 1420
2012-11-01 11:21:38,019 DEBUG: org.graylog2.messagehandlers.gelf.GELFClientHandlerThread - Received message is chunked. Handling now.
2012-11-01 11:21:38,019 DEBUG: org.graylog2.messagehandlers.gelf.ChunkedGELFClientHandler - Got GELF message chunk: GELFClientChunk:
    Hash: bbb459916c657373  Sequence: 2/4   Arrival: 1351768898 Data size: 1420
2012-11-01 11:21:38,019 DEBUG: org.graylog2.messagehandlers.gelf.GELFClientHandlerThread - Received message is chunked. Handling now.
2012-11-01 11:21:38,020 DEBUG: org.graylog2.messagehandlers.gelf.ChunkedGELFClientHandler - Got GELF message chunk: GELFClientChunk:
    Hash: bbb459916c657373  Sequence: 3/4   Arrival: 1351768898 Data size: 504

problem with send log from log4j2 to graylog2 via gelfj

## used dependency and config log4j.xml with

<configuration status="OFF" packages="org.graylog2.log4j2"> <appenders> <Console name="Console" target="SYSTEM_OUT"> <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" /> </Console> <appender name="GrayLog2" class="org.graylog2.log.GelfAppender"> <param name="graylogHost" value="192.168.248.128"/> <param name="graylogPort" value="12201"/> <param name="extractStacktrace" value="true"/> <param name="addExtendedInformation" value="true"/> <param name="facility" value="yourfacility"/> <param name="Threshold" value="WARN"/> <param name="additionalFields" value="{'environment': 'PROD', 'application': yourapplicationname'}"/> </appender> <GELF name="gelfAppender" server="192.168.248.128" port="12201" hostName="appserver01.example.com" additionalFields="foo=bar"/> </appenders> <loggers> <root level="INFO"> <appender-ref ref="GrayLog2"/> <appender-ref ref="gelfAppender"/> <AppenderRef ref="Console" /> </root> </loggers> </configuration>
## now when I run maven project I see this errors and I can't log into graylog2 server.

2016-06-23 05:21:55,658 ERROR Error processing element appender: CLASS_NOT_FOUND
2016-06-23 05:21:56,410 ERROR Unable to locate appender GrayLog2 for logger
2016-06-23 05:21:56,411 ERROR Unable to locate appender gelfAppender for logger

Log4j backwards compatibility

Hey,

Getting ready to try this out. Talking with joemiller on #graylog2, gelfj requires Log4j 1.2.16. Here's his quick fix for backwards compatibility, which could possibly be determined programmatically and incorporated into gelfj:

+++ b/src/main/java/org/graylog2/log/GelfAppender.java
@@ -64,7 +64,9 @@ public class GelfAppender extends AppenderSkeleton {
@OverRide
protected void append(LoggingEvent event) {
Level level = event.getLevel();

•    long timeStamp = event.getTimeStamp();
  

•   // long timeStamp = event.getTimeStamp();
  

•    long timeStamp = System.currentTimeMillis()/1000;
  

release 1.1.5

is it possible that you tag the current dev code as 1.1.5 to have an available version via maven repositories?

numeric additional fields.

I'm trying to use Gelfj to send custom data into Graylog2. I'm using the "addField()" method for this. According to the Gelf Protocol, and additional field can be numeric, but there is only one addField() method and it takes two strings. Should there be additional methods that take numbers as inputs and sends numeric (non-string) values to the DB?

I'm assuming I can use the setAdditonalFields() method, passing it a map full of various objects to pass in different data types? I'll test that theory.