pukkaone / logback-gelf Goto Github PK

View Code? Open in Web Editor NEW

This project forked from t0xa/gelfj

23.0 23.0 11.0 215 KB

Logback appender that sends Graylog Extended Log Format (GELF) messages

License: Other

Java 100.00%

gelf graylog logback

logback-gelf's People

Contributors

Stargazers

Watchers

Forkers

jange mstipanov derrickchilders laurencecao trailrunner-taulia delorien david-sabata cmendible roldoe harbulot ruozhuochen

logback-gelf's Issues

Provide the custom facility to log messages

Thank you for providing the utility to transfer the logs into GrayLog2 server.

Is it possible to sent facility like USER, kern, deamon, mail and etc as referred in below link.
http://en.wikipedia.org/wiki/Syslog

Because when I sent messages, it shows the graylog specific facility.

Thanks,
Satish

Update Logback version

Would it be possible to update the Logback version to 1.1.7? The tests pass locally with the update but no other changes.

GelfSender is not closing streams on exception

logback-gelf/src/main/java/com/github/pukkaone/gelf/protocol/GelfSender.java

Line 17 in 73a6c97

stream.close();

As it is not using try-with-resources or properly handling close on finally() there is a risk of memory leak on "off-heap" memory.
See: http://www.evanjones.ca/java-native-leak-bug.html

GelfAMQPSender channel confirmation issue

Hi.
Using AMQPSender produce IllegalStateException here:
https://github.com/pukkaone/logback-gelf/blob/master/src/main/java/com/github/pukkaone/gelf/protocol/GelfAMQPSender.java#L76
and as a result sends (maxRetries+1) messages. As mentioned in rabbitmq javadoc it happens when calling waitForConfirms() on non-confirm channel: https://www.rabbitmq.com/releases/rabbitmq-java-client/current-javadoc/com/rabbitmq/client/Channel.html#waitForConfirms(). Calling confirmSelect() on channel before waitForConfirms() solves the problem. Contact me if I need to make any changes for merging this fix to fit your repo.

Logback configuration error detected with logback 1.1.7

An unexpected error happened while adding "additionalField" into logback.xml file

It seems that it is a bug in logback 1.1.7, it doesn't produce those errors when I downgrade to logback 1.1.6. If there is no "additionalField" in the logback.xml file, the version 1.1.7 works as well.

How to configure a gelfappender Programmatically?

I mean not using a logback.xml.
jusu using java code.

help plz.

Application fails to start when the appender is not able to communicate with gray log instance

When I try to execute my application using a wrong hostname in the graylog configuration, I got an exception:

15:34:26,847 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
15:34:26,847 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
15:34:26,847 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:{...}/logback.xml]
15:34:27,009 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:34:27,032 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Will scan for changes in [file:{...}/logback.xml]
15:34:27,032 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeTask scanning period to 30 seconds
15:34:27,035 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:34:27,048 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:34:27,060 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:34:27,180 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:34:27,186 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:34:27,208 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@66233253 - No compression will be used
15:34:27,211 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@66233253 - Will use the pattern /var/log/myapplication.%d{yyyy-MM-dd}.log for the active file
15:34:27,217 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/var/log/myapplication.%d{yyyy-MM-dd}.log'.
15:34:27,217 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
15:34:27,221 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Oct 16 15:31:51 BRST 2017
15:34:27,223 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:34:27,226 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /var/log/myapplication.log
15:34:27,226 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/var/log/myapplication.log]
15:34:27,228 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:34:27,228 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE_DEBUG]
15:34:27,230 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1286783232 - No compression will be used
15:34:27,230 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1286783232 - Will use the pattern /var/log/myapplication.%d{yyyy-MM-dd}.log for the active file
15:34:27,232 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/var/log/myapplication_debug.%d{yyyy-MM-dd}.log'.
15:34:27,232 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
15:34:27,233 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Mon Oct 16 15:32:59 BRST 2017
15:34:27,233 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:34:27,236 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE_DEBUG] - Active log file name: /var/log/myapplication_debug.log
15:34:27,236 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE_DEBUG] - File property is set to [/var/log/myapplication_debug.log]
15:34:27,237 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [com.github.pukkaone.gelf.logback.GelfAppender]
15:34:27,246 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [GRAYLOG]
15:34:35,312 |-ERROR in com.github.pukkaone.gelf.logback.GelfAppender[GRAYLOG] - Unknown Graylog2 hostname [myhost] java.net.UnknownHostException: myhost
	at java.net.UnknownHostException: myhost
	at 	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at 	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
	at 	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
	at 	at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
	at 	at java.net.InetAddress.getAllByName(InetAddress.java:1192)
	at 	at java.net.InetAddress.getAllByName(InetAddress.java:1126)
	at 	at java.net.InetAddress.getByName(InetAddress.java:1076)
	at 	at com.github.pukkaone.gelf.protocol.GelfUDPSender.<init>(GelfUDPSender.java:37)
	at 	at com.github.pukkaone.gelf.logback.GelfAppender.getGelfUDPSender(GelfAppender.java:214)
	at 	at com.github.pukkaone.gelf.logback.GelfAppender.start(GelfAppender.java:265)
	at 	at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:90)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
	at 	at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
	at 	at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110)
	at 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
	at 	at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
	at 	at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:150)
	at 	at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
	at 	at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)
	at 	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
	at 	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
	at 	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
	at 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
	at 	at mypackage.myclass.<clinit>(MyClass.java:14)
	at 	at 
15:34:35,315 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [mylogger] to DEBUG
15:34:35,315 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [...] to false
15:34:35,316 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE_DEBUG] to Logger[...]
15:34:35,317 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
15:34:35,317 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:34:35,317 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:34:35,317 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [GRAYLOG] to Logger[ROOT]
15:34:35,317 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:34:35,320 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@77b52d12 - Registering current configuration as safe fallback point

Of course I'm expecting some kind of error once the configuration is wrong, however, the application is not starting when this happens. Is this behavior expected? How can I avoid that?

Update AMQP Client version?

We are running into some version issues when using logback-gelf with other libraries that use AMQP (e.g. Spring Rabbit). These libraries are using newer version of the amqp-client (e.g. Spring Rabbit uses amqp-client 3.5.7). Locking the dependency to the newer version causes the GELF AMQP sender to barf.

Any chance to update the AMQP client to something more recent?

How to configure TCP over SSL protocol

Hello,

Since this property "sslTrustAllCertificates" accepts the boolean argument, where and how the java application find the trust and key store, certificates and passwords, to communicate over ssl.

Please explain the configuration of java application to make this work.

~Regards

Add project to Graylog Marketplace

It would be great if you would add this project to the Graylog Marketplace.

RabbitMQ UnexpectedConnectionDriverException leads to the deadlock

Then the AMQP Connection thread handles an UnexpectedConnectionDriverException, a MissedHeartbeatException for example, it tries to log the error message. In the process of logging, this thread takes lock on GelfAppender monitor and starts to wait for confirms on the unresponsive rmq connection without timeout. All others' application threads became BLOCKED when they are trying to log something via GelfAppender.

One of my unresponsive spring apps thread dump parts:

"AMQP Connection 10.3.1.123:5672" #16 prio=5 os_prio=0 cpu=9786.67ms elapsed=511530.01s tid=0x00007fde410a3000 nid=0x17 in Object.wait()  [0x00007fde037fa000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.Object.wait([email protected]/Object.java:328)
	at com.rabbitmq.client.impl.ChannelN.waitForConfirms(ChannelN.java:218)
	- waiting to re-lock in wait() <0x00000000e110d400> (a java.util.Collections$SynchronizedSortedSet)
	at com.rabbitmq.client.impl.ChannelN.waitForConfirms(ChannelN.java:195)
	at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.waitForConfirms(AutorecoveringChannel.java:691)
	at com.github.pukkaone.gelf.protocol.GelfAMQPSender.sendMessage(GelfAMQPSender.java:111)
	at com.github.pukkaone.gelf.logback.GelfAppender.append(GelfAppender.java:298)
	at com.github.pukkaone.gelf.logback.GelfAppender.append(GelfAppender.java:24)
	at ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:82)
	- locked <0x00000000e0fc9610> (a com.github.pukkaone.gelf.logback.GelfAppender)
	at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
	at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270)
	at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)
	at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
	at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
	at ch.qos.logback.classic.Logger.error(Logger.java:538)
	at com.rabbitmq.client.impl.ForgivingExceptionHandler.log(ForgivingExceptionHandler.java:119)
	at com.rabbitmq.client.impl.ForgivingExceptionHandler.handleUnexpectedConnectionDriverException(ForgivingExceptionHandler.java:36)
	at com.rabbitmq.client.impl.AMQConnection.handleFailure(AMQConnection.java:777)
	at com.rabbitmq.client.impl.AMQConnection.access$400(AMQConnection.java:47)
	at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:674)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-11" #134 daemon prio=5 os_prio=0 cpu=10.79ms elapsed=2538.77s tid=0x00007fddd4012000 nid=0x8c waiting for monitor entry  [0x00007fddc57c4000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:63)
	- waiting to lock <0x00000000e0fc9610> (a com.github.pukkaone.gelf.logback.GelfAppender)
	at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
	at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270)
	at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)
	at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
	at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
	at ch.qos.logback.classic.Logger.log(Logger.java:765)
	at org.apache.logging.slf4j.SLF4JLogger.logMessage(SLF4JLogger.java:232)
	at org.jboss.logging.Log4j2Logger.doLog(Log4j2Logger.java:54)
	at org.jboss.logging.Logger.logv(Logger.java:2226)

(one of 24 similar threads)

appender conf:

    <appender name="GELF-AMQP" class="com.github.pukkaone.gelf.logback.GelfAppender">
        <amqpURI>${AMQP_URI}</amqpURI>
        <amqpExchange>${AMQP_KEY}</amqpExchange>
        <amqpRoutingKey>${AMQP_KEY}</amqpRoutingKey>
        <amqpMaxRetries>5</amqpMaxRetries>
        <amqpAutomaticRecovery>true</amqpAutomaticRecovery>
        <originHost>localhost</originHost>
        <facility>gelf-java</facility>
        <additionalField>application=${APP_NAME}</additionalField>
        <mdcIncluded>true</mdcIncluded>
    </appender>

logback-gelf 1.1.11

Upgrade to logback 1.1.7 breaks this library when additionalFields are used - tests missing?

When the 'additionalFiels' element is used in XML configuration, logback 1.1.7 results in the following configuration error:

java.lang.IllegalStateException: Logback configuration error detected: 
ERROR in ch.qos.logback.core.joran.action.NestedBasicPropertyIA - Unexpected aggregationType AS_BASIC_PROPERTY_COLLECTION

The ERROR line is repeated as many times as there are additionalFields.

Basically, logback 1.1.7 is broken: http://jira.qos.ch/browse/LOGBACK-1158, http://jira.qos.ch/browse/LOGBACK-1164 and possible more, all fixed here: qos-ch/logback@2ba396a20e44d2f. The fix will be available in 1.1.8, not yet released.

I guess there is nothing to be done as the only difference between logback-gelf 1.1.8 and 1.1.9 is the upgrade to logback 1.1.7, and people can just force the earlier lib. But, there seems to be a test missing which tries a full configuration with logback, which would catch such a logback regression.