Quarkus Amazon CloudWatch Logging extension
License: Apache License 2.0
This extension allows you to send your application logs to AWS CloudWatch.
Check out the documentation on how to use the extension.
Thanks goes to these wonderful people (emoji key):
 Heiko W. Rupp 💻 🚧 |
 Bennet Schulz 💻 🚧 |
 Gwenneg Lepage 💻 🚧 |
This project follows the all-contributors specification. Contributions of any kind welcome!
![allcontributors[bot] avatar](https://avatars.githubusercontent.com/in/23186?v=4 "allcontributors[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
The % character seems to be handled differently by the default Quarkus standard output logging and this library. This line:
logger.info("Progress: 10%")leads to the following output:
2023-08-11 13:19:21,924 INFO [REDACTED] (Quarkus Main Thread) Progress: 10%
java.util.logging.ErrorManager: 1: Nested handler publication threw an exception
java.util.UnknownFormatConversionException: Conversion = '%'
at java.base/java.util.Formatter.parse(Formatter.java:2826)
at java.base/java.util.Formatter.format(Formatter.java:2763)
at java.base/java.util.Formatter.format(Formatter.java:2717)
at java.base/java.lang.String.format(String.java:4155)
at io.quarkiverse.logging.cloudwatch.LoggingCloudWatchHandler.publish(LoggingCloudWatchHandler.java:82)
at org.jboss.logmanager.ExtHandler.publishToNestedHandlers(ExtHandler.java:97)
at io.quarkus.bootstrap.logging.QuarkusDelayedHandler.doPublish(QuarkusDelayedHandler.java:81)
at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:66)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:327)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:334)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:334)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:334)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:334)
at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:334)
at org.jboss.logmanager.Logger.logRaw(Logger.java:750)
at org.slf4j.impl.Slf4jLogger.log(Slf4jLogger.java:574)
at org.slf4j.impl.Slf4jLogger.info(Slf4jLogger.java:288)
[REDACTED]
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:104)
While the line is logged as requested by Quarkus, it is not sent to Cloudwatch. Instead, an exception is printed to standard out.
Considering Log4Shell, I'd probably prefer if the string I pass to the logging functions was printed without any processing. However, if some sort of conversion is enabled, it would be nice if it was documented and the error message and stack trace should also be sent to Cloudwatch, so people relying on that log know that there was some logging that failed.
https://quarkus.io/extensions/io.quarkiverse.logging.cloudwatch/quarkus-logging-cloudwatch/ shows the parent pom's description, because the logging-cloudwatch pom does not have a description.
Hi Team,
We are running a quarkus app with quarkus-logging-cloudwatch extention to push logs to cloudwatch. we are running the app with default quarkus-logging-cloudwatch extention configs,
https://github.com/quarkiverse/quarkus-logging-cloudwatch
quarkus-logging-cloudwatch extention version: 5.4.0
our quarkus platform version: 2.13.0.Final
After some time the app stops pushing logs to cloudwatch, we see these errors in the app logs, would you let us how to fix this issue,
2024-06-03T08:43:51.133976517Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1773,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << HTTP/1.1 400 Bad Request","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","pro
cessId":1}
2024-06-03T08:43:51.133988847Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1774,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << x-amzn-RequestId: 125decf5-51f6-46cd-be57-af4919a3abff","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","proce
ssName":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.133991568Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1775,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << Content-Type: application/x-amz-json-1.1","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quark
us-run.jar","processId":1}
2024-06-03T08:43:51.133998188Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1776,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << Content-Length: 107","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","processI
d":1}
2024-06-03T08:43:51.134000398Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1777,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << Date: Mon, 03 Jun 2024 08:43:50 GMT","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-ru
n.jar","processId":1}
2024-06-03T08:43:51.134064129Z stdout F {"timestamp":"2024-06-03T08:43:51.133Z","sequence":1778,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.headers","level":"
DEBUG","message":"http-outgoing-0 << Connection: close","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","processId"
:1}
2024-06-03T08:43:51.1345481Z stdout F {"timestamp":"2024-06-03T08:43:51.134Z","sequence":1779,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"software.amazon.awssdk.requestId","level":"DEBUG",
"message":"Received failed response: 400, Request ID: 125decf5-51f6-46cd-be57-af4919a3abff, Extended Request ID: not available","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"r
est-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.134570519Z stdout F {"timestamp":"2024-06-03T08:43:51.134Z","sequence":1780,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"software.amazon.awssdk.request","level":"DEBUG",
"message":"Received failed response: 400, Request ID: 125decf5-51f6-46cd-be57-af4919a3abff, Extended Request ID: not available","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"r
est-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.135589531Z stdout F {"timestamp":"2024-06-03T08:43:51.135Z","sequence":1781,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.wire","level":"DEB
UG","message":"http-outgoing-0 << "{"__type":"InvalidParameterException","message":"Upload too large: 1064089 bytes exceeds limit of 1048576"}"","threadName":"pool-12-thread-1","threadId":32,"mdc
":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.135661173Z stdout F {"timestamp":"2024-06-03T08:43:51.135Z","sequence":1782,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.impl.conn.DefaultM
anagedHttpClientConnection","level":"DEBUG","message":"http-outgoing-0: Close connection","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processN
ame":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.135938688Z stdout F {"timestamp":"2024-06-03T08:43:51.135Z","sequence":1783,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"software.amazon.awssdk.http.apache.internal.net.
SdkSslSocket","level":"DEBUG","message":"closing logs.us-east-1.amazonaws.com/3.236.94.242:443","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","pr
ocessName":"quarkus-run.jar","processId":1}
2024-06-03T08:43:51.136365877Z stdout F {"timestamp":"2024-06-03T08:43:51.136Z","sequence":1784,"loggerClassName":"org.apache.commons.logging.impl.JBossLog","loggerName":"org.apache.http.impl.execchain.Mai
nClientExec","level":"DEBUG","message":"Connection discarded","threadName":"pool-12-thread-1","threadId":32,"mdc":{},"ndc":"","hostName":"rest-api-qa-86b7799858-68skq","processName":"quarkus-run.jar","proc
essId":1}
i have attached entire logfile.
Regards
Shekhar
Quarkus version: 2.14.0-Final
quarkus-logging-cloudwatch version: 5.0.0
I have the following error while building native image: mvn install -Dnative -DskipTests -Dquarkus.native.container-build=true
[1/7] Initializing... (0,0s @ 0,15GB) Error: Substitution target for io.quarkiverse.logging.cloudwatch.graal.EqualJitterBackoffStrategySubstitutions is not loaded. Use field 'onlyWith' in the 'TargetClass' annotation to make substitution only active when needed. com.oracle.svm.core.util.UserError$UserException: Substitution target for io.quarkiverse.logging.cloudwatch.graal.EqualJitterBackoffStrategySubstitutions is not loaded. Use field 'onlyWith' in the 'TargetClass' annotation to make substitution only active when needed. at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.abort(UserError.java:72) at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.findTargetClass(AnnotationSubstitutionProcessor.java:1011) at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:346) at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:324) at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.createAnnotationSubstitutionProcessor(NativeImageGenerator.java:941) at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:854)
See #549 for details.
Warning: /home/runner/work/quarkus-logging-cloudwatch/quarkus-logging-cloudwatch/quarkus/independent-projects/arc/runtime/src/main/java/io/quarkus/arc/impl/Qualifiers.java:[138,24] isAccessible() in java.lang.reflect.AccessibleObject has been deprecated
Warning: /home/runner/work/quarkus-logging-cloudwatch/quarkus-logging-cloudwatch/quarkus/independent-projects/arc/runtime/src/main/java/io/quarkus/arc/impl/Reflections.java:[124,29] isAccessible() in java.lang.reflect.AccessibleObject has been deprecated
Warning: /home/runner/work/quarkus-logging-cloudwatch/quarkus-logging-cloudwatch/quarkus/independent-projects/arc/runtime/src/main/java/io/quarkus/arc/impl/Reflections.java:[140,23] isAccessible() in java.lang.reflect.AccessibleObject has been deprecated
Warning: /home/runner/work/quarkus-logging-cloudwatch/quarkus-logging-cloudwatch/quarkus/independent-projects/arc/runtime/src/main/java/io/quarkus/arc/impl/Reflections.java:[152,23] isAccessible() in java.lang.reflect.AccessibleObject has been deprecated
Warning: /home/runner/work/quarkus-logging-cloudwatch/quarkus-logging-cloudwatch/quarkus/independent-projects/arc/runtime/src/main/java/io/quarkus/arc/impl/Reflections.java:[164,24] isAccessible() in java.lang.reflect.AccessibleObject has been deprecated
When using application.properties per profile:
application-dev.properties
application-test.properties
application-prod.properties
As indicated here --> https://quarkus.io/guides/config-reference#profile-aware-files
The specific properties required for this plugin are not found, returning:
Caused by: io.quarkus.runtime.configuration.ConfigurationException: One or more configuration errors have prevented the application from starting. The errors are:
instead of hard failing when the extension runs and you don't have the 4 properties set we recommend that
in devmode that you log a warn/error that configuration is missing and actually in the error message have the missing properties.
That makes it so you as the user does not need to go search the documentation for the keys.
Check if quarkus-arc is enough or if the quarkus-core dependency is necessary.
When a pod start it shows:
INFO: Logging to log-group: Optional[policies-fedramp-stage] and log-stream: Optional[policies-engine-74fb84cb9c-q78mj]
It should show:
`INFO: Logging to log-group: "policies-fedramp-stage" and log-stream: "policies-engine-74fb84cb9c-q78mj"
When a failure happens during a payload processing in notifications-backend, it is logged at INFO level with its cause.
In the OpenShift logs, the resulting entry looks like this:
2021-06-01 06:01:42,520 INFO [com.red.clo.not.eve.EventConsumer] (vert.x-eventloop-thread-0) Could not process the payload: [...]: javax.persistence.PersistenceException: org.hibernate.HibernateException: io.vertx.pgclient.PgException: insert or update on table "notification_history" violates foreign key constraint "notification_history_endpoint_id_fkey"
In Kibana, the resulting entry looks like this:
msg=[Could not process the payload: [...], stacktrace=[ org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
The details about the failure cause (SQL constraint violation in the example above) are missing in Kibana.
Using LogRecord#getThrown() in the following code may help fixing this issue:
Edit: Actually, we may want to add an additional field which would contain LogRecord#getThrown() instead of changing the code highlighted above. 😃
Suggestion: If that logger is enabled it may makes sense to log (INFO) that the cloudwatch logger is enabled and mention the log-group and log-stream that is being used.
See https://github.com/quarkiverse/quarkus-logging-cloudwatch/runs/7856793631?check_suite_focus=true
@gsmet Do you have any idea why the releasing is blocked by unstashed changes (see the end of the log)? I'm quite new at releasing quarkiverse extensions, I might have missed something. I wonder if the issue is related to the fact that I "squashed and merged" a PR earlier today instead of creating a merge commit.
Right now this plugin supports only the static credentials type where access_key and secret_key needs to provisioned in application.properties
Can you please add support for other credentials provider like quarkus s3 plugin?
quarkus.s3.aws.credentials.type=default
https://quarkiverse.github.io/quarkiverse-docs/quarkus-amazon-services/dev/amazon-s3.html#_configuring_s3_clients
I would like to document it to be more clear which logging levels are supported: https://en.wikipedia.org/wiki/Java_logging_framework#Feature_comparison
Is it possible to customize the logs and integrate for example a customer id in the log message?
I would like to be able to distinguish the logs from the different Quarkus Apps in AWS cloudwatch using metric filters:
Like this:
{
"customerId": "sdf9-sfwe9w-waef",
"@timestamp": "2023-10-23T07:09:18.000Z",
"log.level": "INFO",
"message": "heartbeat",
"process.thread.name": "executor-thread-1495",
"log.logger": "com.test.app.HeartbeatScheduler"
}
{
"customerId": "12-4332-4555",
"@timestamp": "2023-10-23T07:09:18.000Z",
"log.level": "ERROR",
"message": "lorem ipsum blabla",
"process.thread.name": "executor-thread-1495",
"log.logger": "com.test.app.HeartbeatScheduler"
}Log entries currently contain a service.environment field with a misleading value as it is not the environment name but the Quarkus active profile name. Quarkus will run with the prod profile most of the time regardless of the environment where it is executed.
The extension should only add the service.environment field if the environment name was provided as a configuration value which itself will probably be set from an environment variable at runtime.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.