Comments (17)
It works as suggested, see the updated serializer class.
from quarkus-logging-json.
hello,
I had same problem... my solution was simply to disable all all other fields:
Hope it helps!
from quarkus-logging-json.
yes, I had exactly same constraint!
You can disable everything and you just get a starting {
from quarkus-logging-json.
well, you have several options:
- parse and copy all keys to the generator. Pseudo-code:
Json json = Json.parse(ecsFormatter.format(event));
writeToGenerator(generator, key, json)
def writeToGenerator(generator, key, value) {
if (value instanceof JsonNumber) {
generator.writeNumber(key, value);
} else if (value instanceof JsonString) {
generator.writeString(key, value);
} else if (value instanceof JsonObject) {
for (key in keys(value)) {
writeToGenerator(generator, key, value.get(key))
}
} // TODO: handle arrays too
it could be a bit tricky since it is a recursive algorithm. And it is not very efficient too (since you have to parse JSON and recreate it with another shape).
- reimplement the ECS format directly using the generator. You could use
EcsJsonFormatter
as reference:
in both cases you have to disable all providers (like per my original comment).
- Another option would be to write a custom quarkus extension that uses directly the
EcsFormatter
to format the log. Basically providing another implementation ofExtFormatter
, like this library:
Hope it helps :-)
PS
In my company we started with option 3. and then switched to option 2. few months back and we are quite happy now as we don't have to maintain a custom quarkus extension!
from quarkus-logging-json.
@nadworny you're welcome fellow Zuricher ;)
According to the stacktrace:
java.lang.RuntimeException: com.fasterxml.jackson.core.JsonGenerationException: Can not start an object, expecting field name (context: Object)
at io.quarkiverse.loggingjson.JsonFormatter.format(JsonFormatter.java:37)
there is an unexpected JSON object. What is on line 37? Maybe this line?
generator.writeObject(json);
if so, just use:
generator.writeObjectFieldStart("log"); // will write "log": {
generator.writeObjectFieldStart("origin"); // will write "origin": {
generator.writeObjectFieldStart("file"); // will write "file": {
NB
with the generator you can skip "quoting as string" as the generator is safe
from quarkus-logging-json.
here: https://github.com/elastic/ecs-logging-java
from quarkus-logging-json.
I have released version 1.1.0, with ecs logging format.
quarkus.log.console.json.log-format=ecs
You are free to test that and give some feedback.
from quarkus-logging-json.
I have added ecs.version
in 1.1.1
.
I am not sure about the log.origin
part.
from quarkus-logging-json.
Thanks but what I need is to replace the whole log statement with the ECS formatter. At the moment I'm only able to add the ECS formatted string as a new field which isn't going to be properly ingested by Elastic.
from quarkus-logging-json.
So if I understood you correctly, when I disable all the fields and write a new field using generator, it should wrap it as a main json? I did the following but it still writes it as a new field and on top of that is also escaping the " in the strings. Here is how it looks:
{"":"{\"@timestamp\":\"2021-09-27T07:43:08.945Z\", \"log.level\": \"INFO\", \"message\":\"Live reload total time: 0.636s \", \"ecs.version\": \"1.2.0\",\"process.thread.name\":\"Timer-0\",\"log.logger\":\"io.quarkus.deployment.dev.RuntimeUpdatesProcessor\"}\n"}
Code:
@Override
public void writeTo(JsonGenerator generator, ExtLogRecord event) throws IOException {
var ecsFormatter = new EcsFormatter();
generator.writeStringField("", ecsFormatter.format(event));
}
Could you please provide a code snippet of how you set it up?
from quarkus-logging-json.
I cannot share my code (it is a custom formatter for another "unfied log format" we have internally in SIX).
But of course you can't do that:
generator.writeStringField("", ecsFormatter.format(event));
ecsFormatter is giving you a JSON as a string... so you should first parse it back and then "copy" all the content in the generator.
Otherwise, you could propose a PR in the ECS repository to extend quarkus logging directly.
from quarkus-logging-json.
I think I got you. But how do I "copy" all the content in the generator? Which method do you use? You don't have to paste here the whole code, just the one that is invoking the generator interface.
from quarkus-logging-json.
Thanks a lot fellow Zuricher @dfa1 ;)
This is what I came up with:
- Serializer: https://gist.github.com/nadworny/5fc96555c5f5d24c919fcacc482e820c
- Formatter: https://gist.github.com/nadworny/a80d405b4b4fc50f6e078ca14f451ab4
Everything seems to be working properly except of the following:
public static void serializeOrigin(JsonGenerator generator, String fileName, String methodName, int lineNumber) throws IOException {
StringBuilder builderFileName = new StringBuilder();
StringBuilder builderFunction = new StringBuilder();
JsonUtils.quoteAsString(fileName, builderFileName);
JsonUtils.quoteAsString(methodName, builderFunction);
generator.writeStartObject();
var json = Json.createObjectBuilder()
.add("log", Json.createObjectBuilder()
.add("origin", Json.createObjectBuilder()
.add("file", Json.createObjectBuilder()
.add("name", builderFileName.toString())
.add("line", lineNumber))
.add("function", builderFunction.toString()))).build();
generator.writeObject(json);
generator.writeEndObject();
}
I'm getting this error: https://gist.github.com/nadworny/3fd4a8a79fd0c7e8112cfcc1aabaf786
Any idea what am I doing wrong?
from quarkus-logging-json.
Thanks, I'll try it tomorrow.
You mentioned once:
Otherwise, you could propose a PR in the ECS repository to extend quarkus logging directly.
Which repository did you mean? Which part of my code do you think I could propose?
from quarkus-logging-json.
Awesome, I'll test it this week
from quarkus-logging-json.
hey @SlyngDK, it works, thanks a lot!
The only thing I'm missing but it's not a must atm are the log.origin
and ecs.version
fields.
from quarkus-logging-json.
Take a look at the serializeOrigin https://gist.github.com/nadworny/5fc96555c5f5d24c919fcacc482e820c#file-ecsjsonquarkusserializer-java-L104 but I'm not sure if it's also part of 1.1.1 version? We are using 1.2.0...
from quarkus-logging-json.
Related Issues (20)
- enabling file logging as well HOT 10
- Document changes for new releases HOT 2
- Logging JSON Objects
- Produced ECS log events are invalid HOT 4
- Unable to create new CustomLogHandler HOT 1
- How is this extension different from io.quarkus:quarkus-logging-json? HOT 1
- Ignore some MDC fields HOT 1
- Exclude keys from log HOT 1
- logger does not respect application object mapper HOT 1
- > Null MDC Data when Accessing in public void writeTo(final JsonGenerator generator, final ExtLogRecord event) HOT 1
- GELF format HOT 2
- ECS format - Log event field order does not seem to comply with the ECS standard HOT 1
- The console has custom fields, the log file does not HOT 2
- Import in pom.xml and issue with JSON enablement HOT 2
- io.quarkiverse.jsonlogging - log format is ignored HOT 1
- Adding dynamic additional value? HOT 1
- Updating ECS version from 1.12.1 to 8.x ?
- Malformed JSON with duplicated structured arguments
- Is this the official Quarkus json logging extension? HOT 1
- Please consider releasing a new version HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quarkus-logging-json.